1 Situation – Why Deplate?

Deplate is a central part of our production system and we have created an RPM package as well as a set of macros for it.

At A2E.de and FFII.org we used to write web pages in

• a wiki-like text source, converted by an extended version of ‘txt2html’
• a highly programmable lisp source (mlht, see e.g. http://swpat.ffii.org/papers/europarl0309/)

The former was easy to use but very limited in its capabilities, e.g. unable to produce good PDF output.

The latter required too much learning for most people and became bloated over time.

Using wikis directly was also not an alternative, because

• Input is bound to the webform textarea, i.e. users have to be online and immediately submit what they write
• A wiki reinvents the operating system environment. We want to work in the Unix Shell, not at the level of some application server that reproduces the same kind of functionality, only in a less rich and less standardised way. For collaboration, we want to use Subversion, rather than a wiki-specific collaboration system.

We have found Apache Forrest an web-based document managment system to be quite close to what we need.

However, to really be able to tailor Forrest to our more complex needs, we will need to work in the Cocoon framework. While this looks powerful, it may well, in some respects, be less powerful than the shell environment, in which the Makefiles operate which currently are used to generate our object documents. We can not know when we will master the Cocoon/Forrest framework well enough to be confident to migrate content to it on a large scale. We know the limitations of XML/SGML/XSL. We moreover can not be sure that the FO project will produce a full-value replacement for LaTeX.

Deplate works in our shell-based production environment. It produces high-quality PDF files through LaTeX. It has capabilities for extension which are similar to the lisp-based system which we used. Beginners can immediately feel confident to edit the source texts. At least for mainly static websites, Deplate seems to provide the best of our previously separate worlds. It surpasses all the other text converters which we have tried, and it can be conveniently extended with a very powerful language. Combinability with PHP and similar languages (e.g. Embperl, Eruby) for dynamic content is also quite good. It is fairly trivial to set up a wiki-like webform input system which uses Deplate as its backend.

2 Problems and Solutions

To be worked out, see also the Deplate documentation and forum.

2.1 Deplate inserts unwanted linebreaks

When converting long lines to input for LaTeX and HTML, Deplate inserts linebreaks. This is not needed and the results can contravene the intentions of the author. In some cases of cumulated macros execution, several linefeeds are inserted in one place so that a paragraph break is created which will lead not only to unwanted results but sometimes also to failure of LaTeX compilation.

I encountered an example of this in this document when trying to insert a footnote which was supposed to point to a local copy, using a macro for the reference (hyperlink). The phrase ‘lokale Kopie’ was split and a paragraph boundary inserted in the middle.

Also, in the Quote region long lines are split according to Deplate’s rules, and the linebreaks are doubled, as can be observed from the Quote of Borsellino on that same page. Deplate should leave wrapping and typesetting to LaTeX.

My temporary solution is to avoid the Quote region and certain cumulations of macros. Another solution is to patch Deplate so as to disable the invocation of wrap_text in the format_quote function definition in fmt/latex.rb.

The author encourages this approach.

Moreover, the author hints that setting the variable wrapMargin to 0 may also do the job. We’ve integrated this into the default template in our hypertext generation system.

Currently our RPM package does not contain the patch because we haven’t figured out how to patch a Ruby gem.

2.2 LaTeX input

We have written a pre-processor to make up for some of the insufficiencies of current multibyte (CJK) utf8-based text processing. Without this pre-processor, which is invoked by our Makefile system, LaTeX would not compile the deplate output.

The preprocessor also makes up for non-functioning of the noheadings=plain directive and other shortcomings that are appearing. We expect this processor to be no longer ‘simple’ very soon.

2.2.1 heading=plain does not work for LaTeX

The LaTeX output still has numbered headings even if this variable is set

#VAR: headings = plain

in the source text. This can be compensated by simple postprocessing.

2.2.2 LaTeX tables buggy

Deplate has a very intuitive source format for tables which works great for HTML but not only to a limited extent for LaTeX. Bugs, to be documented in more detail, include the following

• text is normally not wrapped, the table gets as wide as the text is long
• this can be remedied by specifying column widths, but then …
  • … multicolumn text is wrapped at the width of one column only

2.3 Reduced Capability of Regions in 0.8.1

In certain self-defined regions, such as BlogRek as defined in etc/lib/blog.dpl, starting from v 0.8.1, the macros of my mod/a2e.rb package were no longer accessible when I switched from Deplate V 0.8 to 0.8.1.

This needs further research.

My temporary solution is to use the Perl Template Toolkit for the macro programming for which I previously used Deplate.

2.4 Macro Programming Language

The documentation seems very sparse on this subject. You have to learn by a lot of trying and even poking at source code.

2.5 Development process

The sources are in a CVS archive. Subversion would be better nowadays.

2.6 Site-wide configuration

I’d like to put configuration files for Deplate into /etc/deplate/, not just ~/.deplate/. As far as I can see from the documentation, this is not possible at the moment.

2.7 Variable settings in deplate.ini

The documentation at http://deplate.sourceforge.net/Variables.html doesn’t seem to give any examples of how the listed variables are actually set. Nor does the file deplate.ini itself. E.g. it says

;    encoding :: The default encoding; dependent 
on the output format (STRING)
;       HTML  :: ISO-8859-1
;       LaTeX :: latin1
;       XML   :: UTF-8

Now is the following OK

encoding = 'utf-8'

or the following

encoding html='utf-8'

I find no hint in the docs.

2.8 Literary Programming

I’d like to write programs as deplate source documents, such that the program code is placed in special regions which are extracted when compiling the document.

Currently this can be done with the Write macro, but I can’t combine this macro with anything else. Once I write text into a file, it is lost in that file and I can only insert the file later with another command. I can’t chose to both write to the file and display the text in a certain form at the same time.

3 Resources

• Official Deplate Site, see especially the Documentation and the Forum
• local tools
  • tratext: translation memory toolkit that parses deplate source text and reports errors
  • local extension modules
  • local templates
  • pth2dpl: script for migrating a2e legacy documents to deplate
  • u2tex: script that converts ideal input to working input for a utf-8 based multilingual LaTeX environment, taking account of weaknesses of processing of utf-8 characters under LaTeX, limitations of the CJK package, etc
  • config.rb: local Ruby macros for Deplate
  • mktdir: generate a documentation node, i.e. a directory with source files, menu files, Makefiles and other document components
  • Deplate @ A2E: macros and templates used here
• Hometravelagency Dictionary lists an English verb “to deplate” which means “to withdraw the right of a travel agency to issue tickets for an airline”. According to some online English-Chinese vocablaries, “to deplate” can mean “除鍍層” i.e. to remove the plating of a metal. This would make sense as an etymology, even if seems to be the inverse of what the Deplate software does.

4 To Do

4.1 Learning, Adaption

see Deplate Macros dossier.

4.2 Future Extension Work

In some cases we can hopefully write clear specifications first and then pay someone to do the programming:

• use better templates and style-sheets, adapt and improve them
• write a separate tool for extraction and reinsertion of translations
• optionally replace database invocations with textfile-based cache

5 Chronology

• 2008-07-09 Hint by Thomas Link on Sourceforge: wrapMarginInner variable may also be needed in order to turn off harmful line-wrapping in LaTeX/HTML object text
• 2008-07-08 Hint by Thomas Link on Sourceforge: wrapMargin variable helps turn off harmful line-wrapping in LaTeX/HTML object text
• 2008-02-18 We have defined and implemented a database input syntax which allows deplate-based texts to be used for writing data to a PostgreSQL database. The a2e extension package has been enhanced to support this.
• 2007-08-29 We are building an extension module to Deplate that should greatly help multilinguisation of documents and abstraction over large bodies of multilingual data, as has been previously achieved by the MLHT package
• 2007-08-03 After studying various serialisation formats such as YAML and JSON, we decide to use the Deplate macro notation for representing data records and to write Deplate macros which both present the data and, when told so by a switch, write them into the database
• 2007-03-05 phm starts playing with FFII.de templates and css, in order to adapt them to production with Deplate
• 2007-02-19 phm publishes Deplate macros and templates used for local web production system
• 2007-02-09 phm creates this page
• dat: 2008-02-21; prs: phm: apache created this directory