\chapter{Renderers}

Renderers allow you to convert a \plasTeX\ document object into viewable
output such as HTML, RTF, or PDF, or simply a data structure format such
as DocBook or tBook.  Since the \plasTeX\ document object gives you
everything that you could possibly want to know about the \LaTeX\ document,
it should, in theory, be possible to generate any type of output from
the \plasTeX\ document object while preserving as much information as the
output format is capable of.  In addition, since the document object is
not affected by the rendering process, you can apply multiple renderers
in sequence so that the \LaTeX\ document only needs to be parsed one time
for all output types.

While it is possible to write a completely custom renderer, a couple of
renderer implementations are included with the \plasTeX\ framework.
While the rendering process in this implementation is fairly simple,
it is also very powerful.  Some of the main features are listed below.
\begin{itemize}
\item ability to generate multiple output files
\item automatic splitting of files is configurable by section level,
    or can be invoked using ad-hoc methods in the
    \member{filenameoverride} property
\item powerful output filename generation utility
\item image generation for portions of the document that cannot be
    easily rendered in a particular output formate (e.g. TikZ pictures in HTML)
\item theming support
\item hooks for post-processing of output files
\item configurable output encodings
\end{itemize}

The API of the renderer itself is very small.  In fact, there are only
a couple of methods that are of real interest to an end user: \method{render}
and \method{cleanup}.  The \method{render} method is the method that starts
the rendering process.  Its only argument is a \plasTeX\ document object.
The \method{cleanup} method is called at the end of the rendering process.
It is passed the document object and a list of all of the files that were
generated.  This method allows you to do post-processing on the output files.
In general, this method will probably only be of interest to someone
writing a subclass of the \class{Renderer} class, so most users of
\plasTeX\ will only use the \method{render} method.  The real work of
the rendering process is handled in the \class{Renderable} class which
is discussed later in this chapter.

The \class{Renderer} class is a subclass of the Python dictionary.
Each key in the renderer corresponds to the name of a node in the
document object.  The value stored under each key is a function.
As each node in the document object is traversed, the renderer is queried
to see if there is a key that matches the name of the node.  If a
key is found, the value at that key (which must be a function) is
called with the node as its only argument.  The return value from this
call must be a string object that contains the rendered output.
Based on the configuration, the renderer will handle all of the file
generation and encoding issues.

If a node is traversed that doesn't correspond to a key in the
renderer dictionary, the default rendering method is called.  The
default rendering method is stored in the \member{default} attribute.
One exception to this rule is for text nodes.  The default
rendering method for text nodes is actually stored in \member{textDefault}.
Again, these attributes simply need to reference any Python function
that returns a string object of the rendered output.  The
default method in both of these attributes is the \function{str}
built-in function.

As mention previously, most of the work of the renderer is actually
done by the \class{Renderable} class.  This is a mixin class\footnote{
A mixin class is simply a class that is merely a collection of methods
that are intended to be included in the namespace of another class.} that is
mixed into the \class{Node} class in the \method{render} method.
It is unmixed at the end of the \method{render} method.  The details
of the \class{Renderable} class are discussed in section \ref{sec:renderable}.

\section{Simple Renderer Example}\label{sec:simple-renderer-ex}

It is possible to write a renderer with just a couple of methods:
\method{default} and \method{textDefault}.
The code below demonstrates how one might create a generic XML
renderer that simply uses the node names as XML tag names.
The text node renderer escapes the <, >, and \& characters.
\begin{verbatim}
import string
from plasTeX.Renderers import Renderer as _Renderer

class Renderer(_Renderer):

    def default(self, node):
        """ Rendering method for all non-text nodes """
        s = []

        # Handle characters like \&, \$, \%, etc.
        if len(node.nodeName) == 1 and node.nodeName not in string.ascii_letters:
            return self.textDefault(node.nodeName)

        # Start tag
        s.append('<%s>' % node.nodeName)

        # See if we have any attributes to render
        if node.hasAttributes():
            s.append('<attributes>')
            for key, value in node.attributes.items():
                # If the key is 'self', don't render it
                # these nodes are the same as the child nodes
                if key == 'self':
                    continue
                s.append('<%s>%s</%s>' % (key, str(value), key))
            s.append('</attributes>')

        # Invoke rendering on child nodes
        s.append(str(node))

        # End tag
        s.append('</%s>' % node.nodeName)

        return '\n'.join(s)

    def textDefault(self, node):
        """ Rendering method for all text nodes """
        return node.replace('&','&amp;').replace('<','&lt;').replace('>','&gt;')
\end{verbatim}

To use the renderer, simply parse a \LaTeX\ document and apply the renderer
using the \method{render} method.
\begin{verbatim}
# Import renderer from previous code sample
from MyRenderer import Renderer

from plasTeX.TeX import TeX

# Instantiate a TeX processor and parse the input text
tex = TeX()
tex.ownerDocument.config['files']['split-level'] = -100
tex.ownerDocument.config['files']['filename'] = 'test.xml'
tex.input(r'''
\documentclass{book}
\begin{document}

Previous paragraph.

\section{My Section}

\begin{center}
Centered text with <, >, and \& charaters.
\end{center}

Next paragraph.

\end{document}
''')
document = tex.parse()

# Render the document
renderer = Renderer()
renderer.render(document)
\end{verbatim}

The output from the renderer, located in \file{test.xml}, looks like the
following.
\begin{verbatim}
<document>
<par>
Previous paragraph.
</par><section>
    <attributes>
        <toc>None</toc>
        <*modifier*>None</*modifier*>
        <title>My Section</title>
    </attributes>
<par>
<center>
 Centered text with &lt;, &gt;, and &amp; charaters.
</center>
</par><par>
Next paragraph.
</par>
</section>
</document>
\end{verbatim}


\subsection{Extending the Simple Renderer}

Now that we have a simple renderer working, it is very simple to extend
it to do more specific operations.  Let's say that the default renderer
is fine for most nodes, but for the \macro{section} node we want to do
something special.  For the section node, we want the title argument
to correspond to the title attribute in the output XML\footnote{This
will only work properly in XML if the content of the title is plain text
since other nodes will generate markup.}.  To do this we need a
method like the following.
\begin{verbatim}
def handle_section(node):
    return '\n\n<%s title="%s">\n%s\n</%s>\n' % \
            (node.nodeName, str(node.attributes['title']),
             str(node), node.nodeName)
\end{verbatim}

Now we simply insert the rendering method into the renderer under the
appropriate key.  Remember that the key in the renderer should match
the name of the node you want to render.  Since the above rendering
method will work for all section types, we'll insert it into the
renderer for each \LaTeX\ sectioning command.
\begin{verbatim}
renderer = Renderer()
renderer['section'] = handle_section
renderer['subsection'] = handle_section
renderer['subsubsection'] = handle_section
renderer['paragraph'] = handle_section
renderer['subparagraph'] = handle_section
renderer.render(document)
\end{verbatim}

Running the same \LaTeX\ document as in the previous example, we now get
this output.
\begin{verbatim}
<document>
<par>
Previous paragraph.
</par>

<section title="My Section">
<par>
<center>
 Centered text with &lt;, &gt;, and &amp; charaters.
</center>
</par><par>
Next paragraph.
</par>
</section>

</document>
\end{verbatim}

Of course, you aren't limited to using just Python methods.  Any function
that accepts a node as an argument can be used.  The
Page Template renderer included with \plasTeX\ is an example
of how to write a renderer that uses a templating language to render
the nodes (see section \ref{sec:pt}).

\subsection{Using a Renderer from the plastex Script}
\label{subsec:renderer-from-script}

In the preceding sections, the simple renderer example was called from
a custom python script. In order to use it through the
main plastex script (described in Chapter~\ref{sec:command-line}), it
needs to be located in some directory
\verb+plasTeX/Renderers/SimpleRenderer+, where \verb+plasTeX+ is the
directory containing the plastex script. This directory must contain a
\verb+__init__.py+ file defining the \var{Renderer} class (with this
name). This directory can also contain a \verb+Themes+ directory in
order to use the theme option described in
Section~\ref{sec:general-options}. Each subdirectory in the
\verb+Themes+ directory is considered as a theme.

Each renderer can define its own configuration options which are loaded
by the plastex script. This is done in a file named \verb+Config.py+
in the renderer directory. This file must define a variable named
\var{config} which is a ConfigManager instance, as described in
Section~\ref{sec:configuration-api}. Inspiration can be drawn from the
file defining the global configuration which is
\verb+plasTeX/Config.py+.

For instance, one could add a file
\verb+plasTeX/Renderers/SimpleRenderer/Config.py+ containing:

\begin{verbatim}
from plasTeX.ConfigManager import *

config = ConfigManager()

section = config.add_section('simplerenderer')

config.add_category('simplerenderer', 'Simple Renderer Options')

section['my-option'] = StringOption(
    """ My option """,
    options='--my-option',
    category='simplerenderer',
    default='',
)
\end{verbatim}

Options values are attached to the document currently rendered. For
instance, in the \var{default} method implemented in
Section~\ref{sec:simple-renderer-ex}, which takes a node argument, one
could access the value of the option defined above as
\verb+node.ownerDocument.config['simplerenderer']['my-option']+.


\section{Renderable Objects\label{sec:renderable}}

The \class{Renderable} class is the real workhorse of the rendering process.
It traverses the document object, looks up the appropriate rendering
methods in the renderer, and generates the output files.  It also
invokes the image generating process when needed for parts of a document
that cannot be rendered in the given output format.

Most of the work of the \class{Renderable} class is done in the
\method{__str__} method.  This is rather convenient since each of
the rendering methods in the renderer are required to return a string
object.  When the \function{str} function is called with a renderable
object as its argument, the document traversal begins for that node.
This traversal includes iterating through each of the node's child nodes, and
looking up and calling the appropriate rendering method in the renderer.
If the child node is configured to generate a new output file, the
file is created and the rendered output is written to it; otherwise,
the rendered output is appended to the rendered output of previous nodes.
Once all of the child nodes have been rendered, the string object containing
that output is returned.  This recursive process continues until the
entire document has been rendered.

There are a few useful things to know about renderable objects such as
how they determine which rendering method to use, when to generate new
files, what the filenames will be, and how to generate images.  These
things are discussed below.


\subsection{Determining the Correct Rendering Method}

Looking up the correct rendering method is quite straight-forward.
If the node is a text node, the \member{textDefault} attribute on
the renderer is used.  If it is not a text node, then the node's name
determines the key name in the renderer.  In most cases, the node's
name is the same name as the \LaTeX\ macro that created it.  If the
macro used some type of modifier argument (i.e. *, +, -), a name
with that modifier applied to it is also searched for first.  For example,
if you used the \environment{tablular*} environment in your \LaTeX\
document, the renderer will look for ``tabular*'' first, then ``tabular''.
This allows you to use different rendering methods for modified and
unmodified macros.  If no rendering method is found, the method
in the renderer's \member{default} attribute is used.


\subsection{Generating Files}

Any node in a document has the ability to generate a new file.
During document traversal, each node is queried for a file name.  If
a non-\var{None} is returned, a new file is created for the content
of that node using the given file name.  The querying for the file name
is simply done by accessing the \member{filename} property of the
node.  This property is added to the node's namespace during the
mixin process.  The default behavior for this property is to only
return file names for sections with a level less than the split-level
given in the configuration (see section \ref{sec:config-files}).
The file names generated by this routine are very flexible.  They can
be statically given names, or names based on the ID and/or title,
or simply generically numbered.  For more information on configuring
file names see section \ref{sec:config-files}.

While the file naming mechanism is very powerful, you may want to name
your files based on some other information.  This is possible through
the \member{filenameoverride} attribute.  If the \member{filenameoverride}
is set, the name returned by that attribute is used as the filename.
The string in \member{filenameoverride} is still processed in the same
way as the filename specifier in the configuration so that you can
use things like the ID or title of the section in the overridden filename.

The string used to specify filenames can also contain directory paths.
This is not terribly useful at the moment since there is no way to
get the relative URLs between two nodes for linking purposes.

If you want to use a filename override, but want to do it conditionally
you can use a Python property to do this.  Just calculate the filename
however you wish, if you decide that you don't want to use that filename
then raise an \exception{AttributeError} exception.  An example of this
is shown below.
\begin{verbatim}
class mymacro{Command):
    args = '[ filename:str ] self'
    @property
    def filenameoverride(self):
        # See if the attributes dictionary has a filename
        if self.attributes['filename'] is not None:
            return self.attributes['filename']
        raise AttributeError, 'filenameoverride'
\end{verbatim}
\note{The filename in the \member{filenameoverride} attribute must
   contain any directory paths as well as a file extension.}


\subsection{Generating Images}

Not all output types that you might render are going to support everything
that \LaTeX\ is capable of.  For example, HTML has no way of representing
TikZ pictures directly, and most output types won't be capable of rendering
\LaTeX's \environment{picture} environment.  In cases like these, you
can let \plasTeX\ generate images of the document node.  Generating
images is done with a subclass of \class{plasTeX.Imagers.Imager}.
The imager is responsible for creating a \LaTeX\ document from the
requested document fragments, compiling the document and converting
each page of the output document into individual images.
See section \ref{sec:config-images} on how to select an imager.
The next section will explain how to generate \emph{vector} images such
as SVG images, which are often a better solution because they scale much better.

To generate an image of a document node, simply access the \member{image}
property during the rendering process.  This property will return
an \class{plasTeX.Imagers.Image} instance.  In most cases, the image
file will not be available until the rendering process is finished
since most renderers will need the generated \LaTeX\ document to be
complete before compiling it and generating the final images.

The example below demonstrates how to generate an image for the
\environment{tikzcd} environment (this is for illustration purposes only,
\plasTeX actually comes with a better handling of this package).
\begin{verbatim}
# Import renderer from first renderer example
from MyRenderer import Renderer

from plasTeX.TeX import TeX

def handle_tikzcd(node):
    return '<div><img src="%s"/></div>' % node.image.url

# Instantiate a TeX processor and parse the input text
tex = TeX()
tex.input(r'''
\documentclass{book}
\begin{document}

Previous paragraph.

\begin{tikzcd}
A \dar \rar & B \dar \\
C \rar & D
\end{tikzcd}

Next paragraph.

\end{document}
''')
document = tex.parse()

# Instantiate the renderer
renderer = Renderer()

# Insert the rendering method into all of the environment that might need it
renderer['tikzcd'] = handle_tikzcd

# Render the document
renderer.render(document)
\end{verbatim}

The rendered output looks like the following, and the image is generated
is located in \file{images/img-0001.png}.
\begin{verbatim}
<document>
<par>
Previous paragraph.
</par><par>
<div><img src="images/img-0001.png"/></div>
</par><par>
Next paragraph.
</par>
</document>
\end{verbatim}

The names of the image files are determined by the document's configuration.
The filename generator is very powerful, and is in fact, the same filename
generator used to create the other output filenames.  For more information
on customizing the image filenames see section \ref{sec:config-images}.

In addition, the image types are customizable as well.  \plasTeX\ uses
the Python Imaging Library (PIL) to do the final cropping and saving of the
image files, so any image format that PIL supports can be used.  The
format that PIL saves the images in is determined by the file extension
in the generated filenames, so you must use a file extension that
PIL recognizes.

It is possible to write your own \class{Imager} subclass if necessary.
See the \class{Imager} API documentation for more information (see
\ref{sec:imager-api}).


\subsection{Generating Vector Images}

If you have a vector imager configured (such as pdf2svg), you
can generate a vector version of the requested image. The nice thing about
vector versions of images is that they can scale infinitely and not loose
resolution.

Generating a vector image is just as easy as generating a bitmap image,
you simply access the \member{vectorImage} property of the node that
you want an image of.  This will return an \class{plasTeX.Imagers.Image}
instance that corresponds to the vector image.

Everything that was described about generating images in the previous
section is also true of vector images with the exception of cropping.
\plasTeX\ does not attempt to crop vector images.  The program that
converts the \LaTeX\ output to a vector image is expected to crop the
image down to the image content.  Depending on the renderer configuration, a
bitmap may be generated to determine the proper height, width and depth of
the vector image.


\subsection{Static Images}

There are some images in a document that don't need to be generated, they
simply need to be copied to the output directory and possibly converted
to an appropriate formate.  This is accomplished with the
\member{imageoverride} attribute.  When the \member{image} property
is accessed, the \member{imageoverride} attribute is checked to see if
an image is already available for that node.  If there is, the image
is copied to the image output directory using a name generated
using the same method as described in the previous section.  The image
is copied to that new filename and converted to the appropriate
image format if needed.  While it would be possible to simply copy the
image over using the same filename, this may cause filename collisions
depending on the directory structure that the original images were
store in.

Below is an example of using \member{imageoverride} for copying
stock icons that are used throughout the document.
\begin{verbatim}
from plasTeX import Command

class dangericon(Command):
    imageoverride = 'danger.gif'

class warningicon(Command):
    imageoverride = 'warning.gif'
\end{verbatim}

It is also possible to make \member{imageoverride} a property
so that the image override can done conditionally.  In the case
where no override is desired in a property implementation, simply
raise an \exception{AttributeError} exception.

\section{Page Template Renderer\label{sec:pt}}

The Page Template (PT) renderer is a renderer for \plasTeX\ document
objects that supports various page template engines such as
\href{http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/ZPT.stx}{Zope
Page Templates (ZPT)}, \href{http://jinja.pocoo.org/}{Jinja2 templates},
\href{http://www.cheetahtemplate.org/}{Cheetah templates},
\href{http://kid-templating.org/}{Kid templates},
\href{http://genshi.edgewall.org/}{Genshi templates},
\href{http://docs.python.org/lib/node40.html}{Python string templates},
as well as plain old \href{http://docs.python.org/lib/typesseq-strings.html}{Python string formatting}.
It is also possible to add support for other template engines.
Note that all template engines except ZPT, Python formats,
and Python string templates must be installed in your Python installation.
They are not included. In particular the Jinja2 template engine must be
installed in order to use the HTML5 renderer (this is already taken care of
if you installed \plasTeX\ in the recommended way).

The Jinja2 engine is used for all of the \plasTeX\ delivered templates
in the HTML5 renderer; however, the other templates work in a very similar way.
The actual ZPT implementation used is SimpleTAL
(\url{http://www.owlfish.com/software/simpleTAL/}).  This implementation
implements almost all of the ZPT API and is very stable.  However, some
changes were made to this package to make it more convenient to use
within \plasTeX.  These changes are discussed in detail in the
ZPT Tutorial (see Appendix~\ref{sec:zpttutorial}).

Since the above template engines can be used to generate any form of
XML or HTML, the PT renderer is in particular a general solution for rendering
XML or HTML from a \plasTeX\ document object.  When switching from one DTD to
another,you simply need to use a different set of templates.

As in all \class{Renderer}-based renderers, each key in the PT renderer
returns a function.  These functions are actually generated when the
template files are parsed by the PT renderer.
As is the case with all rendering methods, the only argument is the node to be
rendered, and the output is a string object containing the rendered
output. In addition to the rendering methods, the \method{textDefault} method
escapes all characters that are special in XML and HTML (i.e. <, >, and \&).

The following sections describe how templates are loaded into the
renderer, how to extend the set of templates with your own, as well
as a theming mechanism that allows you to apply different looks to
output types that are visual (e.g. HTML).

\subsection{Defining and Using Templates}

By default, templates are loaded from the directory where the
renderer module was imported from.  In addition, the templates from
each of the parent renderer class modules are also loaded.  This makes
it very easy to extend a renderer and add just a few new templates
to support the additions that were made.

The template files in the module directories can either contain a single
template, corresponding to a single type of node, or several templates.
In the first case, the basename of the template file is used as the key to
store the template in the renderer, and the extension specifies the template
engine to use. Keep in mind that the names of the keys in the renderer
correspond to the node names in the document object. The extensions used for
all templating engines are shown in the table below.

\begin{tableiii}{l|l|l}{}{Engine}{Extension}{Output Type}
\lineiii{Jinja2}{.jinja2}{Any}
\lineiii{ZPT}{.html, .htm, .zpt}{HTML}
\lineiii{}{.xhtml, .xhtm, .xml}{XML/XHTML}
\lineiii{Python string formatting}{.pyt}{Any}
\lineiii{Python string templates}{.st}{Any}
\lineiii{Kid}{.kid}{XML/XHTML}
\lineiii{Cheetah}{.che}{XML/XHTML}
\lineiii{Genshi}{.gen}{HTML}
\end{tableiii}

The file listing below is an example of a directory of template files.
In this case the templates correspond to nodes in the document created
by the \environment{description} environment, the \environment{tabular}
environment, \macro{textbf}, and \macro{textit}.
\begin{verbatim}
description.xml
tabular.xml
textbf.html
textit.html
\end{verbatim}

Since there are a lot of templates that are merely one line, it would be
inconvenient to have to create a new file for each template.  In cases
like this, you can use the \file{.zpts} extension for collections of
ZPT templates, or the \file{.jinja2s} extension for collections of
Jinja2 templates, or more generally \file{.pts} for collections of various
template types.  Files with this
extension have multiple templates in them.  Each template is separated
from the next by the template metadata which includes things like the
name of the template, the type (xml, html, or text), and can also alias
template names to another template in the renderer.  The following
metadata names are currently supported.
\begin{tableii}{l|p{4in}}{}{Name}{Purpose}
\lineii{engine}{the name of the templating engine to use.  At the time of
    this writing, the value could be jinja2, zpt, tal (same as zpt),
    html (ZPT HTML template), xml (ZPT XML template), jinja2,
    python (Python formatted string), string (Python string template),
    kid, cheetah, or genshi.}
\lineii{name}{the name or names of the template that is to follow.
    This name is used as the key in the renderer, and also
    corresponds to the node name that will be rendered by the template.
    If more than one name is desired, they are simply separated by
    spaces.}
\lineii{type}{the type of the template: xml, html, or text.  XML templates
    must contain a well-formed XML fragment.  HTML templates are more
    forgiving, but do not support all features of ZPT (see the SimpleTAL
    documentation).}
\lineii{alias}{specifies the name of another template that the given
    names should be aliased to.  This allows you to simply reference
    another template to use rather than redefining one.  For example,
    you might create a new section heading called \macro{introduction}
    that should render the same way as \macro{section}.  In this case,
    you would set the name to ``introduction'' and the alias to
    ``section''.}
\end{tableii}

There are also some defaults that you can set at the top of the file that
get applied to the entire file unles overridden by the meta-data on a
particular template.
\begin{tableii}{l|p{4in}}{}{Name}{Purpose}
\lineii{default-engine}{the name of the engine to use for all templates in
    the file.}
\lineii{default-type}{the default template type for all templates in the file.}
\end{tableii}

The code sample below shows the basic format of a jinja2s file.
\begin{verbatim}
name: textbf bfseries
<b>{{ obj }}</b>

name: textit
<i>{{ obj }}</i>

name: introduction introduction*
alias: section

name: description
<dl>
{% for item in obj %}
  <dt>{{ item.attributes.term }}</dt>
  <dd>{{ item }}</dd>
{% endfor %}
</dl>
\end{verbatim}

The code above is a jinja2s file that contains four templates.  Each template
begins when a line starts with ``name:''.  Other directives have the same
format (i.e. the name of the directive followed by a colon) and must
immediately follow the name directive.  The first template definition
actually applies to two types of nodes \var{textbf} and \var{bfseries}.
You can specify ony number of names on the name line.  The third template
isn't a template at all; it is an alias.  When an alias is specified,
the name (or names) given use the same template as the one specified
in the alias directive. Notice also that starred versions of a macro
can be specified separately. This means that they can use a different
template than the un-starred versions of the command.
The last template shows a loop example.

Here is an example of using various templates engines in a single file.
\begin{verbatim}
name: equation
engine: jinja2
<div class="equation" id="{{ obj.id }}">
  <span class="equation_label">{{ obj.ref }}</span>
  {{ obj }}
</div>

name: textbf
engine: python
<b>%(self)s</b>

name: textit
engine: string
<i>${self}</i>

name: textsc
engine: cheetah
<span class="textsc">${here}</span>

name: textrm
engine: kid
<span class="textrm" py:content="XML(str(here))">normal text</span>

name: textup
engine: genshi
<span class="textup" py:content="markup(here)">upcase text</span>
\end{verbatim}

There are several variables inserted into the template namespace.  Here is
a list of the variables and the templates that support them.

\begin{center}
\begin{tabular}{|l|l|l|l|l|}\hline
\textbf{Object} & \textbf{ZPT/Python Formats/String Template} &
\textbf{Jinja2} &
    \textbf{Cheetah} & \textbf{Kid/Genshi}\\\hline
document node & \var{self} or \var{here} & \var{obj} or \var{here} &  \var{here} & \var{here} \\
parent node & \var{container} & \var{container} & \var{container} & \var{container} \\
document config & \var{config} & \var{config} & \var{config} & \var{config} \\
template instance & \var{template} &  & & \\
renderer instance & \var{templates} & \var{templates} & \var{templates} & \var{templates} \\\hline
\end{tabular}
\end{center}

You'll notice that Kid and Genshi templates require some extra processing
of the variables in order to get the proper markup.  By default, these templates
escape characters like <, >, and \&.  In order to get HTML/XML markup from
the variables you must wrap them in the code shown in the example above.
Hopefully, this limitation will be removed in the future.

When using Jinja2 templates, the default configuration trims white spaces
before and after template tags (see trim_blocks and lstrip_blocks in Jinja2's
documentation).
Also, when developing Jinja2 templates, inserting \verb+{{ debug() }}+
will launch a python debugger session to allow inspection of the
\var{context} variable during rendering.

\subsubsection{Template Overrides\label{sec:tmploverrides}}

It is possible to override the templates located in a renderer's directory
with templates defined elsewhere.  This can be done using the
\verb+--extra-templates+ option or using the
\environment{*TEMPLATES} environment variable.  The ``*'' in the name
\environment{*TEMPLATES} is a wildcard and must be replaced by the name of the
renderer.  For example, if you are using the HTML5 renderer, the
environment variable would be \environment{HTML5TEMPLATES}.  For the PageTemplate
renderer, the environment variable would be \environment{PAGETEMPLATETEMPLATES}.

The format of this variable is the same as that of the \environment{PATH}
environment variable which means that you can put multiple directory
names in this variable.  In addition, the environment variables for
each of the parent renderers is also used, so that you can use
multiple layers of template directories.

You can actually create an entire renderer just using overrides and the
PT renderer.  Since the PT renderer doesn't actually define any templates,
it is just a framework for defining other XML/HTML renderers, you can
simply load the PT renderer and set the \environment{PAGETEMPLATETEMPLATES}
environment
variable to the locations of your templates.  This method of creating
renderers will work for any XML/HTML that doesn't require any special
post-processing.


\subsection{Defining and Using Themes}

In addition to the templates that define how each node should be rendered,
there are also templates that define page layouts.  Page layouts are used
whenever a node in the document generates a new file.   Page layouts
generally include all of the markup required to make a complete document
of the desired DTD, and may include things like navigation buttons,
tables of contents, breadcrumb trails, etc. to link the current file to
other files in the document.

When rendering files, the content of the
node is generated first, then that content is wrapped in a page layout.
The page layouts are defined the same way as regular templates; however,
they all include ``-layout'' at the end of the template name.  For
example the sectioning commands in \LaTeX\ would use the layout templates
``section-layout'', ``subsection-layout'', ``subsubsection-layout'', etc.
Again, these templates can exist in files by themselves or multiply
specified in a zpts file.  If no layout template exists for a particular
node, the template name ``default-layout'' is used.

Since there can be several themes defined within a renderer, theme files
are stored in a subdirectory of a renderer directory.  This directory
is named \file{Themes}.  The \file{Themes} directory itself only contains
directories that correspond to the themes themselves where the name
of the directory corresponds to the name of the theme.  These theme
directories generally only consist of the layout files described above,
but can override other templates as well.  Below is a file listing
demonstrating the structure of a renderer with multiple themese.
\begin{verbatim}
# Renderer directory: contains template files
HTML5/

# Theme directory: contains theme directories
HTML5/Themes/

# Theme directories: contain page layout templates
HTML5/Themes/default/
HTML5/Themes/minimal/
HTML5/Themes/fragment/
\end{verbatim}
\note{If no theme is specified in the document configuration, a theme
    with the name ``default'' is used.}

Since all template directories are created equally, you can also define
themes in template directories specified by environment variables as
described in section \ref{sec:tmploverrides}.  Also, theme files are
searched in the same way as regular templates, so any theme defined
in a renderer superclass' directory is valid as well.

\section{HTML5 Renderer}\label{sec:html5}

\subsection{Basic use and configuration}

The HTML5 Renderer is a subclass of the Page Template Renderer (Section
\ref{sec:pt}). Therefore most of the work is done in its collection of
templates, all written using the Jinja2 template engine.

In addition, this renderer allows packages to override certain templates,
add css or javascript files, and push output files through various
filters, see Section~\ref{sec:html5-pkg}.

Options described in Section~\ref{sec:config-html5} provide easy ways to
customize output. In particular the \longprogramopt{extra-css} option
allows to override CSS styles. Since the generated HTML contains no
inline style (except for some size specification for images and tables),
these CSS overrides allow to completely change the output style.

For large scale CSS changes, one can create a new
CSS theme, either from scratch or by customizing an existing theme.
Existing themes are generated using the CSS extension language SASS (see
\url{http://sass-lang.com/}). Their sources are located in
\file{plasTeX/Renderers/HTML5/sources/sass/}. One way to use customize
them is to do the following (assuming SASS is available on your system).
Copy the above directory somewhere else, say in \file{mysass}, copy
\file{theme-blue.scss} to \file{theme-custom.scss} and replace ``blue''
in the first line by ``custom'', copy \file{_variables_blue.scss} to
\file{_variables_custom.scss}, modify a number of values in this file
and compile using
\begin{verbatim}
sass --update --sourcemap=none mysass:build
\end{verbatim}
This will create \file{build/theme-custom.css} which can be copied to
your project and used by plasTeX using
\begin{verbatim}
plastex ---no-theme-css --extra-css=theme-custom.css mytexfile.tex
\end{verbatim}
Of course one can also change other \file{scss} files for larger
changes. Note that distributed theme css also went through autoprefixer
(\url{https://autoprefixer.github.io/}) to ensure cross browser
compatibility and cssnano (\url{http://cssnano.co/}) to reduce their
size (all those steps are performed by
\file{plasTeX/Renderers/HTML5/sources/build-css.sh}). If CSS
modifications are not enough, one can override templates as discussed in
Section~\ref{sec:tmploverrides}.

The normal way to handle mathematics in this renderer is to use MathJax
(see \url{https://www.mathjax.org}) to render mathematics on client side.
This is controlled by the \longprogramopt{use-mathjax} option which is
set to true by default. Option \longprogramopt{mathjax-url} indicates
where to find the MathJax library. By default it uses a CDN which
ensures using the latest version but prevents offline use.
Instead of client-side rendering, one can use filters to handle
mathematics on the author side. For instance one can use mathjax-node
(\url{https://github.com/mathjax/MathJax-node}) or KaTeX
(\url{https://khan.github.io/KaTeX/}). In this case, one can disable
inclusion of MathJax using option \longprogramopt{no-mathjax} and
use option \longprogramopt{filters} to call the author-side
mathematics renderer.

\subsection{Interactions with packages}\label{sec:html5-pkg}

This section is useful for packages authors. The HTML5 Renderer allows
packages to interact with the rendering process in several ways through
packages resources. Each such resource is wrapped in an object whose
class inherits from \class{PackageResource} located in the module
\module{plasTeX.PackageResource}. Those objects are typically attached to
the document during package loading using the document
\method{addPackageResource} method inside the \function{ProcessOptions}
function.

\subsubsection*{Extra input files}

The following example shows how a package can register templates
overriding regular renderer templates, as well as extra CSS and
javascript files.
The following function should be part of a python package, say
\file{mypkg.py} which imports \class{Path} from \module{pathlib}.
Next to this file, there should be a
folder \file{mypkg} containing files \file{test.css} and \file{test.js},
and a \file{templates} subfolder.

\begin{verbatim}
def ProcessOptions(options, document):
    css = PackageCss(
        renderers='html5',
        path=Path(__file__).parent/'mypkg'/'test.css')
    js = PackageJs(
        renderers='html5',
        path=Path(__file__).parent/'mypkg'/'test.js')
    tpl = PackageTemplateDir(
        renderers='html5',
        path=Path(__file__).parent/'mypkg'/'templates')

    document.addPackageResource([css, js, tpl])
\end{verbatim}

When rendering any document containing \verb+\usepackage{mypkg}+,
the HTML5 renderer will
\begin{itemize}
\item
    copy \file{mypkg/test.css}
    to the \file{styles} subdirectory of the output directory
    and call it from the default html layout
\item
    copy \file{mypkg/test.js}
    to the \file{js} subdirectory of the output directory
    and call it from the default html layout
\item
    load any template contained in \file{mypkg/templates}, overiding the renderer default templates if needed.
\end{itemize}

If you want to copy a CSS or Javascript file to the output directory but
without linking to it from the default html layout, you can
use the keyword argument \verb+copy_only=True+.
The \verb+renderer+ argument can also be a list of renderers, and
its default value is \verb+html5+.

\subsubsection*{Extra output files and filters}

The HTML5 Renderer allows packages to produce extra output files and
define filters that are applied to any output file.

In order to create new output files, any package can register a callback
function taking a document as input a returning a list of created file
names. As an example, let us write a package which adds some document
statistics to an extra output file \file{stats.html} (we will not try to
produce valid html below, only illustrate our point). The package module
only needs to contain the following.
\begin{verbatim}
import os

def ProcessOptions(options, document):
    def makeStats(document):
        nbChap = len(document.getElementsByTagName('chapter'))
        nbSec = len(document.getElementsByTagName('section'))
        with open('stats.html', 'w') as outfile:
            outfile.write("%d chapters and %d sections" % (nbChap, nbSec))
        return ['stats.html']
    cb = PackagePreCleanupCB(
        renderers='html5',
        data=makeStats)
    document.addPackageResource(cb)
\end{verbatim}

As suggested by the key name, those pre-cleanup callbacks are called before
the renderer \method{cleanup} method.

Packages can also register filters to be applied during the cleanup
process. Those filters are functions that take a document and a string
to filter and return the filtered string (using the document object to
provide context if needed). They will be called on the content of each
rendered file, after all files have been rendered.  Registration is
analogous to pre-cleanup callbacks but replacing
\class{PackagePreCleanupCB} by \class{PackageProcessFileContent}.

\subsection{Themes}

The theming support in the HTML5 renderer is a superset of that of the
Page Template Renderer. Any template directory can have a subdirectory called
\file{Themes} which contains theme directories with sets of templates
in them.  The names of the directories in the \file{Themes} directory
corresponds to the name of the theme. Each of these directories can contain
folders named \file{css} and \file{js}, and containing CSS and javascript files
respectively. Those files are copied while rendering, unless
option \longprogramopt{copy-theme-extras} is unset.

There are currently three themes included with \plasTeX: default, minimal and
fragment. The first two themes produce standalone HTML files, including a head
section. The fragment theme produces output that is meant to be inserted in a
HTML document already containg a head section. The default theme include some
javascript and CSS file. One can choose between three variations on CSS using
option \longprogramopt{theme-css} which can be \optval{white}, \optval{blue} or
\optval{green}. The default value is \optval{white} which uses a sober black on
white color scheme with no gradient background of shadow. The \optval{green}
CSS theme uses the historical \plasTeX{} green and yellow color scheme, with
gradients and shadows. The \optval{blue} one uses a blue and gray color scheme,
with gradients and shadows.

When displaying the table of contents navigation panel, the default theme adds
at the end the elements of the list
\verb+document.rendererdata['html5']['extra_toc_items']+.
Each element should be a dictionary with keys \verb+text+ and \verb+url+.

\section{XHTML Renderer}

The XHTML renderer used to be the default renderer of plasTeX but is now
deprecated in favor of the HTML5 renderer, and kept only for
backward compatibility purposes.
The XHTML renderer is a subclass of the Page Template Renderer (section
\ref{sec:pt}). Since the Page Template Renderer can render any variant
of XML or HTML, the XHTML renderer has very little to do in the Python
code. Almost all of the additional processing in the XHTML renderer has
to do with generated images.
Since MathJax (or KaTeX) didn't exist at the time when XHTML was used,
all \LaTeX equations are converted to images by this renderer. In order
for inline equations to line up correctly with the text around them, CSS
attributes are used to adjust the vertical alignment.  Since the images
aren't generated until after all of the document has been rendered, this
CSS information is added in post-processing (i.e. the \method{cleanup}
method).

In addition to the processing of images, all characters with a ordinal
greater than 127 are converted into numerical entities.  This should
prevent any rendering problems due to unknown encodings.

Most of the work in this renderer was in creating the templates for
every \LaTeX\ construct.  Since this renderer was intended to be the
basis of all HTML-based renderers, it must be capable of rendering
all \LaTeX\ constructs; therefore, there are ZPT templates for every
\LaTeX\ command, and the commands in some common \LaTeX\ packages.

\subsection{Themes}

The theming support in the XHTML renderer is the same as that of the
Page Template Renderer.  Any template directory can have a subdirectory called
\file{Themes} which contains theme directories with sets of templates
in them.  The names of the directories in the \file{Themes} directory
corresponds to the name of the theme.  There are currently two themes
included with \plasTeX: default and plain.  The default theme is a
minor variation of the one used in the Python 1.6 documentation.  The
plain theme is a theme with no extra navigation bars.

\section{Other builtin renderers}

In addition to the renderers covered in previous sections, \plasTeX\
comes with a couple of smaller renderers that are less tested but can
still be useful, at least as technological examples.

The simplest output format is given by the Text renderer that will
output a simple unicode text file. It is not based on a templating
engine, it is therefore a good example for people who want to write
a renderer from scratch.

The ManPage renderer outputs files ready for consumption by groff
(directly of through man). Like the Text renderer, it is written
from scratch, without any template engine.

The DocBook renderer is based on the Page Template Renderer.
It supports two themes: book and article having different root elements.
The templates for this renderer using the Cheetah engine which is not
installed by default when installing \plasTeX. Be careful to install
the Cheetah3 fork which is compatible with python 3.

There is also a EPUB renderer based on the XHTML renderer.
