1 <?xml version="1.0" encoding="utf-8"?><!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'><html lang="en" xmlns="http://www.w3.org/1999/xhtml">
3 <title>Twisted Documentation: Extending the Lore Documentation System</title>
4 <link href="stylesheet.css" rel="stylesheet" type="text/css"/>
8 <h1 class="title">Extending the Lore Documentation System</h1>
9 <div class="toc"><ol><li><a href="#auto0">Overview</a></li><li><a href="#auto1">Inputs and Outputs</a></li><ul><li><a href="#auto2">Creating New Inputs</a></li></ul></ol></div>
13 <h2>Overview<a name="auto0"/></h2>
15 <p>The <a href="lore.html" shape="rect">Lore Documentation System</a>, out of the box, is
16 specialized for documenting Twisted. Its markup includes CSS classes for
17 Python, HTML, filenames, and other Twisted-focused categories. But don't
18 think this means Lore can't be used for other documentation tasks! Lore is
19 designed to allow extensions, giving any Python programmer the ability to
20 customize Lore for documenting almost anything.</p>
22 <p>There are several reasons why you would want to extend Lore. You may want
23 to attach file formats Lore does not understand to your documentation. You
24 may want to create callouts that have special meanings to the reader, to give a
25 memorable appearance to text such as, <q>WARNING: This software was written by
26 a frothing madman!</q> You may want to create color-coding for a different
27 programming language, or you may find that Lore does not provide you with
28 enough structure to mark your document up completely. All of these situations
29 can be solved by creating an extension.</p>
31 <h2>Inputs and Outputs<a name="auto1"/></h2>
33 <p>Lore works by reading the HTML source of your document, and
34 producing whatever output the user specifies on the command line. If
35 the HTML document is well-formed XML that meets a certain minimum
36 standard, Lore will be able to to produce some output. All Lore
37 extensions will be written to redefine the <em>input</em>, and most
38 will redefine the output in some way. The name of the default input
39 is <q>lore</q>. When you write your extension, you will come up with
40 a new name for your input, telling Lore what rules to use to process
43 <p>Lore can produce XHTML, LaTeX, and DocBook document formats, which can be
44 displayed directly if you have a user agent capable of viewing them, or
45 processed into a third form such as PostScript or PDF. Another output is
46 called <q>lint</q>, after the static-checking utility for C, and is used for
47 the same reason: to statically check input files for problems. The
48 <q>lint</q> output is just a stream of error messages, not a formatted
49 document, but is important because it gives users the ability to validate
50 their input before trying to process it. For the first example, the only
51 output we will be concerned with is LaTeX.</p>
53 <h3>Creating New Inputs<a name="auto2"/></h3>
54 <p>Create a new input to tell Lore that your document is marked up differently
55 from a vanilla Lore document. This gives you the power to define a new tag
56 class, for example:</p>
57 <pre xml:space="preserve">
58 <p>The Frabjulon <span class="productname">Limpet 2000</span>
59 is the <span class="marketinglie">industry-leading</span> aquatic
60 mollusc counter, bar none.</p>
63 <p>The above HTML is an instance of a new input to Lore, which we will call
64 MyHTML, to differentiate it from the <q>lore</q> input. We want it to have
65 the following markup:</p>
67 <li>A <code>productname</code> class for the <span> tag, which
68 produces underlined text</li>
69 <li>A <code>marketinglie</code> class for <span> tag, which
70 produces larger type, bold text</li>
72 <p>Note that I chose class names that are valid Python identifiers. You will
73 see why shortly. To get these two effects in Lore's HTML output, all we have
74 to do is create a cascading stylesheet (CSS), and use it in the Lore XHTML
75 Template. However, we also want these effects to work in LaTeX, and we want
76 the output of lint to produce no warnings when it sees lines with these 2
77 classes. To make LaTeX and lint work, we start by creating a plugin.</p>
79 <div class="py-listing"><pre><p class="py-linenumber"> 1
90 </p><span class="py-src-keyword">from</span> <span class="py-src-variable">zope</span>.<span class="py-src-variable">interface</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">implements</span>
92 <span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">plugin</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">IPlugin</span>
93 <span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">lore</span>.<span class="py-src-variable">scripts</span>.<span class="py-src-variable">lore</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">IProcessor</span>
95 <span class="py-src-keyword">class</span> <span class="py-src-identifier">MyHTML</span>(<span class="py-src-parameter">object</span>):
96 <span class="py-src-variable">implements</span>(<span class="py-src-variable">IPlugin</span>, <span class="py-src-variable">IProcessor</span>)
98 <span class="py-src-variable">name</span> = <span class="py-src-string">"myhtml"</span>
99 <span class="py-src-variable">moduleName</span> = <span class="py-src-string">"myhtml.factory"</span>
100 </pre><div class="caption">
101 Listing 1: The Plugin File - <a href="listings/lore/a_lore_plugin.py"><span class="filename">listings/lore/a_lore_plugin.py</span></a></div></div>
103 <p>Create this file in a <code class="shell">twisted/plugins/</code>
104 directory (<em>not</em> a package) which is located in a directory in the
105 Python module search path. See the <a href="../../core/howto/plugin.html" shape="rect">Twisted
106 plugin howto</a> for more details on plugins.</p>
108 <p>Users of your extension will pass the value of your plugin's <code class="python">name</code> attribute to lore with the <code class="shell">--input</code> parameter on the command line to select it. For
109 example, to select the plugin defined above, a user would pass <code class="shell">--input myhtml</code>. The <code class="python">moduleName</code> attribute tells Lore where to find the code
110 implementing the plugin. In particular, this module should have a <code class="python">factory</code> attribute which defines a <code class="python">generator_</code>-prefixed method for each output format it
111 supports. Next we'll look at this module.</p>
113 <div class="py-listing"><pre><p class="py-linenumber">1
122 </p><span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">lore</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">default</span>
123 <span class="py-src-keyword">from</span> <span class="py-src-variable">myhtml</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">spitters</span>
125 <span class="py-src-keyword">class</span> <span class="py-src-identifier">MyProcessingFunctionFactory</span>(<span class="py-src-parameter">default</span>.<span class="py-src-parameter">ProcessingFunctionFactory</span>):
126 <span class="py-src-variable">latexSpitters</span>={<span class="py-src-variable">None</span>: <span class="py-src-variable">spitters</span>.<span class="py-src-variable">MyLatexSpitter</span>,
129 <span class="py-src-comment"># initialize the global variable factory with an instance of your new factory</span>
130 <span class="py-src-variable">factory</span>=<span class="py-src-variable">MyProcessingFunctionFactory</span>()
131 </pre><div class="caption">Listing 2: The Input
132 Factory - <a href="listings/lore/factory.py-1"><span class="filename">listings/lore/factory.py-1</span></a></div></div>
134 <p>In Listing 2, we create a subclass of ProcessingFunctionFactory.
135 This class provides a hook for you, a class variable
136 named <code>latexSpitters</code>. This variable tells Lore what new
137 class will be generating LaTeX from your input format. We
138 redefine <code>latexSpitters</code> to <code>MyLatexSpitter</code> in
139 the subclass because this class knows what to do with the new input we
140 have already defined. Last, you must define the module-level
141 variable <code class="py-src-identifier">factory</code>. It should be
142 an instance with the same interface
143 as <code class="py-src-identifier">ProcessingFunctionFactory</code>
144 (e.g. an instance of a subclass, in this
145 case, <code class="py-src-identifier">MyProcessingFunctionFactory</code>).</p>
147 <p>Now let's actually write some code to generate the LaTeX. Doing this
148 requires at least a familiarity with the LaTeX language. Search Google for
149 <q>latex tutorial</q> and you will find any number of useful LaTeX
152 <div class="py-listing"><pre><p class="py-linenumber"> 1
170 </p><span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">lore</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">latex</span>
171 <span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">lore</span>.<span class="py-src-variable">latex</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">processFile</span>
172 <span class="py-src-keyword">import</span> <span class="py-src-variable">os</span>.<span class="py-src-variable">path</span>
174 <span class="py-src-keyword">class</span> <span class="py-src-identifier">MyLatexSpitter</span>(<span class="py-src-parameter">latex</span>.<span class="py-src-parameter">LatexSpitter</span>):
175 <span class="py-src-keyword">def</span> <span class="py-src-identifier">visitNode_span_productname</span>(<span class="py-src-parameter">self</span>, <span class="py-src-parameter">node</span>):
176 <span class="py-src-comment"># start an underline section in LaTeX</span>
177 <span class="py-src-variable">self</span>.<span class="py-src-variable">writer</span>(<span class="py-src-string">'\\underline{'</span>)
178 <span class="py-src-comment"># process the node and its children</span>
179 <span class="py-src-variable">self</span>.<span class="py-src-variable">visitNodeDefault</span>(<span class="py-src-variable">node</span>)
180 <span class="py-src-comment"># end the underline block</span>
181 <span class="py-src-variable">self</span>.<span class="py-src-variable">writer</span>(<span class="py-src-string">'}'</span>)
183 <span class="py-src-keyword">def</span> <span class="py-src-identifier">visitNode_span_marketinglie</span>(<span class="py-src-parameter">self</span>, <span class="py-src-parameter">node</span>):
184 <span class="py-src-comment"># this example turns on more than one LaTeX effect at once</span>
185 <span class="py-src-variable">self</span>.<span class="py-src-variable">writer</span>(<span class="py-src-string">'\\begin{bf}\\begin{Large}'</span>)
186 <span class="py-src-variable">self</span>.<span class="py-src-variable">visitNodeDefault</span>(<span class="py-src-variable">node</span>)
187 <span class="py-src-variable">self</span>.<span class="py-src-variable">writer</span>(<span class="py-src-string">'\\end{Large}\\end{bf}'</span>)
188 </pre><div class="caption">Listing 3:
189 spitters.py - <a href="listings/lore/spitters.py-1"><span class="filename">listings/lore/spitters.py-1</span></a></div></div>
191 <p>The method <code>visitNode_span_productname</code> is our handler
192 for <span> tags with the <code>class="productname"</code>
193 identifier. Lore knows to try methods <code>visitNode_span_*</code>
194 and <code>visitNode_div_*</code> whenever it encounters a new class in
195 one of these tags. This is why the class names have to be valid
196 Python identifiers.</p>
198 <p>Now let's see what Lore does with these new classes with the following
201 <div class="html-listing"><pre class="htmlsource">
204 <title>My First Example</title>
207 <h1>My First Example</h1>
208 <p>The Frabjulon <span class="productname">Limpet 2000</span>
209 is the <span class="marketinglie">industry-leading</span> aquatic
210 mollusc counter, bar none.</p>
214 </pre><div class="caption">Listing 4:
215 1st_example.html - <a href="listings/lore/1st_example.html"><span class="filename">listings/lore/1st_example.html</span></a></div></div>
217 <p>First, verify that your package is laid out correctly. Your directory
218 structure should look like this:</p>
220 <pre xml:space="preserve">
230 <p>In the parent directory of myhtml (that is, <code>myhtml/..</code>), run
231 lore and pdflatex on the input:</p>
233 <pre class="shell" xml:space="preserve">
234 $ lore --input myhtml --output latex 1st_example.html
235 [########################################] (*Done*)
237 $ pdflatex 1st_example.tex
238 [ . . . latex output omitted for brevity . . . ]
239 Output written on 1st_example.pdf (1 page, 22260 bytes).
240 Transcript written on 1st_example.log.
243 <p>And here's what the rendered PDF looks like:</p>
245 <p><img src="../img/myhtml-output.png"/></p>
247 <p>What happens when we run lore on this file using the lint output?</p>
249 <pre class="shell" xml:space="preserve">
250 $ lore --input myhtml --output lint 1st_example.html
251 1st_example.html:7:47: unknown class productname
252 1st_example.html:8:38: unknown class marketinglie
253 [########################################] (*Done*)
256 <p>Lint reports these classes as errors, even though our spitter knows how to
257 process them. To fix this problem, we must add to <code class="py-filename">factory.py</code>.</p>
259 <div class="py-listing"><pre><p class="py-linenumber"> 1
279 </p><span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">lore</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">default</span>
280 <span class="py-src-keyword">from</span> <span class="py-src-variable">myhtml</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">spitters</span>
282 <span class="py-src-keyword">class</span> <span class="py-src-identifier">MyProcessingFunctionFactory</span>(<span class="py-src-parameter">default</span>.<span class="py-src-parameter">ProcessingFunctionFactory</span>):
283 <span class="py-src-variable">latexSpitters</span>={<span class="py-src-variable">None</span>: <span class="py-src-variable">spitters</span>.<span class="py-src-variable">MyLatexSpitter</span>,
286 <span class="py-src-comment"># redefine getLintChecker to validate our classes</span>
287 <span class="py-src-keyword">def</span> <span class="py-src-identifier">getLintChecker</span>(<span class="py-src-parameter">self</span>):
288 <span class="py-src-comment"># use the default checker from parent</span>
289 <span class="py-src-variable">checker</span> = <span class="py-src-variable">lint</span>.<span class="py-src-variable">getDefaultChecker</span>()
290 <span class="py-src-variable">checker</span>.<span class="py-src-variable">allowedClasses</span> = <span class="py-src-variable">checker</span>.<span class="py-src-variable">allowedClasses</span>.<span class="py-src-variable">copy</span>()
291 <span class="py-src-variable">oldSpan</span> = <span class="py-src-variable">checker</span>.<span class="py-src-variable">allowedClasses</span>[<span class="py-src-string">'span'</span>]
292 <span class="py-src-variable">checkfunc</span>=<span class="py-src-keyword">lambda</span> <span class="py-src-variable">cl</span>: <span class="py-src-variable">oldSpan</span>(<span class="py-src-variable">cl</span>) <span class="py-src-keyword">or</span> <span class="py-src-variable">cl</span> <span class="py-src-keyword">in</span> [<span class="py-src-string">'marketinglie'</span>,
293 <span class="py-src-string">'productname'</span>]
294 <span class="py-src-variable">checker</span>.<span class="py-src-variable">allowedClasses</span>[<span class="py-src-string">'span'</span>] = <span class="py-src-variable">checkfunc</span>
295 <span class="py-src-keyword">return</span> <span class="py-src-variable">checker</span>
297 <span class="py-src-comment"># initialize the global variable factory with an instance of your new factory</span>
298 <span class="py-src-variable">factory</span>=<span class="py-src-variable">MyProcessingFunctionFactory</span>()
299 </pre><div class="caption">Listing 5: Input
300 Factory with Lint Support - <a href="listings/lore/factory.py-2"><span class="filename">listings/lore/factory.py-2</span></a></div></div>
302 <p>The method <code class="py-src-identifier">getLintChecker</code> is called
303 by Lore to produce the lint output. This modification adds our classes to the
304 list of classes lint ignores:</p>
306 <pre class="shell" xml:space="preserve">
307 $ lore --input myhtml --output lint 1st_example.html
308 [########################################] (*Done*)
312 <p>Finally, there are two other sub-outputs of LaTeX, for a total of three
313 different ways that Lore can produce LaTeX: the default way, which produces as
314 output an entire, self-contained LaTeX document; with <code class="shell">--config section</code> on the command line, which produces a
315 LaTeX \section; and with <code class="shell">--config chapter</code>, which
316 produces a LaTeX \chapter. To support these options as well, the solution is
317 to make the new spitter class a mixin, and use it with the <code class="py-src-identifier">SectionLatexSpitter</code> and <code class="py-src-identifier">ChapterLatexSpitter</code>, respectively.
318 Comments in the following listings tell you everything you need to know about
319 making these simple changes:</p>
322 <li><div class="py-listing"><pre><p class="py-linenumber"> 1
343 </p><span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">lore</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">default</span>
344 <span class="py-src-keyword">from</span> <span class="py-src-variable">myhtml</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">spitters</span>
346 <span class="py-src-keyword">class</span> <span class="py-src-identifier">MyProcessingFunctionFactory</span>(<span class="py-src-parameter">default</span>.<span class="py-src-parameter">ProcessingFunctionFactory</span>):
347 <span class="py-src-comment"># 1. add the keys "chapter" and "section" to latexSpitters to handle the</span>
348 <span class="py-src-comment"># --config chapter and --config section options</span>
349 <span class="py-src-variable">latexSpitters</span>={<span class="py-src-variable">None</span>: <span class="py-src-variable">spitters</span>.<span class="py-src-variable">MyLatexSpitter</span>,
350 <span class="py-src-string">"section"</span>: <span class="py-src-variable">spitters</span>.<span class="py-src-variable">MySectionLatexSpitter</span>,
351 <span class="py-src-string">"chapter"</span>: <span class="py-src-variable">spitters</span>.<span class="py-src-variable">MyChapterLatexSpitter</span>,
354 <span class="py-src-keyword">def</span> <span class="py-src-identifier">getLintChecker</span>(<span class="py-src-parameter">self</span>):
355 <span class="py-src-variable">checker</span> = <span class="py-src-variable">lint</span>.<span class="py-src-variable">getDefaultChecker</span>()
356 <span class="py-src-variable">checker</span>.<span class="py-src-variable">allowedClasses</span> = <span class="py-src-variable">checker</span>.<span class="py-src-variable">allowedClasses</span>.<span class="py-src-variable">copy</span>()
357 <span class="py-src-variable">oldSpan</span> = <span class="py-src-variable">checker</span>.<span class="py-src-variable">allowedClasses</span>[<span class="py-src-string">'span'</span>]
358 <span class="py-src-variable">checkfunc</span>=<span class="py-src-keyword">lambda</span> <span class="py-src-variable">cl</span>: <span class="py-src-variable">oldSpan</span>(<span class="py-src-variable">cl</span>) <span class="py-src-keyword">or</span> <span class="py-src-variable">cl</span> <span class="py-src-keyword">in</span> [<span class="py-src-string">'marketinglie'</span>,
359 <span class="py-src-string">'productname'</span>]
360 <span class="py-src-variable">checker</span>.<span class="py-src-variable">allowedClasses</span>[<span class="py-src-string">'span'</span>] = <span class="py-src-variable">checkfunc</span>
361 <span class="py-src-keyword">return</span> <span class="py-src-variable">checker</span>
363 <span class="py-src-variable">factory</span>=<span class="py-src-variable">MyProcessingFunctionFactory</span>()
364 </pre><div class="caption">factory.py - <a href="listings/lore/factory.py-3"><span class="filename">listings/lore/factory.py-3</span></a></div></div></li>
365 <li><div class="py-listing"><pre><p class="py-linenumber"> 1
391 </p><span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">lore</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">latex</span>
392 <span class="py-src-keyword">from</span> <span class="py-src-variable">twisted</span>.<span class="py-src-variable">lore</span>.<span class="py-src-variable">latex</span> <span class="py-src-keyword">import</span> <span class="py-src-variable">processFile</span>
393 <span class="py-src-keyword">import</span> <span class="py-src-variable">os</span>.<span class="py-src-variable">path</span>
395 <span class="py-src-comment"># 2. Create a new mixin that does what the old MyLatexSpitter used to do:</span>
396 <span class="py-src-comment"># process the new classes we defined</span>
397 <span class="py-src-keyword">class</span> <span class="py-src-identifier">MySpitterMixin</span>:
398 <span class="py-src-keyword">def</span> <span class="py-src-identifier">visitNode_span_productname</span>(<span class="py-src-parameter">self</span>, <span class="py-src-parameter">node</span>):
399 <span class="py-src-variable">self</span>.<span class="py-src-variable">writer</span>(<span class="py-src-string">'\\underline{'</span>)
400 <span class="py-src-variable">self</span>.<span class="py-src-variable">visitNodeDefault</span>(<span class="py-src-variable">node</span>)
401 <span class="py-src-variable">self</span>.<span class="py-src-variable">writer</span>(<span class="py-src-string">'}'</span>)
403 <span class="py-src-keyword">def</span> <span class="py-src-identifier">visitNode_span_marketinglie</span>(<span class="py-src-parameter">self</span>, <span class="py-src-parameter">node</span>):
404 <span class="py-src-variable">self</span>.<span class="py-src-variable">writer</span>(<span class="py-src-string">'\\begin{bf}\\begin{Large}'</span>)
405 <span class="py-src-variable">self</span>.<span class="py-src-variable">visitNodeDefault</span>(<span class="py-src-variable">node</span>)
406 <span class="py-src-variable">self</span>.<span class="py-src-variable">writer</span>(<span class="py-src-string">'\\end{Large}\\end{bf}'</span>)
408 <span class="py-src-comment"># 3. inherit from the mixin class for each of the three sub-spitters</span>
409 <span class="py-src-keyword">class</span> <span class="py-src-identifier">MyLatexSpitter</span>(<span class="py-src-parameter">MySpitterMixin</span>, <span class="py-src-parameter">latex</span>.<span class="py-src-parameter">LatexSpitter</span>):
410 <span class="py-src-keyword">pass</span>
412 <span class="py-src-keyword">class</span> <span class="py-src-identifier">MySectionLatexSpitter</span>(<span class="py-src-parameter">MySpitterMixin</span>, <span class="py-src-parameter">latex</span>.<span class="py-src-parameter">SectionLatexSpitter</span>):
413 <span class="py-src-keyword">pass</span>
415 <span class="py-src-keyword">class</span> <span class="py-src-identifier">MyChapterLatexSpitter</span>(<span class="py-src-parameter">MySpitterMixin</span>, <span class="py-src-parameter">latex</span>.<span class="py-src-parameter">ChapterLatexSpitter</span>):
416 <span class="py-src-keyword">pass</span>
417 </pre><div class="caption">spitters.py - <a href="listings/lore/spitters.py-2"><span class="filename">listings/lore/spitters.py-2</span></a></div></div></li>
424 <p><a href="index.html">Index</a></p>
425 <span class="version">Version: 12.1.0</span>