1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
2 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
4 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
5 <meta name="generator" content="Docutils 0.14: http://docutils.sourceforge.net/" />
6 <title>Document loading and URL resolving</title>
7 <link rel="stylesheet" href="style.css" type="text/css" />
8 <script type="text/javascript">
9 function trigger_menu(event) {
10 var sidemenu = document.getElementById("sidemenu");
11 var classes = sidemenu.getAttribute("class");
12 classes = (classes.indexOf(" visible") === -1) ? classes + " visible" : classes.replace(" visible", "");
13 sidemenu.setAttribute("class", classes);
14 event.preventDefault();
15 event.stopPropagation();
17 function hide_menu() {
18 var sidemenu = document.getElementById("sidemenu");
19 var classes = sidemenu.getAttribute("class");
20 if (classes.indexOf(" visible") !== -1) {
21 sidemenu.setAttribute("class", classes.replace(" visible", ""));
24 </script><meta content="width=device-width, initial-scale=1" name="viewport" /></head>
25 <body onclick="hide_menu()">
26 <div class="document" id="document-loading-and-url-resolving">
27 <div class="sidemenu" id="sidemenu"><div class="menutrigger" onclick="trigger_menu(event)">Menu</div><div class="menu"><ul id="lxml-section"><li><span class="section title">lxml</span><ul class="menu foreign" id="index-menu"><li class="menu title"><a href="index.html">lxml</a><ul class="submenu"><li class="menu item"><a href="index.html#introduction">Introduction</a></li><li class="menu item"><a href="index.html#support-the-project">Support the project</a></li><li class="menu item"><a href="index.html#documentation">Documentation</a></li><li class="menu item"><a href="index.html#download">Download</a></li><li class="menu item"><a href="index.html#mailing-list">Mailing list</a></li><li class="menu item"><a href="index.html#bug-tracker">Bug tracker</a></li><li class="menu item"><a href="index.html#license">License</a></li><li class="menu item"><a href="index.html#old-versions">Old Versions</a></li><li class="menu item"><a href="index.html#legal-notice-for-donations">Legal Notice for Donations</a></li></ul></li></ul><ul class="menu foreign" id="intro-menu"><li class="menu title"><a href="intro.html">Why lxml?</a><ul class="submenu"><li class="menu item"><a href="intro.html#motto">Motto</a></li><li class="menu item"><a href="intro.html#aims">Aims</a></li></ul></li></ul><ul class="menu foreign" id="installation-menu"><li class="menu title"><a href="installation.html">Installing lxml</a><ul class="submenu"><li class="menu item"><a href="installation.html#where-to-get-it">Where to get it</a></li><li class="menu item"><a href="installation.html#requirements">Requirements</a></li><li class="menu item"><a href="installation.html#installation">Installation</a></li><li class="menu item"><a href="installation.html#building-lxml-from-dev-sources">Building lxml from dev sources</a></li><li class="menu item"><a href="installation.html#using-lxml-with-python-libxml2">Using lxml with python-libxml2</a></li><li class="menu item"><a href="installation.html#source-builds-on-ms-windows">Source builds on MS Windows</a></li><li class="menu item"><a href="installation.html#source-builds-on-macos-x">Source builds on MacOS-X</a></li></ul></li></ul><ul class="menu foreign" id="performance-menu"><li class="menu title"><a href="performance.html">Benchmarks and Speed</a><ul class="submenu"><li class="menu item"><a href="performance.html#general-notes">General notes</a></li><li class="menu item"><a href="performance.html#how-to-read-the-timings">How to read the timings</a></li><li class="menu item"><a href="performance.html#parsing-and-serialising">Parsing and Serialising</a></li><li class="menu item"><a href="performance.html#the-elementtree-api">The ElementTree API</a></li><li class="menu item"><a href="performance.html#xpath">XPath</a></li><li class="menu item"><a href="performance.html#a-longer-example">A longer example</a></li><li class="menu item"><a href="performance.html#lxml-objectify">lxml.objectify</a></li></ul></li></ul><ul class="menu foreign" id="compatibility-menu"><li class="menu title"><a href="compatibility.html">ElementTree compatibility of lxml.etree</a></li></ul><ul class="menu foreign" id="FAQ-menu"><li class="menu title"><a href="FAQ.html">lxml FAQ - Frequently Asked Questions</a><ul class="submenu"><li class="menu item"><a href="FAQ.html#general-questions">General Questions</a></li><li class="menu item"><a href="FAQ.html#installation">Installation</a></li><li class="menu item"><a href="FAQ.html#contributing">Contributing</a></li><li class="menu item"><a href="FAQ.html#bugs">Bugs</a></li><li class="menu item"><a href="FAQ.html#id1">Threading</a></li><li class="menu item"><a href="FAQ.html#parsing-and-serialisation">Parsing and Serialisation</a></li><li class="menu item"><a href="FAQ.html#xpath-and-document-traversal">XPath and Document Traversal</a></li></ul></li></ul></li></ul><ul id="Developing with lxml-section"><li><span class="section title">Developing with lxml</span><ul class="menu foreign" id="tutorial-menu"><li class="menu title"><a href="tutorial.html">The lxml.etree Tutorial</a><ul class="submenu"><li class="menu item"><a href="tutorial.html#the-element-class">The Element class</a></li><li class="menu item"><a href="tutorial.html#the-elementtree-class">The ElementTree class</a></li><li class="menu item"><a href="tutorial.html#parsing-from-strings-and-files">Parsing from strings and files</a></li><li class="menu item"><a href="tutorial.html#namespaces">Namespaces</a></li><li class="menu item"><a href="tutorial.html#the-e-factory">The E-factory</a></li><li class="menu item"><a href="tutorial.html#elementpath">ElementPath</a></li></ul></li></ul><ul class="menu foreign" id="api index-menu"><li class="menu title"><a href="api/index.html">API reference</a></li></ul><ul class="menu foreign" id="api-menu"><li class="menu title"><a href="api.html">APIs specific to lxml.etree</a><ul class="submenu"><li class="menu item"><a href="api.html#lxml-etree">lxml.etree</a></li><li class="menu item"><a href="api.html#other-element-apis">Other Element APIs</a></li><li class="menu item"><a href="api.html#trees-and-documents">Trees and Documents</a></li><li class="menu item"><a href="api.html#iteration">Iteration</a></li><li class="menu item"><a href="api.html#error-handling-on-exceptions">Error handling on exceptions</a></li><li class="menu item"><a href="api.html#error-logging">Error logging</a></li><li class="menu item"><a href="api.html#serialisation">Serialisation</a></li><li class="menu item"><a href="api.html#incremental-xml-generation">Incremental XML generation</a></li><li class="menu item"><a href="api.html#cdata">CDATA</a></li><li class="menu item"><a href="api.html#xinclude-and-elementinclude">XInclude and ElementInclude</a></li></ul></li></ul><ul class="menu foreign" id="parsing-menu"><li class="menu title"><a href="parsing.html">Parsing XML and HTML with lxml</a><ul class="submenu"><li class="menu item"><a href="parsing.html#parsers">Parsers</a></li><li class="menu item"><a href="parsing.html#the-target-parser-interface">The target parser interface</a></li><li class="menu item"><a href="parsing.html#the-feed-parser-interface">The feed parser interface</a></li><li class="menu item"><a href="parsing.html#incremental-event-parsing">Incremental event parsing</a></li><li class="menu item"><a href="parsing.html#iterparse-and-iterwalk">iterparse and iterwalk</a></li><li class="menu item"><a href="parsing.html#python-unicode-strings">Python unicode strings</a></li></ul></li></ul><ul class="menu foreign" id="validation-menu"><li class="menu title"><a href="validation.html">Validation with lxml</a><ul class="submenu"><li class="menu item"><a href="validation.html#validation-at-parse-time">Validation at parse time</a></li><li class="menu item"><a href="validation.html#id1">DTD</a></li><li class="menu item"><a href="validation.html#relaxng">RelaxNG</a></li><li class="menu item"><a href="validation.html#xmlschema">XMLSchema</a></li><li class="menu item"><a href="validation.html#id2">Schematron</a></li><li class="menu item"><a href="validation.html#id3">(Pre-ISO-Schematron)</a></li></ul></li></ul><ul class="menu foreign" id="xpathxslt-menu"><li class="menu title"><a href="xpathxslt.html">XPath and XSLT with lxml</a><ul class="submenu"><li class="menu item"><a href="xpathxslt.html#xpath">XPath</a></li><li class="menu item"><a href="xpathxslt.html#xslt">XSLT</a></li></ul></li></ul><ul class="menu foreign" id="objectify-menu"><li class="menu title"><a href="objectify.html">lxml.objectify</a><ul class="submenu"><li class="menu item"><a href="objectify.html#the-lxml-objectify-api">The lxml.objectify API</a></li><li class="menu item"><a href="objectify.html#asserting-a-schema">Asserting a Schema</a></li><li class="menu item"><a href="objectify.html#objectpath">ObjectPath</a></li><li class="menu item"><a href="objectify.html#python-data-types">Python data types</a></li><li class="menu item"><a href="objectify.html#how-data-types-are-matched">How data types are matched</a></li><li class="menu item"><a href="objectify.html#what-is-different-from-lxml-etree">What is different from lxml.etree?</a></li></ul></li></ul><ul class="menu foreign" id="lxmlhtml-menu"><li class="menu title"><a href="lxmlhtml.html">lxml.html</a><ul class="submenu"><li class="menu item"><a href="lxmlhtml.html#parsing-html">Parsing HTML</a></li><li class="menu item"><a href="lxmlhtml.html#html-element-methods">HTML Element Methods</a></li><li class="menu item"><a href="lxmlhtml.html#running-html-doctests">Running HTML doctests</a></li><li class="menu item"><a href="lxmlhtml.html#creating-html-with-the-e-factory">Creating HTML with the E-factory</a></li><li class="menu item"><a href="lxmlhtml.html#working-with-links">Working with links</a></li><li class="menu item"><a href="lxmlhtml.html#forms">Forms</a></li><li class="menu item"><a href="lxmlhtml.html#cleaning-up-html">Cleaning up HTML</a></li><li class="menu item"><a href="lxmlhtml.html#html-diff">HTML Diff</a></li><li class="menu item"><a href="lxmlhtml.html#examples">Examples</a></li></ul></li></ul><ul class="menu foreign" id="cssselect-menu"><li class="menu title"><a href="cssselect.html">lxml.cssselect</a><ul class="submenu"><li class="menu item"><a href="cssselect.html#the-cssselector-class">The CSSSelector class</a></li><li class="menu item"><a href="cssselect.html#the-cssselect-method">The cssselect method</a></li><li class="menu item"><a href="cssselect.html#supported-selectors">Supported Selectors</a></li><li class="menu item"><a href="cssselect.html#namespaces">Namespaces</a></li></ul></li></ul><ul class="menu foreign" id="elementsoup-menu"><li class="menu title"><a href="elementsoup.html">BeautifulSoup Parser</a><ul class="submenu"><li class="menu item"><a href="elementsoup.html#parsing-with-the-soupparser">Parsing with the soupparser</a></li><li class="menu item"><a href="elementsoup.html#entity-handling">Entity handling</a></li><li class="menu item"><a href="elementsoup.html#using-soupparser-as-a-fallback">Using soupparser as a fallback</a></li><li class="menu item"><a href="elementsoup.html#using-only-the-encoding-detection">Using only the encoding detection</a></li></ul></li></ul><ul class="menu foreign" id="html5parser-menu"><li class="menu title"><a href="html5parser.html">html5lib Parser</a><ul class="submenu"><li class="menu item"><a href="html5parser.html#differences-to-regular-html-parsing">Differences to regular HTML parsing</a></li><li class="menu item"><a href="html5parser.html#function-reference">Function Reference</a></li></ul></li></ul></li></ul><ul id="Extending lxml-section"><li><span class="section title">Extending lxml</span><ul class="menu current" id="resolvers-menu"><li class="menu title"><a href="resolvers.html">Document loading and URL resolving</a><ul class="submenu"><li class="menu item"><a href="resolvers.html#xml-catalogs">XML Catalogs</a></li><li class="menu item"><a href="resolvers.html#uri-resolvers">URI Resolvers</a></li><li class="menu item"><a href="resolvers.html#document-loading-in-context">Document loading in context</a></li><li class="menu item"><a href="resolvers.html#i-o-access-control-in-xslt">I/O access control in XSLT</a></li></ul></li></ul><ul class="menu foreign" id="extensions-menu"><li class="menu title"><a href="extensions.html">Python extensions for XPath and XSLT</a><ul class="submenu"><li class="menu item"><a href="extensions.html#xpath-extension-functions">XPath Extension functions</a></li><li class="menu item"><a href="extensions.html#xslt-extension-elements">XSLT extension elements</a></li></ul></li></ul><ul class="menu foreign" id="element classes-menu"><li class="menu title"><a href="element_classes.html">Using custom Element classes in lxml</a><ul class="submenu"><li class="menu item"><a href="element_classes.html#background-on-element-proxies">Background on Element proxies</a></li><li class="menu item"><a href="element_classes.html#element-initialization">Element initialization</a></li><li class="menu item"><a href="element_classes.html#setting-up-a-class-lookup-scheme">Setting up a class lookup scheme</a></li><li class="menu item"><a href="element_classes.html#generating-xml-with-custom-classes">Generating XML with custom classes</a></li><li class="menu item"><a href="element_classes.html#id1">Implementing namespaces</a></li></ul></li></ul><ul class="menu foreign" id="sax-menu"><li class="menu title"><a href="sax.html">Sax support</a><ul class="submenu"><li class="menu item"><a href="sax.html#building-a-tree-from-sax-events">Building a tree from SAX events</a></li><li class="menu item"><a href="sax.html#producing-sax-events-from-an-elementtree-or-element">Producing SAX events from an ElementTree or Element</a></li><li class="menu item"><a href="sax.html#interfacing-with-pulldom-minidom">Interfacing with pulldom/minidom</a></li></ul></li></ul><ul class="menu foreign" id="capi-menu"><li class="menu title"><a href="capi.html">The public C-API of lxml.etree</a><ul class="submenu"><li class="menu item"><a href="capi.html#passing-generated-trees-through-python">Passing generated trees through Python</a></li><li class="menu item"><a href="capi.html#writing-external-modules-in-cython">Writing external modules in Cython</a></li><li class="menu item"><a href="capi.html#writing-external-modules-in-c">Writing external modules in C</a></li></ul></li></ul></li></ul><ul id="Developing lxml-section"><li><span class="section title">Developing lxml</span><ul class="menu foreign" id="build-menu"><li class="menu title"><a href="build.html">How to build lxml from source</a><ul class="submenu"><li class="menu item"><a href="build.html#cython">Cython</a></li><li class="menu item"><a href="build.html#github-git-and-hg">Github, git and hg</a></li><li class="menu item"><a href="build.html#building-the-sources">Building the sources</a></li><li class="menu item"><a href="build.html#running-the-tests-and-reporting-errors">Running the tests and reporting errors</a></li><li class="menu item"><a href="build.html#building-an-egg-or-wheel">Building an egg or wheel</a></li><li class="menu item"><a href="build.html#building-lxml-on-macos-x">Building lxml on MacOS-X</a></li><li class="menu item"><a href="build.html#static-linking-on-windows">Static linking on Windows</a></li><li class="menu item"><a href="build.html#building-debian-packages-from-svn-sources">Building Debian packages from SVN sources</a></li></ul></li></ul><ul class="menu foreign" id="lxml source howto-menu"><li class="menu title"><a href="lxml-source-howto.html">How to read the source of lxml</a><ul class="submenu"><li class="menu item"><a href="lxml-source-howto.html#what-is-cython">What is Cython?</a></li><li class="menu item"><a href="lxml-source-howto.html#where-to-start">Where to start?</a></li><li class="menu item"><a href="lxml-source-howto.html#lxml-etree">lxml.etree</a></li><li class="menu item"><a href="lxml-source-howto.html#python-modules">Python modules</a></li><li class="menu item"><a href="lxml-source-howto.html#lxml-objectify">lxml.objectify</a></li><li class="menu item"><a href="lxml-source-howto.html#lxml-html">lxml.html</a></li></ul></li></ul><ul class="menu foreign" id="changes 4 4 3-menu"><li class="menu title"><a href="changes-4.4.3.html">Release Changelog</a></li></ul><ul class="menu foreign" id="credits-menu"><li class="menu title"><a href="credits.html">Credits</a><ul class="submenu"><li class="menu item"><a href="credits.html#main-contributors">Main contributors</a></li><li class="menu item"><a href="credits.html#special-thanks-goes-to">Special thanks goes to:</a></li></ul></li></ul></li><li><a href="/sitemap.html">Sitemap</a></li></ul></div></div><h1 class="title">Document loading and URL resolving</h1>
29 <div class="contents topic" id="contents">
30 <p class="topic-title first">Contents</p>
32 <li><a class="reference internal" href="#xml-catalogs" id="id2">XML Catalogs</a></li>
33 <li><a class="reference internal" href="#uri-resolvers" id="id3">URI Resolvers</a></li>
34 <li><a class="reference internal" href="#document-loading-in-context" id="id4">Document loading in context</a></li>
35 <li><a class="reference internal" href="#i-o-access-control-in-xslt" id="id5">I/O access control in XSLT</a></li>
38 <p>The normal way to load external entities (such as DTDs) is by using
39 XML catalogs. Lxml also has support for user provided document
40 loaders in both the parsers and XSL transformations. These so-called
41 resolvers are subclasses of the etree.Resolver class.</p>
42 <div class="section" id="xml-catalogs">
44 <p>When loading an external entity for a document, e.g. a DTD, the parser
45 is normally configured to prevent network access (see the
46 <tt class="docutils literal">no_network</tt> parser option). Instead, it will try to load the
47 entity from their local file system path or, in the most common case
48 that the entity uses a network URL as reference, from a local XML
50 <p><a class="reference external" href="http://www.oasis-open.org/committees/entity/spec.html">XML catalogs</a> are the preferred and agreed-on mechanism to load
51 external entities from XML processors. Most tools will use them, so
52 it is worth configuring them properly on a system. Many Linux
53 installations use them by default, but on other systems they may need
54 to get enabled manually. The <a class="reference external" href="http://xmlsoft.org/">libxml2 site</a> has some documentation
55 on <a class="reference external" href="http://xmlsoft.org/catalog.html">how to set up XML catalogs</a></p>
57 <div class="section" id="uri-resolvers">
58 <h1>URI Resolvers</h1>
59 <p>Here is an example of a custom resolver:</p>
60 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">lxml</span> <span class="kn">import</span> <span class="n">etree</span>
62 <span class="gp">>>> </span><span class="k">class</span> <span class="nc">DTDResolver</span><span class="p">(</span><span class="n">etree</span><span class="o">.</span><span class="n">Resolver</span><span class="p">):</span>
63 <span class="gp">... </span> <span class="k">def</span> <span class="nf">resolve</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="nb">id</span><span class="p">,</span> <span class="n">context</span><span class="p">):</span>
64 <span class="gp">... </span> <span class="k">print</span><span class="p">(</span><span class="s2">"Resolving URL '</span><span class="si">%s</span><span class="s2">'"</span> <span class="o">%</span> <span class="n">url</span><span class="p">)</span>
65 <span class="gp">... </span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">resolve_string</span><span class="p">(</span>
66 <span class="gp">... </span> <span class="s1">'<!ENTITY myentity "[resolved text: </span><span class="si">%s</span><span class="s1">]">'</span> <span class="o">%</span> <span class="n">url</span><span class="p">,</span> <span class="n">context</span><span class="p">)</span>
68 <p>This defines a resolver that always returns a dynamically generated DTD
69 fragment defining an entity. The <tt class="docutils literal">url</tt> argument passes the system URL of
70 the requested document, the <tt class="docutils literal">id</tt> argument is the public ID. Note that any
71 of these may be None. The context object is not normally used by client code.</p>
72 <p>Resolving is based on the methods of the Resolver object that build
73 internal representations of the result document. The following
76 <li><tt class="docutils literal">resolve_string</tt> takes a parsable string as result document</li>
77 <li><tt class="docutils literal">resolve_filename</tt> takes a filename</li>
78 <li><tt class="docutils literal">resolve_file</tt> takes an open file-like object that has at least a read() method</li>
79 <li><tt class="docutils literal">resolve_empty</tt> resolves into an empty document</li>
81 <p>The <tt class="docutils literal">resolve()</tt> method may choose to return None, in which case the next
82 registered resolver (or the default resolver) is consulted. Resolving always
83 terminates if <tt class="docutils literal">resolve()</tt> returns the result of any of the above
84 <tt class="docutils literal"><span class="pre">resolve_*()</span></tt> methods.</p>
85 <p>Resolvers are registered local to a parser:</p>
86 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="n">parser</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">XMLParser</span><span class="p">(</span><span class="n">load_dtd</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
87 <span class="gp">>>> </span><span class="n">parser</span><span class="o">.</span><span class="n">resolvers</span><span class="o">.</span><span class="n">add</span><span class="p">(</span> <span class="n">DTDResolver</span><span class="p">()</span> <span class="p">)</span>
89 <p>Note that we instantiate a parser that loads the DTD. This is not done by the
90 default parser, which does no validation. When we use this parser to parse a
91 document that requires resolving a URL, it will call our custom resolver:</p>
92 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="n">xml</span> <span class="o">=</span> <span class="s1">'<!DOCTYPE doc SYSTEM "MissingDTD.dtd"><doc>&myentity;</doc>'</span>
93 <span class="gp">>>> </span><span class="n">tree</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">StringIO</span><span class="p">(</span><span class="n">xml</span><span class="p">),</span> <span class="n">parser</span><span class="p">)</span>
94 <span class="go">Resolving URL 'MissingDTD.dtd'</span>
95 <span class="gp">>>> </span><span class="n">root</span> <span class="o">=</span> <span class="n">tree</span><span class="o">.</span><span class="n">getroot</span><span class="p">()</span>
96 <span class="gp">>>> </span><span class="k">print</span><span class="p">(</span><span class="n">root</span><span class="o">.</span><span class="n">text</span><span class="p">)</span>
97 <span class="go">[resolved text: MissingDTD.dtd]</span>
99 <p>The entity in the document was correctly resolved by the generated DTD
102 <div class="section" id="document-loading-in-context">
103 <h1>Document loading in context</h1>
104 <p>XML documents memorise their initial parser (and its resolvers) during their
105 life-time. This means that a lookup process related to a document will use
106 the resolvers of the document's parser. We can demonstrate this with a
107 resolver that only responds to a specific prefix:</p>
108 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="k">class</span> <span class="nc">PrefixResolver</span><span class="p">(</span><span class="n">etree</span><span class="o">.</span><span class="n">Resolver</span><span class="p">):</span>
109 <span class="gp">... </span> <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">prefix</span><span class="p">):</span>
110 <span class="gp">... </span> <span class="bp">self</span><span class="o">.</span><span class="n">prefix</span> <span class="o">=</span> <span class="n">prefix</span>
111 <span class="gp">... </span> <span class="bp">self</span><span class="o">.</span><span class="n">result_xml</span> <span class="o">=</span> <span class="s1">'''</span><span class="se">\</span>
112 <span class="gp">... </span><span class="s1"> <xsl:stylesheet</span>
113 <span class="gp">... </span><span class="s1"> xmlns:xsl="http://www.w3.org/1999/XSL/Transform"></span>
114 <span class="gp">... </span><span class="s1"> <test xmlns="testNS"></span><span class="si">%s</span><span class="s1">-TEST</test></span>
115 <span class="gp">... </span><span class="s1"> </xsl:stylesheet></span>
116 <span class="gp">... </span><span class="s1"> '''</span> <span class="o">%</span> <span class="n">prefix</span>
117 <span class="gp">... </span> <span class="k">def</span> <span class="nf">resolve</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">url</span><span class="p">,</span> <span class="n">pubid</span><span class="p">,</span> <span class="n">context</span><span class="p">):</span>
118 <span class="gp">... </span> <span class="k">if</span> <span class="n">url</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">prefix</span><span class="p">):</span>
119 <span class="gp">... </span> <span class="k">print</span><span class="p">(</span><span class="s2">"Resolved url </span><span class="si">%s</span><span class="s2"> as prefix </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">prefix</span><span class="p">))</span>
120 <span class="gp">... </span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">resolve_string</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">result_xml</span><span class="p">,</span> <span class="n">context</span><span class="p">)</span>
122 <p>We demonstrate this in XSLT and use the following stylesheet as an example:</p>
123 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="n">xml_text</span> <span class="o">=</span> <span class="s2">"""</span><span class="se">\</span>
124 <span class="gp">... </span><span class="s2"><xsl:stylesheet version="1.0"</span>
125 <span class="gp">... </span><span class="s2"> xmlns:xsl="http://www.w3.org/1999/XSL/Transform"></span>
126 <span class="gp">... </span><span class="s2"> <xsl:include href="honk:test"/></span>
127 <span class="gp">... </span><span class="s2"> <xsl:template match="/"></span>
128 <span class="gp">... </span><span class="s2"> <test></span>
129 <span class="gp">... </span><span class="s2"> <xsl:value-of select="document('hoi:test')/*/*/text()"/></span>
130 <span class="gp">... </span><span class="s2"> </test></span>
131 <span class="gp">... </span><span class="s2"> </xsl:template></span>
132 <span class="gp">... </span><span class="s2"></xsl:stylesheet></span>
133 <span class="gp">... </span><span class="s2">"""</span>
135 <p>Note that it needs to resolve two URIs: <tt class="docutils literal">honk:test</tt> when compiling the XSLT
136 document (i.e. when resolving <tt class="docutils literal">xsl:import</tt> and <tt class="docutils literal">xsl:include</tt> elements) and
137 <tt class="docutils literal">hoi:test</tt> at transformation time, when calls to the <tt class="docutils literal">document</tt> function
138 are resolved. If we now register different resolvers with two different
139 parsers, we can parse our document twice in different resolver contexts:</p>
140 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="n">hoi_parser</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">XMLParser</span><span class="p">()</span>
141 <span class="gp">>>> </span><span class="n">normal_doc</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">StringIO</span><span class="p">(</span><span class="n">xml_text</span><span class="p">),</span> <span class="n">hoi_parser</span><span class="p">)</span>
143 <span class="gp">>>> </span><span class="n">hoi_parser</span><span class="o">.</span><span class="n">resolvers</span><span class="o">.</span><span class="n">add</span><span class="p">(</span> <span class="n">PrefixResolver</span><span class="p">(</span><span class="s2">"hoi"</span><span class="p">)</span> <span class="p">)</span>
144 <span class="gp">>>> </span><span class="n">hoi_doc</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">StringIO</span><span class="p">(</span><span class="n">xml_text</span><span class="p">),</span> <span class="n">hoi_parser</span><span class="p">)</span>
146 <span class="gp">>>> </span><span class="n">honk_parser</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">XMLParser</span><span class="p">()</span>
147 <span class="gp">>>> </span><span class="n">honk_parser</span><span class="o">.</span><span class="n">resolvers</span><span class="o">.</span><span class="n">add</span><span class="p">(</span> <span class="n">PrefixResolver</span><span class="p">(</span><span class="s2">"honk"</span><span class="p">)</span> <span class="p">)</span>
148 <span class="gp">>>> </span><span class="n">honk_doc</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">StringIO</span><span class="p">(</span><span class="n">xml_text</span><span class="p">),</span> <span class="n">honk_parser</span><span class="p">)</span>
150 <p>These contexts are important for the further behaviour of the documents. They
151 memorise their original parser so that the correct set of resolvers is used in
152 subsequent lookups. To compile the stylesheet, XSLT must resolve the
153 <tt class="docutils literal">honk:test</tt> URI in the <tt class="docutils literal">xsl:include</tt> element. The <tt class="docutils literal">hoi</tt> resolver cannot
155 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="n">transform</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">XSLT</span><span class="p">(</span><span class="n">normal_doc</span><span class="p">)</span>
156 <span class="gt">Traceback (most recent call last):</span>
157 <span class="c">...</span>
158 <span class="gr">lxml.etree.XSLTParseError</span>: <span class="n">Cannot resolve URI honk:test</span>
160 <span class="gp">>>> </span><span class="n">transform</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">XSLT</span><span class="p">(</span><span class="n">hoi_doc</span><span class="p">)</span>
161 <span class="gt">Traceback (most recent call last):</span>
162 <span class="c">...</span>
163 <span class="gr">lxml.etree.XSLTParseError</span>: <span class="n">Cannot resolve URI honk:test</span>
165 <p>However, if we use the <tt class="docutils literal">honk</tt> resolver associated with the respective
166 document, everything works fine:</p>
167 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="n">transform</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">XSLT</span><span class="p">(</span><span class="n">honk_doc</span><span class="p">)</span>
168 <span class="go">Resolved url honk:test as prefix honk</span>
170 <p>Running the transform accesses the same parser context again, but since it now
171 needs to resolve the <tt class="docutils literal">hoi</tt> URI in the call to the document function, its
172 <tt class="docutils literal">honk</tt> resolver will fail to do so:</p>
173 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="n">result</span> <span class="o">=</span> <span class="n">transform</span><span class="p">(</span><span class="n">normal_doc</span><span class="p">)</span>
174 <span class="gt">Traceback (most recent call last):</span>
175 <span class="c">...</span>
176 <span class="gr">lxml.etree.XSLTApplyError</span>: <span class="n">Cannot resolve URI hoi:test</span>
178 <span class="gp">>>> </span><span class="n">result</span> <span class="o">=</span> <span class="n">transform</span><span class="p">(</span><span class="n">hoi_doc</span><span class="p">)</span>
179 <span class="gt">Traceback (most recent call last):</span>
180 <span class="c">...</span>
181 <span class="gr">lxml.etree.XSLTApplyError</span>: <span class="n">Cannot resolve URI hoi:test</span>
183 <span class="gp">>>> </span><span class="n">result</span> <span class="o">=</span> <span class="n">transform</span><span class="p">(</span><span class="n">honk_doc</span><span class="p">)</span>
184 <span class="gt">Traceback (most recent call last):</span>
185 <span class="c">...</span>
186 <span class="gr">lxml.etree.XSLTApplyError</span>: <span class="n">Cannot resolve URI hoi:test</span>
188 <p>This can only be solved by adding a <tt class="docutils literal">hoi</tt> resolver to the original parser:</p>
189 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="n">honk_parser</span><span class="o">.</span><span class="n">resolvers</span><span class="o">.</span><span class="n">add</span><span class="p">(</span> <span class="n">PrefixResolver</span><span class="p">(</span><span class="s2">"hoi"</span><span class="p">)</span> <span class="p">)</span>
190 <span class="gp">>>> </span><span class="n">result</span> <span class="o">=</span> <span class="n">transform</span><span class="p">(</span><span class="n">honk_doc</span><span class="p">)</span>
191 <span class="go">Resolved url hoi:test as prefix hoi</span>
192 <span class="gp">>>> </span><span class="k">print</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">result</span><span class="p">)[:</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
193 <span class="go"><?xml version="1.0"?></span>
194 <span class="go"><test>hoi-TEST</test></span>
196 <p>We can see that the <tt class="docutils literal">hoi</tt> resolver was called to generate a document that
197 was then inserted into the result document by the XSLT transformation. Note
198 that this is completely independent of the XML file you transform, as the URI
199 is resolved from within the stylesheet context:</p>
200 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="n">result</span> <span class="o">=</span> <span class="n">transform</span><span class="p">(</span><span class="n">normal_doc</span><span class="p">)</span>
201 <span class="go">Resolved url hoi:test as prefix hoi</span>
202 <span class="gp">>>> </span><span class="k">print</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">result</span><span class="p">)[:</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
203 <span class="go"><?xml version="1.0"?></span>
204 <span class="go"><test>hoi-TEST</test></span>
206 <p>It may be seen as a matter of taste what resolvers the generated document
207 inherits. For XSLT, the output document inherits the resolvers of the input
208 document and not those of the stylesheet. Therefore, the last result does not
209 inherit any resolvers at all.</p>
211 <div class="section" id="i-o-access-control-in-xslt">
212 <h1>I/O access control in XSLT</h1>
213 <p>By default, XSLT supports all extension functions from libxslt and libexslt as
214 well as Python regular expressions through EXSLT. Some extensions enable
215 style sheets to read and write files on the local file system.</p>
216 <p>XSLT has a mechanism to control the access to certain I/O operations during
217 the transformation process. This is most interesting where XSL scripts come
218 from potentially insecure sources and must be prevented from modifying the
219 local file system. Note, however, that there is no way to keep them from
220 eating up your precious CPU time, so this should not stop you from thinking
221 about what XSLT you execute.</p>
222 <p>Access control is configured using the <tt class="docutils literal">XSLTAccessControl</tt> class. It can be
223 called with a number of keyword arguments that allow or deny specific
225 <div class="syntax"><pre><span></span><span class="gp">>>> </span><span class="n">transform</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">XSLT</span><span class="p">(</span><span class="n">honk_doc</span><span class="p">)</span>
226 <span class="go">Resolved url honk:test as prefix honk</span>
227 <span class="gp">>>> </span><span class="n">result</span> <span class="o">=</span> <span class="n">transform</span><span class="p">(</span><span class="n">normal_doc</span><span class="p">)</span>
228 <span class="go">Resolved url hoi:test as prefix hoi</span>
230 <span class="gp">>>> </span><span class="n">ac</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">XSLTAccessControl</span><span class="p">(</span><span class="n">read_network</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
231 <span class="gp">>>> </span><span class="n">transform</span> <span class="o">=</span> <span class="n">etree</span><span class="o">.</span><span class="n">XSLT</span><span class="p">(</span><span class="n">honk_doc</span><span class="p">,</span> <span class="n">access_control</span><span class="o">=</span><span class="n">ac</span><span class="p">)</span>
232 <span class="go">Resolved url honk:test as prefix honk</span>
233 <span class="gp">>>> </span><span class="n">result</span> <span class="o">=</span> <span class="n">transform</span><span class="p">(</span><span class="n">normal_doc</span><span class="p">)</span>
234 <span class="gt">Traceback (most recent call last):</span>
235 <span class="c">...</span>
236 <span class="gr">lxml.etree.XSLTApplyError</span>: <span class="n">xsltLoadDocument: read rights for hoi:test denied</span>
238 <p>There are a few things to keep in mind:</p>
240 <li>XSL parsing (<tt class="docutils literal">xsl:import</tt>, etc.) is not affected by this mechanism</li>
241 <li><tt class="docutils literal">read_file=False</tt> does not imply <tt class="docutils literal">write_file=False</tt>, all controls are
243 <li><tt class="docutils literal">read_file</tt> only applies to files in the file system. Any other scheme
244 for URLs is controlled by the <tt class="docutils literal">*_network</tt> keywords.</li>
245 <li>If you need more fine-grained control than switching access on and off, you
246 should consider writing a custom document loader that returns empty
247 documents or raises exceptions if access is denied.</li>
252 <hr class="footer" />
253 Generated on: 2020-01-29.
projects / platform / upstream / python-lxml.git / blob