1 <html><head><title>A Twisted Web Tutorial</title></head><body>
3 <h1>A Twisted Web Tutorial</h1>
5 <h2>Twisted Web -- The Tutorial</h2><ul>
8 <li>Gimmick -- Buffy quotes</li>
12 <em>Sweet (Once More With Feeling, season 6) -- Showtime</em>
13 <h2>Twisted Web</h2><ul>
14 <li>Web server, using Twisted<ul><li>Like Apache, Zope...</li>
17 <li>Serve static files</li>
21 <li>Other uses...</li>
25 <em>Giles (I Robot -- You Jane, season 1) -- There's a demon in the internet</em>
26 <h2>Short Example: Putting a Server Up</h2><ul>
27 <li>Here's all you need to know to bring up a server</li></ul>
30 % mktap --uid=33 --gid=33 web --path=/var/www/htdocs --port=80
31 % sudo twistd -f web.tap
33 <ul><li>The rest of the talk will explain what that means...</li>
35 <li>...and how to do more complicated things</li>
39 <em>Buffy (Once More, With Feeling, season 6) -- I've got a theory. It doesn't matter.</em>
40 <h2>Setup and Configuration Utilities</h2><ul>
45 <li>websetroot<ul><li>Won't be covered, unless there is time</li>
50 <em>Xander (The Harvest, season 1) -- crosses, garlic, stake through the heart</em>
51 <h2>Digression: What are TAPs</h2><ul>
52 <li>Pickled 'application configuration'</li>
54 <li>Object which contains all the information about application</li>
56 <li>The canonical way to represent configurations in Twisted</li>
58 <li>Machine editable</li>
62 <em>Master (The Wish, season 3) -- Behold the technical wonder</em>
64 <li>General usage</li>
66 <li>Flexibility and Power</li>
70 <em>Buffy (Bad Eggs, season 2) -- I'm gonna need a *big* weapon</em>
71 <h2>mktap web: Common Useful Options</h2><ul>
72 <li>--path: serve from given path</li>
74 <li>--port: listen on given port</li>
76 <li>--user: serve from users' directories and personal servers</li>
78 <li>--logfile: log to NCSA compatible logfile given</li>
80 <li>--processor: add a special processor for a given extension</li>
84 <em>Buffy (Bad Eggs, season 2) -- That's probably not gonna be the winning argument, is it?</em>
87 <li>Start a Twisted Application</li>
89 <li>Loads an instance of twisted.internet.app.Application from a file</li>
91 <li>Daemonizes, binds to appropriate ports, and starts the Twisted mainloop</li>
95 <em>Giles (Teacher's Pet, season 1) -- That's all he said? Fork Guy?</em>
96 <h2>What's a Resource?</h2><ul>
97 <li>Everything represented as twisted.web.resource.Resource</li>
99 <li>Important interface:<ul><li>getChild()</li>
106 <em>Sean (Go Fish, season 3) -- You're soakin' in it, bud.</em>
107 <h2>Resource Examples</h2><ul>
108 <li>Files<ul><li>MIME</li>
113 <li>Others<ul><li>Virtual hosts</li>
115 <li>User directories</li>
120 <em>Xander (As You Were, season 6) -- We have friends, family and demons</em>
121 <h2>Web Development</h2><ul>
122 <li>Processors<ul><li>Inherited from resource.Resource</li>
124 <li>Interpret files as code rather than data</li>
127 <li>Default processors<ul><li>.php -- default PHP</li>
129 <li>.cgi -- Common Gateway Interface</li>
131 <li>.rpy -- Correct way, Python scripting</li>
133 <li>.trp -- Resource pickles</li>
137 <li>You can also write your own</li>
141 <em>Xander (Family, season 5) -- That was a tangled web</em>
142 <h2>Custom Processor</h2><ul>
143 <li>A custom processor to handle Perl CGIs (in a module called PerlScript)</li></ul>
146 from twisted.web import static, twcgi
148 class PerlScript(twcgi.FilteredScript):
149 filter = '/usr/bin/perl' # Points to the perl parser
151 <ul><li>Use:<ul><li>mktap web --path=/home/nafai/public_html --processor=.pl=PerlScript.PerlScript</li>
156 <em>Tara (Family, season 4) -- There was the front of a camel</em>
157 <h2>Resource Scripting</h2><ul>
158 <li>Subclass resource.Resource</li>
160 <li>Write a render(self, request) method<ul><li>Return string for immediate response</li>
162 <li>Return NOT_DONE_YET and write to request</li>
165 <li>Create an .rpy file that sets 'resource' to an instance</li>
169 <em>Tara (Once More, With Feeling, season 6) -- You make me complete</em>
170 <h2>.rpy example</h2>
172 from twisted.web import resource as resourcelib
174 class MyGreatResource(resourcelib.Resource):
175 def render(self, request):
176 return "<html>foo</html>"
178 resource = MyGreatResource()
181 <em>Willow (Welcome to the Hellmouth, season 1) -- It's probably easy for you.</em>
182 <h2>Alternative Configuration Formats</h2><ul>
191 <em>Ben (The Gift, season 5) -- I wish there was another way</em>
192 <h2>Alternative Configuration Formats -- Python</h2><ul>
193 <li>Write manually</li>
195 <li>Just uses twisted.web API</li>
197 <li>Possible to do anything<ul><li>Write loops</li>
199 <li>Read other files</li>
201 <li>(not recommended) Define functions or classes</li>
206 <em>Buffy (The I In Team, season 4) -- But I've learned that it pays to be flexible in life.</em>
208 <h2>Python Configuration Example</h2><ul>
210 <li>Create application</li>
212 <li>Make it listen on port 80 for web requests...</li>
214 <li>...which should be served from /var/www/htdocs</li></ul>
217 from twisted.internet import app
218 from twisted.web import static, server
220 application = app.Application('web')
221 application.listenTCP(80,
222 server.Site(static.File("/var/www/htdocs")))
225 <em>Willow (The Pack, season 1) -- It's simple, really.</em>
227 <h2>Bannerfish -- A Case Study in Deployment</h2><ul>
228 <li>Serves banner ads</li>
230 <li>Has algorithms to maintain randomness and fairness</li>
232 <li><a href="http://itamarst.org/software/bannerfish/">http://itamarst.org/software/bannerfish/</a></li>
234 <li>But how to deploy?</li>
238 <em>Xander (Halloween, season 2) -- Let's move out.</em>
239 <h2>Bannerfish -- Standalone tap</h2><ul>
240 <li>mktap bannerfish</li>
242 <li>twistd -f bannerfish.tap</li>
244 <li>For full list of options<ul><li>mktap --help bannerfish</li>
249 <em>Ethan (Halloween, season 2) -- Don't wish to blow my own trumpet, but --</em>
250 <h2>Bannerfish -- Standalone tap (behind reverse proxy)</h2><ul>
251 <li>mktap bannerfish --port 81 --proxyhost=example.com</li>
253 <li>twistd -f bannerfish.tap</li>
255 <li>Now can work on internal server behind firewall</li>
257 <li>If main server is Twisted Web, following Resource script will serve from bannerfish</li></ul>
260 resource = proxy.ReverseProxyResource('localhost', 81, '/')
263 <em>Buffy (Halloween, season 2) -- You're sweet. A terrible liar, but sweet.</em>
264 <h2>Bannerfish -- Standalone Python</h2>
266 from twisted.internet import app
267 from twisted.cred import authorizer
268 from twisted.web import server
269 from bannerfish import service
271 application = app.Application("bannerfish")
272 auth = authorizer.DefaultAuthorizer(app)
273 svc = service.BannerService('/var/bannerfish',
274 "bannerfish", application, auth)
275 site = server.Site(svc.buildResource(None, None))
276 application.listenTCP(80, site)
279 <em>Spike (Halloween, season 2) -- Shaking. Terrified. Alone. Lost little lamb.</em>
280 <h2>Bannerfish -- /etc/twisted-web/local.d Drop In</h2>
282 from twisted.cred import authorizer
283 from bannerfish import service
285 auth = authorizer.DefaultAuthorizer(app)
286 svc = service.BannerService('/var/bannerfish',
287 "bannerfish", application, auth)
288 resource = svc.buildResource(None, None)
289 default.addChild("bannerfish", resource)
292 <em>Cordelia (Halloween, season 2) -- Well, I guess you better get them back to their parents.</em>
293 <h2>Bannerfish -- Resource Script</h2>
295 from twisted.cred import authorizer
296 from twisted.internet import app
297 from bannerfish import service
299 application = registry.getComponent(app.Application)
300 auth = authorizer.DefaultAuthorizer(application)
301 svc = service.BannerService('/var/bannerfish',
302 "bannerfish", application, auth)
303 resource = svc.buildResource(None, None)
305 <ul><li>But see later, about registry</li>
309 <em>Xander (Innocence, season 2) -- They like to see the big guns.</em>
310 <h2>Bannerfish -- Distributed (Slave)</h2>
313 from twisted.internet import application
314 from twisted.cred import authorizer
315 from twisted.web import server
316 from bannerfish import service
317 application = app.Application("bannerfish")
318 auth = authorizer.DefaultAuthorizer(application)
319 svc = service.BannerService('/var/bannerfish',
320 "bannerfish", application, auth)
321 site = server.Site(svc.buildResource(None, None))
322 fact = pb.BrokerFactory(site)
323 site = server.Site(root)
324 application.listenUNIX('/var/run/bannerfish', fact)
326 <h2>Bannerfish -- Distributed (Master, Resource Script)</h2>
329 from twisted.web import distrib
331 resource = distrib.ResourceSubscription('unix',
332 '/var/run/bannerfish')
335 <em>Oz (Innocence, season 2) -- So, do you guys steal weapons from the Army a lot?</em>
336 <h2>Bannerfish -- Other options</h2><ul>
337 <li>Mix and match possible</li>
339 <li>Can serve same content multiple ways simultaneously</li>
341 <li>Might be useful as a way to serve same ads different ways</li>
343 <li>...or serve ads from several bannerfish servers...</li>
345 <li>...each deployed differently.</li>
349 <em>Buffy (Tabula Rasa, season 6) -- I'm like a superhero or something</em>
350 <h2>Bannerfish -- Conclusions</h2><ul>
351 <li>What are the tradeoffs?</li>
353 <li>Everything works in the simple cases</li>
355 <li>Not enough complicated cases to have data</li>
357 <li>Luckily, easy to move between them</li>
359 <li>Motto -- move deployment choices as late as possible</li>
363 <em>Giles (Killed By Death, season 2) -- Simple enough, but, but</em>
364 <h2>Further Reading</h2><ul>
365 <li>Short overview -- doc/howto/web-overview.html</li>
367 <li>In depth review -- doc/howto/using-twistedweb.html</li>
369 <li>Using databases -- doc/howto/enterprise.html</li>
371 <li>Deferred execution -- doc/howto/deferred.html</li>
373 <li>Resource script examples -- doc/examples/*.rpy.py</li>
377 <em>Giles (I Was Made to Love You, Season 5) -- There's an enormous amount of research we should do before -- no I'm lying</em>
379 <em>Vampire Willow (Dopplegangland, season 3): Questions? Comments?</em>
380 <h2>Bonus Slides</h2>
381 <em>Xander (The Dark Age, season 2) -- A bonus day of class plus Cordelia.</em>
383 <h2>Python Configuration -- Hints</h2><ul>
384 <li>Working with persistence</li>
390 <li>Virtual Hosts</li>
394 <em>Buffy (Phases, season 2) -- Have you dropped any hints?</em>
395 <h2>Python Configuration -- Persistence</h2><ul>
396 <li>Don't define functions or classes</li>
398 <li>Don't modify class attributes</li>
402 <em>Spike (Once More, With Feeling) -- Let me rest in peace</em>
403 <h2>Python Configuration -- Processors</h2>
406 from twisted.internet import app
407 from twisted.web import static, server
408 from twisted.web import twcgi
410 root = static.File("/var/www")
411 root.processors = {".cgi": twcgi.CGIScript}
412 application = app.Application('web')
413 application.listenTCP(80, server.Site(root))
416 <em>Manny (Doublemeat Palace, season 6) -- It's a meat process</em>
417 <h2>Python Configuration -- Indices</h2>
420 root = static.File("/var/www")
421 root.indices = ['index.rpy', 'index.html']
424 <em>Willow (Buffy vs. Dracula, season 5) -- Labelling your amulets and indexing your diaries</em>
425 <h2>Python Configuration -- Virtual Hosts</h2>
428 from twisted.web import vhost
429 default = static.File("/var/www")
430 foo = static.File("/var/foo")
431 root = vhost.NamedVirtualHost(default)
432 root.addHost('foo.com', foo)
435 <em>Fritz (I Robot, You Jane, season 1) -- The only reality is virtual.</em>
436 <h2>Python Configuration -- uber example</h2>
439 from twisted.internet import app
440 from twisted.web import static, server, vhost, script
442 default = static.File("/var/www")
443 default.processors = {".rpy", script.ResourceScript}
444 root = vhost.NamedVirtualHost(default
445 foo = static.File("/var/foo")
446 foo.indices = ['index.xhtml', 'index.html']
447 root.addHost('foo.com', foo)
448 site = server.Site(root)
449 application = app.Application('web')
450 application.listenTCP(80, site, interface='127.0.0.1')
453 <em>Buffy (Potential, season 7) -- It was putting a lot of stock in that uber-vamp</em>
454 <h2>Python Configuration -- Splitting With Reverse Proxy</h2><ul>
455 <li>Original use case - SVN</li></ul>
458 from twisted.web import proxy
461 proxy.ReverseProxyResource('localhost',
465 <em>Buffy (Once More, With Feeling, season 6 -- So I will walk through the fire</em>
466 <h2>mktap examples</h2><ul>
469 <li>mktap web --path=/var/www --logfile=/var/log/twistedweb.log</li>
471 <li>mktap web --port=80 --path=/var/www --mime-type=text/plain</li>
473 <li>mktap web --path=/home/nafai/public_html --processor=.pl=PerlProcessor.PerlProcessor --index=index.pl</li>
477 <em>Anya (I Was Made to Love You, season 4) -- You can also see the website I designed for the magic shop</em>
478 <h2>mktap examples (cont'd)</h2><ul>
479 <li>mktap web --users</li>
481 <li>mktap web --ignore-ext=.cgi</li>
483 <li>mktap web --index=index.cgi --index=index.rpy --index=index.html</li>
487 <em>Buffy (Once More, With Feeling, season 6) -- All the twists and bends</em>
488 <h2>mktap examples (alternate formats)</h2><ul>
489 <li>mktap --type=source web</li>
491 <li>mktap --type=xml web</li>
495 <em>Tara (Seeing Red, season 6) -- It isn't written in any ancient language we could identify.</em>
496 <h2>mktap examples (setting uid)</h2><ul>
497 <li>mktap --uid=33 web</li>
499 <li>mktap --gid=33 web</li>
501 <li>Uid/Gid of www-data on Debian systems</li>
503 <li>Not possible to use username</li>
505 <li>More about this later</li>
509 <em>Buffy (Who Are You?, season 4) -- I would be Buffy</em>
511 <h2>twistd examples</h2><ul>
512 <li>twistd -f web.tap -l /var/log/twisted.log</li>
514 <li>twistd -f web.tap --pidfile /var/run/web.pid</li>
516 <li>twistd -x web.tax<ul><li>For mktap --type=xml</li>
519 <li>twistd -s web.tas<ul><li>For mktap --type=source</li>
524 <em>Xander (Teacher's Pet, season 1) -- How come *that* never came up?</em>
525 <h2>Shutting down twistd</h2><ul>
526 <li>On Unix (in general): <ul><li>kill `cat twistd.pid`</li>
529 <li>On Windows: <ul><li>Cannot daemonize on Windows, so just run twistd in a command prompt</li>
531 <li>Switch to the command prompt, and press Control-C</li>
536 <em>Buffy (Prophecy Girl, season 1) -- I don't wanna die.</em>
537 <h2>Shutdown TAPs</h2><ul>
538 <li>Since TAPs store persistent data for an application, a 'shutdown' TAP is created on twistd shutdown</li>
540 <li>You'll often want to start your Twisted application on subsequent runs with the shutdown TAP</li>
544 <em>Headstone (The Gift, season 5) -- She saved the world. A lot.</em>
545 <h2>twistd and security</h2><ul>
546 <li>When twistd is run as root, it will shed root privileges for the uid and gid of either the user that created the TAP or those specified on the mktap commandline.</li>
550 <em>Buffy (Dopplegangland, season 3) -- I think it's good to be reliable</em>
552 <h2>Resource Call Examples</h2><ul>
553 <li>/foo/bar/baz gets converted to:</li></ul>
556 site.getChild('foo', request
557 ).getChild('bar', request
558 ).getChild('baz', request
562 <em>Willow/Tara (Afterlife, Part 2, season 6) -- Child of words, hear thy makers</em>
563 <h2>Resource Call Examples (cont'd)</h2><ul>
564 <li>/foo/bar/baz/ gets converted to:</li></ul>
567 site.getChild('foo', request
568 ).getChild('bar', request
569 ).getChild('baz', request
570 ).getChild('', request
574 <em>Buffy (Gone, season 6) -- Stop trying to see me.</em>
575 <h2>Distributed Servers -- Theory</h2><ul>
576 <li>Master is a resource</li>
578 <li>Slave is a server</li>
580 <li>Same server can have both master and slave parts</li>
584 <em>Anya (Once More, With Feeling, season 6) -- I've got a theory, it could be bunnies</em>
585 <h2>Distributed Servers -- Manually</h2>
587 from twisted.internet import app, protocol
588 from twisted.web import server, distrib, static
589 from twisted.spread import pb
591 application = app.Application("silly-web")
592 # The "master" server
593 site = server.Site(distrib.ResourceSubscription('unix', '.rp'))
594 application.listenTCP(19988, site)
596 fact = pb.BrokerFactory(distrib.ResourcePublisher(
597 server.Site(static.File('static'))))
598 application.listenUNIX('./.rp', fact)
601 <em>Buffy (Some Assembly Required, season 2) -- Men dig up the corpses and the women have the babies.</em>
602 <h2>Distributed Servers -- Manual (cont'd)</h2><ul>
603 <li>First Server</li></ul>
606 from twisted.internet import app, protocol
607 from twisted.web import server, distrib, static, vhost
608 from twisted.spread import pb
610 application = app.Application("ping-web")
612 default = static.File("/var/www/foo")
613 root = vhost.NamedVirtualHost(default)
614 root.addVhost("foo.com", default)
615 bar = distrib.ResourceSubscription('unix', '.bar')
616 root.addVhost("bar.com", bar)
618 fact = pb.BrokerFactory(static.Site(default))
619 site = server.Site(root)
620 application.listenTCP(19988, site)
621 application.listenUNIX('./.foo', fact)
624 <em>Buffy (Welcome to the Hellmouth, season 1) -- Now, we can do this the hard way, or...</em>
625 <h2>Distributed Servers -- Manual (cont'd 2)</h2><ul>
626 <li>Second Server</li></ul>
629 from twisted.internet import app, protocol
630 from twisted.web import server, distrib, static, vhost
631 from twisted.spread import pb
633 application = app.Application("pong-web")
635 foo = distrib.ResourceSubscription('unix', '.foo')
636 root = vhost.NamedVirtualHost(foo)
637 root.addVhost("foo.com", foo)
638 bar = static.File("/var/www/bar")
639 root.addVhost("bar.com", bar)
641 fact = pb.BrokerFactory(static.Site(bar))
642 site = server.Site(root)
643 application.listenTCP(19989, site)
644 application.listenUNIX('./.bar', fact)
647 <em>Buffy (Welcome to the Hellmouth, season 1) -- ...well, actually there's just the hard way.</em>
648 <h2>Distributed Servers -- User Directory</h2><ul>
651 <li>Child that looks like 'moshez' -- ~moshez/public_html </li>
653 <li>Child that looks like 'moshez.twistd' -- moshez's personal server</li>
657 <em>Master (The Wish, season 3) -- Mass production!</em>
658 <h2>Distributed Servers -- User Directory Server</h2><ul>
659 <li>With mktap: mktap web --user</li>
661 <li>With Python configuration</li></ul>
664 from twisted.internet import app
665 from twisted.web import static, server, distrib
667 root = static.File("/var/www")
668 root.putChild("users", distrib.UserDirectory())
669 site = server.Site(root)
670 application = app.Application('web')
671 application.listenTCP(80, site)
674 <em>Richard (Reptile Boy, season 2) -- In his name.</em>
675 <h2>Distributed Servers -- Personal Servers</h2><ul>
676 <li>With mktap: mktap web --personal ...</li>
678 <li>With Python configuration</li></ul>
681 from twisted.internet import app
682 from twisted.web import static, server, distrib
683 from twisted.spread import pb
685 root = static.File("/home/moshez/twistd")
686 site = server.Site(root)
688 fact = pb.BrokerFactory(distrib.ResourcePublisher(site))
689 application.listenUNIX('/home/moshez/.twisted-web-pb', fact)
692 <em>Giles (Bargaining, season 6) -- It's my personal collection</em>
693 <h2>Debian Configuration</h2><ul>
694 <li>Inside twisted-web package</li>
696 <li>Goal -- look like other web servers to users</li>
698 <li>Goal -- interoperate easily</li>
700 <li>Goal -- allow users to avoid modifying files</li>
704 <em>Buffy (Bad Girls, season 3) -- We can help each other.</em>
705 <h2>Debian Configuration -- Usage</h2><ul>
706 <li>Changing port -- edit /etc/twisted-web/ports</li>
708 <li>Want to use behind reverse proxy? Use rptwisted</li>
710 <li>Change anything else -- drop files in /etc/twisted-web/local.d</li>
714 <em>Faith (Home Coming, season 3) -- we'll use 'em</em>
715 <h2>Debian Configuration -- Drop In Examples</h2>
717 from twisted.web import static
720 vhostDir = '/var/www/vhost/'
722 for file in os.listdir(vhostDir):
723 root.addHost(file, static.File(os.path.join(vhostDir, file)))
726 <em>Buffy (The Freshman, season 4) -- I just thought I'd drop in</em>
727 <h2>Debian Configuration -- Drop In Examples (cont'd)</h2>
729 from twisted.web import script, static
731 default.processors['.rpy'] = script.ResourceScript
732 default.ignoreExt('rpy')
735 <em>Riley (As You Were, season 6) -- Sorry to just drop in on you</em>
736 <h2>Debian Configuration -- Drop In Examples (cont'd 2)</h2>
738 from twisted.web import vhost
740 default.putChild('vhost', vhost.VHostMonsterResource())
743 <em>Sam (As You Were, season 6) -- a hairy night drop into hostile territory</em>
744 <h2>twistedmatrix.com Configuration</h2><ul>
745 <li>Some highlights</li></ul>
749 indexNames = ['index', 'index.html', 'index.xhtml', 'index.rpy','index.cgi']
751 root.putChild('mailman', twcgi.CGIDirectory('/usr/lib/cgi-bin'))
752 root.putChild('users', distrib.UserDirectory())
753 root.putChild('cgi-bin', twcgi.CGIDirectory('/usr/lib/cgi-bin'))
754 root.putChild('doc', static.File('/usr/share/doc'))
756 uid = pwd.getpwnam('www-data')[2]
757 gid = grp.getgrnam('www-data')[2]
759 top = rewrite.RewriterResource(root, rewrite.tildeToUsers)
761 application = app.Application("web", uid=uid, gid=gid)
764 <em>Xander (The Witch, season 1) -- May all lesser cretins bow before me.</em>
765 <h2>Apache vs. Twisted Web</h2><ul>
766 <li>Apache is faster</li>
768 <li>Apache -- Threads/processes model</li>
770 <li>Twisted -- async model</li>
772 <li>Apache -- has C security holes (buffer overflows)</li>
774 <li>Twisted -- easy to set up</li>
776 <li>Twisted -- built in Python programmability</li>
780 <em>Willow (Buffy vs. Dracular, season 5) -- I think we've just put our finger on why we're the sidekicks</em>
781 <h2>Apache/Twisted Web Integration</h2><ul>
784 <li>Apache's reverse proxy works well</li>
786 <li>Easy to have a site which is partially managed by Apache</li>
788 <li>Documentation has examples of configurations</li>
792 <em>Xander (What's My Line, season 2) -- Angel's our friend! Except I don't like him.</em>
793 <h2>Zope vs. Twisted Web</h2><ul>
794 <li>Zope -- fully editable through the web</li>
796 <li>Zope -- uses ZODB, not file system</li>
798 <li>Twisted -- can integrate with other protocols easily</li>
800 <li>Twisted -- extension code has much less overhead</li>
804 <em>Willow (Dopplegangland, season 3) -- Competition is natural and healthy</em>
805 <h2>Zope/Twisted Web Integration</h2><ul>
806 <li>Possible to use Twisted as Zope's network layer</li>
808 <li>Hackish with Zope2</li>
810 <li>Easier with Zope3</li>
814 <em>Snyder (Dopplegangland, season 3) -- It's a perfect match.</em>
815 <h2>Zope/Twisted Web Integration (cont'd)</h2><ul>
816 <li>Less direct -- use Apache</li>
818 <li>Reverse proxy parts to Zope</li>
820 <li>Reverse proxy parts to Twisted Web</li>
824 <em>Wesley (Dopplegangland, season 3) -- Still a little sloppy, though</em>
825 <h2>Applications Appropriate for Twisted Web</h2><ul>
830 <li>Web/other protocol chat systems</li>
834 <em>Sweet (Once More, With Feeling) -- Why don't you come and play?</em>
835 <h2>Behind Reverse Proxy</h2><ul>
836 <li>Sometimes, we want Twisted to pretend to be another host/port</li>
838 <li>Reverse proxies, NATs, etc.</li>
840 <li>Reverse proxy to /vhost/http/<host:port>/</li>
842 <li>Make sure root has a child called vhost of type twisted.web.vhost.VirtualHostingMonster</li>
846 <em>Jenny (I Robot -- You Jane, season 1) -- The divine exists in cyberspace</em>
847 <h2>Rewrite Rules</h2><ul>
848 <li>Change a URL to another</li>
850 <li>Useful for different treatment from outside resources</li>
852 <li>Wraps a resource</li>
856 <em>Spike (What's My Line, season 2) -- Read it again.</em>
857 <h2>Rewrite Rules -- Example</h2>
860 root = static.File("/var/www")
861 root.putChild("users", distrib.UserDirectory())
862 root = rewrite.RewriterResource(root, rewrite.tildeToUsers)
864 <ul><li>Now, /~moshez/ works</li>
868 <em>Spike (What's My Line, season 2) -- I think it's just enough kill.</em>
869 <h2>websetroot</h2><ul>
870 <li>Used to change what the root of the server points to</li>
872 <li>Set it to a Resource contained either in a Python source file or a Pickle file</li>
876 <em>Manny (DoubleMeat Palace, season 6) -- We have a lot of turnover here</em>
877 <h2>Sample websetroot command lines</h2><ul>
878 <li>websetroot -p 80 -f web.tap --script rootResource.py</li>
880 <li>websetroot -p 8080 -f web.tap --pickle rootPickle</li>
884 <em>Manny (DoubleMeat Palace, season 6) -- You can toss it</em>
892 <li>No smooth reloading</li>
898 <em>Bob (Zeppo, season 3) -- He hasn't been initiated.</em>
899 <h2>Special Bonus - How to Configure <user>.example.com</h2>
902 from twisted.web import resource, error, distrib
903 from twisted.protocols import http
905 class UserNameVirtualHost(resource.Resource):
907 def __init__(self, default, tail):
908 resource.Resource.__init__(self)
909 self.default = default
913 def _getResourceForRequest(self, request):
914 host=request.getHeader('host')
915 if host.endswith(tail):
916 username = host[:-len(tail)]
919 if self.users.has_key(username):
920 return self.users[username]
922 (pw_name, pw_passwd, pw_uid, pw_gid, pw_gecos, pw_dir,
923 pw_shell) = pwd.getpwnam(username)
925 return error.ErrorPage(http.NOT_FOUND,
927 "The user %s was not found on this system." %
929 twistdsock = os.path.join(pw_dir, ".twistd-web-pb")
930 rs = distrib.ResourceSubscription('unix',twistdsock)
931 self.users[username] = rs
934 def render(self, request):
935 resrc = self._getResourceForRequest(request)
936 return resrc.render(request)
938 def getChild(self, path, request):
939 resrc = self._getResourceForRequest(request)
940 request.path=request.path[:-1]
941 request.postpath=request.uri.split('/')[1:]
942 print request, request.path, request.postpath
943 return resrc.getChildForRequest(request)
946 <em>Morgan (The Puppet Show, season 1) -- Weird? What d'you mean?</em>
947 <h2>Special Bonus - How to Configure <user>.example.com (cont'd)</h2><ul>
948 <li>Put above in a module (say, uservhost)</li>
950 <li>Use following configuration file</li></ul>
953 from twisted.internet import app
954 from twisted.web import server
957 root = UserNameVirtualHost("www", "example.com")
958 site = server.Site(root)
959 application = app.Application('web')
960 application.listenTCP(80, site)
963 <em>Snyder (The Puppet Show, season 1) -- You need to integrate into this school, people.</em>
964 <h2>Using the Twisted Registry</h2><ul>
965 <li>Use especially in Resource Scripts</li>
967 <li>Save persistent information</li>
969 <li>Uses Twisted's Componentized to be extensible</li>
973 <em>Angel (Helpless, season 3) -- I wanted to keep it safe</em>
974 <h2>Using the Twisted Registry -- example</h2>
977 from twisted.web import distrib
979 resource = registry.getComponent(distrib.UserDirectory)
981 resource = distrib.UserDirectory()
982 registry.setComponent(distrib.UserDirectory, resource)
985 <em>Paul (The Freshman, season 4) -- Do you know where they're distributing the [...] applications?</em>
986 <h2>Using the Twisted Registry -- problems</h2><ul>
987 <li>In most cases -- need to write a custom class</li>
989 <li>Saves data in-memory</li>
991 <li>Won't work as expected unless -shutdown taps are used</li>
995 <em>Anya (Once More, With Feeling, season 6) -- The only trouble is [pause] I'll never tell.</em>
996 <h2>Alternative Configuration Formats -- XML</h2><ul>
997 <li>Can be generated from mktap</li>
999 <li>Editable with any XML editor</li>
1001 <li>Easy to do easy things<ul><li>Change a port</li>
1004 <li>Nontrivial to do hard things<ul><li>Bind to specific IP</li>
1009 <em>Buffy (Once More, With Feeling, season 6) -- To fit in in this glittering world.</em>
1010 <h2>Alternative Configuration Formats -- XML -- example</h2>
1011 <pre class="python">
1012 <?xml version="1.0"?>
1014 <instance class="twisted.internet.app.Application" reference="1">
1017 <string role="key" value="tcpPorts" />
1020 <int value="80" />
1021 <instance class="twisted.web.server.Site">
1024 <string role="key" value="resource" />
1025 <instance class="twisted.web.static.File">
1028 <string role="key" value="path" />
1029 <string value="/var/www" />
1044 <em>Natalie (Teacher's Pet, season 1) -- There's nothing ugly about these creatures</em>
1045 <h2>Alternative Configuration Formats -- Source</h2><ul>
1046 <li>Can be generated from mktap</li>
1048 <li>Editable with any Python source editor</li>
1050 <li>Easy to do easy things<ul><li>Change a port</li>
1053 <li>Nontrivial to do hard things<ul><li>Bind to specific IP</li>
1058 <em>Willow/Giles/Xander (Primeval, season 4) -- You could never hope to grasp the source</em>
1059 <h2>Alternative Configuration Formats -- Source -- Example</h2>
1060 <pre class="python">
1062 Instance('twisted.internet.app.Application',{
1067 Instance('twisted.web.server.Site',
1069 resource=Instance('twisted.web.static.File',{
1079 <em>Tara (Family, season 5) -- You learn her source, and, uh we'll introduce her to her insect reflection</em>
1080 <h2>Twisted Web - Beginnings</h2>
1081 <pre class="python">
1083 <glyphAtWork> the http server was so we could say "Web!" if we ever did
1084 a freshmeat announcement
1085 <glyphAtWork> this makes people excited
1087 <ul><li>Turned out he was right</li>
1091 <em>Dawn (Get It Done, season 7) -- I think it's an origin myth.</em>
1092 <h2>Woven Overview</h2><ul>
1093 <li>HTML templates</li>
1095 <li>Model/View/Controller architecture</li>
1097 <li>Integrated with deferred</li>
1099 <li>Classical systems work badly with async</li>
1101 <li>More -- beyond scope of this tutorial</li>
1105 <em>Razor (Bargaining, season 6) -- A pretty toy</em>