Twisted.web2 Deployment

  1. Standalone HTTP
  2. HTTP behind Apache2
  3. HTTP behind Apache1
  4. SCGI
  5. FastCGI
  6. CGI

There are a number of possibilities for deploying twisted.web2: as standalone HTTP[S] server, HTTP proxied behind another server, SCGI, FastCGI, or CGI.

Deploying as a standalone HTTP/HTTPS server is by far the simplest. Unless you have a reason not to, it is recommended that you choose this option. However, many people already run web servers on their computer and are not willing or able to completely blow it away and replace it with twisted.web2. The next best option is to run twisted.web2 as a server proxied behind your existing webserver, using either HTTP or SCGI.

Standalone HTTP

For completeness, here is a simple standalone HTTP server again.

from twisted.web2 import server, channel, static

# For example, serve the /tmp directory
toplevel = static.File("/tmp")
site = server.Site(toplevel)

# Start up the server
from twisted.application import service, strports
application = service.Application("demoserver")
s = strports.service('tcp:8080', channel.HTTPFactory(site))
s.setServiceParent(application)
Listing 1: A standalone HTTP server - ../examples/deployment/standalone.tac

HTTP behind Apache2

If you use HTTP proxying, you must inform twisted.web2 of the real URL it is being accessed by, or else any URLs it generates will be incorrect. You can do this via the AutoVHostURIRewrite resource when using apache2 as the main server.

On the apache side, configure as follows. Apache automatically sends the original host in the X-Forwarded-Host header, and the original remote IP address in the X-Forwarded-For header. You must additionally send along the original path, and the original scheme.

For proxying a subdirectory:

<Location /whatever/>
ProxyPass http://localhost:8538/
RequestHeader set X-App-Location /whatever/
RequestHeader set X-App-Scheme http

</Location>
    

Or, for serving an entire HTTPS virtual host:

<VirtualHost myip:443>
ServerName example.com
ProxyPass / http://localhost:8538/
RequestHeader set X-App-Location /
RequestHeader set X-App-Scheme https
</VirtualHost>
    

Now, on the twisted.web2 side

from twisted.web2 import server, channel, static, vhost

# For example, serve the /tmp directory
toplevel = static.File("/tmp")
# Use the automatic uri rewriting based on apache2 headers
toplevel = vhost.AutoVHostURIRewrite(toplevel)
site = server.Site(toplevel)

# Start up the server
from twisted.application import service, strports
application = service.Application("demoserver")
s = strports.service('tcp:8538', channel.HTTPFactory(site))
s.setServiceParent(application)
Listing 2: Behind Apache 2 - ../examples/deployment/apache2.tac

HTTP behind Apache1

Apache 1 doesn't provide the X-Forwarded-Host or X-Forwarded-For headers, or the ability to set custom headers in the outgoing proxy request. Therefore, you must provide that information to twisted.web2 directly. This is accomplished by the VHostURIRewrite resource.

Setup apache as follows:

<VirtualHost myip>
ServerName example.com
ProxyPass /foo/ http://localhost:8538/
</VirtualHost>
    

And twisted like so

from twisted.web2 import server, channel, static, vhost

# For example, serve the /tmp directory
toplevel = static.File("/tmp")
# Add the rewriter.
toplevel = vhost.VHostURIRewrite("http://myhostname.com/foo/", toplevel)
site = server.Site(toplevel)

# Start up the server
from twisted.application import service, strports
application = service.Application("demoserver")
s = strports.service('tcp:8538:interface=127.0.0.1', channel.HTTPFactory(site))
s.setServiceParent(application)
Listing 3: Behind Apache 1 - ../examples/deployment/apache1.tac

Because vhost.VHostURIRewrite can exist anywhere in the resource tree, you can have multiple applications running on a single twisted port by making them siblings of a root resource and referencing their full path in the ProxyPass directive.

Setup apache as follows:

<VirtualHost foo.myhostname.com>
ProxyPass / http://localhost:8538/foo/
ServerName example.com
</VirtualHost>

<VirtualHost bar.myhostname.com>
ProxyPass / http://localhost:8538/bar/
ServerName example.com
</VirtualHost>
    

And twisted like so

from twisted.web2 import server, channel, resource, static, vhost

# For example, server the /tmp/foo directory
foo_toplevel = static.File("/tmp/foo")
# And the /tmp/bar directory
bar_toplevel = static.File("/tmp/bar")
# Add the rewriters:
foo_toplevel = vhost.VHostURIRewrite("http://foo.myhostname.com/",
      foo_toplevel)
bar_toplevel = vhost.VHostURIRewrite("http://bar.myhostname.com/",
      bar_toplevel)

toplevel = resource.Resource()
toplevel.putChild('foo', foo_toplevel)
toplevel.putChild('bar', bar_toplevel)
site = server.Site(toplevel)

# Start up the server
from twisted.application import service, strports
application = service.Application("demoserver")
s = strports.service('tcp:8538:interface=127.0.0.1', channel.HTTPFactory(site))
s.setServiceParent(application)
Listing 4: Multiple hosts behind Apache 1 - ../examples/deployment/apache1_twohosts.tac

SCGI

SCGI is an alternative to HTTP proxying. SCGI should work instead of HTTP proxying from servers which support it. Additionally, if all you have access to from the web server is CGI, but are able to run long-running processes, you can use the cgi2scgi C program to channel CGI requests to your twisted.web2 SCGI port. This won't be as efficient as mod_scgi or http proxying, but it will be much better than using twisted directly as a CGI.

FIXME:Someone who has installed mod_scgi in apache should write a bit on it.

Configure Twisted as follows

from twisted.web2 import server, channel, static

# For example, serve the /tmp directory
toplevel = static.File("/tmp")
site = server.Site(toplevel)

# Start up the server
from twisted.application import service, strports
application = service.Application("demoserver")
s = strports.service('tcp:3000', channel.SCGIFactory(site))
s.setServiceParent(application)
Listing 5: An SCGI Server - ../examples/deployment/scgi.tac

FastCGI

FastCGI is another popular way to run a web application. Blah blah.

CGI

CGI is the worst possible deployment environment, yet in some cases it may be all that is possible. It allows only a single request to be served from a process, so any kind of in-memory storage is impossible. Also, the overhead of starting up a new python interpreter for every request can get quite high. You should only consider using it if your hosting provider does not allow you to keep a process running.

However, if it's your only choice, you can deploy a twisted.web2 app using it. Unlike the other examples, where we create a .tac file for running with twistd, in this case, a standalone python script is necessary

#!/usr/bin/env python
  from twisted.web2 import channel, server, static
  toplevel = static.File(/tmp)
  site = server.Site(toplevel)
  channel.startCGI(site)

Index

Version: 8.1.0