Stories
Slash Boxes
Comments

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Matts (1087)

Matts
  (email not shown publicly)
I work for MessageLabs [messagelabs.com] in Gloucester UK, except I live in Kitchener, ON, Canada. I write spam filters, MTA software, high performance network software, string matching algorithms, and other cool stuff mostly in Perl and C. We're always hiring so feel free to email me (no telecommuters though - I'm the only exception :-).

Journal of Matts (1087)

Friday August 25, 2006
05:18 PM

AxKit2 Status: Fastest perl httpd?

[ #30758 ]

Ax2 is turning into a pretty sweet pure perl httpd (whether you want XML processing or not).

I've been working a bit this week on a couple of things - getting the uri_to_filename translation working correctly (thanks to Jörg for a lot of work there) according to how Apache works, and then lots on performance and stability.

It serves about 500 requests/second here on localhost transfers (benchmarking locally with apachebench) with a 2K file. That's not very impressive, but then most httpds don't really hit their maximum requests-per-second transfers anyway - they are bottlenecked by numbers of connections and the speed of the network.

So in testing large files, the throughput causes apachebench to overflow integers. It does about 110MB/sec throughput. That's enough throughput to saturate even the fattest of pipes (well ok, it won't saturate an OC48, but if you're doing that much throughput you can probably afford another box!).

This sort of throughput is achieved using AIO. All the stat()s and open() calls are asynchronous (using the excellent IO::AIO) when I'm doing this benchmark.

Now in theory I could build a faster httpd if I just removed a lot of overhead (logging, and all the nice hooks) but those things are worth having. It has hooks like apache has for uri_translation, auth, fixup, etc, making web app development very simple.

As a long time mod_perl user (I realised I've been using mod_perl for 8 years! yikes) I value its flexibility but always thought things could be a bit simpler. And I find Ax2 simpler to develop for. Partly that's the "I wrote it" factor, but it's so much easier to stop and start with "ctrl-c", then "./axkit" than it ever was to do that via "apachectl stop; sleep 5; apachectl start"... and make test starts the server much faster than Apache::Test can (because we can hook right into the module). Debugging can be done with perl -d axkit. Profiling with perl -MDevel::Profiler axkit. Lots of niceties for those developing web apps.

Now how can I get the Jifty and Catalyst communities interested in using this as their primary web server?


List all Journal entries
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

AxKit2 Status: Fastest perl httpd? 17 Comments More | Login | Reply /

 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • Surely the best way to get not just Catalyst and Jifty but a lot of other Apache-based Perl applications is to allow them to drop in their Apache configs unchanged. In the basic case of a PerlModule being the only thing sitting on a virtual host, this shouldn't be too difficult to do, right? It ought to be just a matter of parsing the config file, and providing an Apache::Fake object to them.
    • Yes, there's a possibility of doing something a bit like that. Though I don't want to get people's hopes up that the Apache API will be fully implemented as some of it simply won't fit (I don't think).

      This is also basically how the SAWA plugin works - it fakes up an Apache::Request-a-like and passes it in.
  • Hi, interesting stuff. I assume you are also using Danga::Socket.

    Where can we look at the code? Sorry if I missed a previous post with the link, but I didn't get it.

    I ask because I also want to make this work with Catalyst.
    --
    life is short
  • Any ideas how to handle database access from this? Something like what POE does? Send the requests off to a separate PPerl/mod_perl?
    • Potentially. I'm not totally set on how to handle DB queries yet - my guess is that with sufficient caching in place it won't matter (and for the places building solutions where they don't care to cache, it probably doesn't matter anyway).

      But yes, likely some sort of separate process we can marshal requests to/from. I'd like to make it as transparent as possible, so what I might do is implement a DBD driver that does this natively.
      • How about things that are not I/O-bound but just compute intensive? Will you do some kind of cooperative time-slicing system similar to what POE does?
        • For that I am writing a class that forks off and sends the response back when ready. If compute intensive is very common I suggest just using a CGI (which we support already).
          • If you could have whatever it is able to support working with the Process.pm API, that would be nice.
  • Am I right in thinking that if you could abstract out the XML stuff (or just make it optional in the plugins—maybe it is already?) that the only requirement for the axkit2 Web server would be Danga::Socket? That's pretty rockin'!

    —Theory
    • Yes. The xmlresponse is already mostly optional. It loads it, and so loads XML::LibXML, but you don't have to implement that phase. I'm sure I could figure out a way to make it entirely optional (or even load XML::LibXML at runtime).

      Of course why you would want a web server without XML::LibXML installed is beyond me ;-)
      • That would rock. I was thinking about using it with Bricolage, for example, where we don't do anything with XML.

        My thought is that it'd be cool if it was a Web server that supports plugins and nothing but a Web server that supports plugins.

        Oh, another q: Would it be possible to allow plugins to be CPAN modules, and then we can just tell the Web server to use a CPAN module as the plugin for a particular Location.

        —Theory
Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.