A few days ago I read James Duncan Davidson's The Web As A Pipe (I think it was linked from a blog entry here). It got me thinking about AxKit 2.0. The question I was asking myself was: why do I need Apache?
All the deployments of AxKit in serious setups probably use a front-end proxy. That's because this is how mod_perl stuff is recommended to be setup. So if the backend is just a system for running bits of perl, why do I need all the extra "apache" stuff?
The other avenue that led me to this (below) is that I've been working a lot on scalable servers lately. I wrote an epoll/kqueue smtp server for our spamtrap (now part of qpsmtpd), and I hack on various logs which I query with dns as I'm parsing them, requiring tens of thousands of parallel DNS queries. For all of this I use Danga::Socket - a sort of POE-like framework for epoll/kqueue, but without all the overhead of POE. I also recently started using djabberd for a jabber server - a very cool project you should check out.
In working on scalable servers (all written in Perl) I got to thinking - "Why isn't there a highly scalable HTTP server written in perl, with decent pluggability like mod_perl, but without all the hassle of setting up apache and mod_perl". So I wrote one.
I borrowed bits of code from danga's perlbal project (a perl http load balancer that they put in front of the livejournal servers, with each perlbal server doing about 40Mb/s of traffic), some of qpsmtpd's ideas, and re-wrote a lot of it as my own.
So now I have a http server. It does basic http stuff, like receive http requests and send responses back. It has pluggable logging (like qpsmtpd). It has an expandable config system, so any plugin can define its own config directives in a MUCH easier manner than apache. It can run CGIs. In benchmarking I can deliver about 1500 req/s (just flat files, and that's unoptimised as I haven't done the fancy AIO stuff yet), but more important is that it's scalable - it can handle 100,000 concurrent connections if you want it to, and easily send large files to slow clients.
Anyway, I'm not going to reveal the source for it just yet (well, some people have seen it - join #axkit-dahut on irc.perl.org if you want to play), but I wanted to blog what I'd been hacking on.