Slash Boxes
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Matts (1087)

  (email not shown publicly)

I work for MessageLabs [] in Toronto, ON, Canada. I write spam filters, MTA software, high performance network software, string matching algorithms, and other cool stuff mostly in Perl and C.

Journal of Matts (1087)

Wednesday February 02, 2005
03:07 PM

New house, and perl DNS stuff

[ #22988 ]

The good news is we're now finally getting settled down in Canada.

Last week we picked up our new (used) car. It was a huge amount of hassle getting the insurance sorted out - passing no-claims bonus details from the UK to Canada, and having forms lost in the process, but eventually it all came to pass. It's a huge relief not to have to rely on the in-laws to drive us everywhere!

Then as luck would have it that week we found the house that was right for us. It ticked all the boxes: Fenced yard, quiet neighbourhood, reasonably close to in-laws, and decent space inside.

About five minutes after I spoke to the Landlady agreeing to rent the place Allied/Pickfords international movers called and said our shipment had arrived, and could we please go to Mississauga to get customs to stamp the papers.

So yesterday the movers arrived. They were such a contrast to the UK movers - pleasant, chatty and personable. They moved everything in really quickly and unpacked about 80% of stuff (we decided to get them to leave things like clothes so that we could go through them before unpacking). Only a few minor damages - a snow globe which froze (we should have known about that of course) and a couple of bashes in other things but nothing at all major to report. I guess we won't know if anything small is missing until months from now when we both go "I'm sure we had a foostick somewhere...", but nothing major is missing.

We won't move in for a few days. Primarily because I need the DSL working before I can move there, but also because we want to get the house nice and sorted before moving in.


As far as work goes I've been a bit swamped. I'm working hard to get a major piece of code done. One component is asynchronous DNS which sort of works with Net::DNS, but there are problems...

Net::DNS uses a new socket for each DNS query when using bgsend/bgread. This becomes a problem of resources when you start doing thousands of DNS queries in parallel. Beyond that it's also rather time consuming setting up an IO::Socket for every query. DNS (the protocol) allows you to use a single socket to do all your queries over, mapping things back to your query via the packet's ID field.

Tony Finch is/was doing some work on Net::DNS to make it do the right thing here, but said that he wasn't likely to finish until March, so work pressure meant I had to go it alone. Rather than try and fix up Net::DNS I decided it would be easier to take some of Net::DNS (the packet construction/parsing code) and make my own DNS client library in the shape that I need it.

So with a bit of work I now have a DNS query system that uses a single socket, does in-memory caching, and plugs nicely into my event loop subsystem. It storms along - I've managed to process about 2.2 million lines of logs, doing reverse DNS for each, in about 20 minutes (though log processing isn't going to be the main use of this code). It should run even faster once I have a decent DSL line.

I posted the details of how to do it to the SpamAssassin dev mailing list if you're interested in doing the same.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
More | Login | Reply
Loading... please wait.
  • Canadian movers are absolutely the most wonderful movers to deal with. I had a really bad experience moving to Canada, and had to pull hen's teeth just to get *one* company to come and give me a quote. (They wound up overcharging me a good deal, but that's all legit according to US Law.)

    The US company packed everything up, and moved it into a warehouse about 3 miles away from my apartment. A Canadian crew picked it up and drove it the remaining 3,000 miles to my new apartment in Canada. The movers the