Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

drhyde (1683)

drhyde
  (email not shown publicly)
http://www.cantrell.org.uk/david

Journal of drhyde (1683)

Saturday October 25, 2008
07:43 AM

CPANdeps efficiency improvements

[ #37736 ]

CPANdeps has been getting more heavily used recently, and all of the users hitting my shonky code have been occasionally DOSsing the server. That's because the old code would eat up to 134MB of memory to generate a set of results. And that's because I was using Parse::CPAN::Packages to convert from module names to package names. That reads the entire 02packages file into memory and converts it into a very large data structure.

The new version of the code stores that in the database instead, so instead of using 134MB I now only use 25MB per request. This means that I'm hitting the database harder, but even so, things should still run a lot better because even serving 25 requests in parallel (the maximum it's configured to do) it will only need 625MB, so you won't all be fighting over swap space.

I would like to thank Jonathan Rockway once again for writing the excellent CPANdeps test suite which lets me be confident that the site still works just as it did before. His test suite is available on the CPAN as Angerwhale :-)

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.