Now that my blog is listed on the Ironman planet, Matt Trout has loaned me his chainsaw and suggested that asking for help here is likely to garner a response, because he says so.
So consider this an official request for assistance to help some overloaded developers fix what I consider to be the most important bug in the toolchain right now.
It's a bug in Archive::Extract, and it's probably not that much work, but neither Jos Boumans nor I have the free time right now to fix it.
The bug in question is that when Archive::Extract uses Archive::Tar to unroll a tarball, it uses the wrong API. Instead of using the memory-efficient streamed extraction API to roll the whole tarball out to disk directly, it instead loads the whole thing into memory and unpacks it from there.
It should probably use code similar to the implementation of Ivor's Archive::Tar::Streamed instead.
This is a big problem because once all the memory inflation and memory copying has happened to allow this loading to, a couple of big pathological distributions on CPAN consume almost the entire 2gig memory limit of the (32-bit) process.
This bug is making the performance of CPAN on Win32 much worse and memory-bloaty, but worse is that it takes CPAN::Mini::Visit over the process limit and crashes it, which also means that this bug is currently blocking work on the GreyPAN scanner experiments (Perl::Metrics2), the META.yml database ORDB::CPANMeta, the permissions-aware replacement for the rather unreliable CPANTS dependency graph (CPANTS::Weight, my unified CPANDB SQLite index, and the sorely-needed accuracy fixes for the Top 100 website.
Improving almost all these things require both accurate and 100% complete coverage of minicpan in order to give answers that are good enough to swap out the original first-generation implementations, and this one relatively approachable bug is preventing the ability to reliably reach 100% coverage.
Because this bug also disproportionately impacts Win32 and is a core module, this bug is also very important for the July release of Strawberry Perl, as well as the Perl 5.10.1 release.
If anyone out there has a few hours to attack this bug and fix it, your efforts will have a huge knock-on effect on the quality of many other parts of the CPAN ecosystem.
If you are able to help us out, you can find Jos (kane), myself (Alias), or other that can point you in the right direction in #toolchain on irc.perl.org.