Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Journal of jjore (6662)

Tuesday April 10, 2007
03:40 AM

Jifty/Catalyst are too dang big

[ #32953 ]

I found a "nifty" bug in CPAN.pm today. If you try to install something gi-normous like Catalyst or Jifty, the CPAN.pm shell craps out halfway through because it deleted somepart of the stuff to be installed during the build for the umpteen billion dependencies.

I didn't track how many modules Catalyst adds to a clean 5.8.8 but Jifty added 174 distributions and eventually failed because of something to do with XML::Parser, WWW::Mechanize, and perhaps some simple thinkos.

Blarg. That's a whole heap of module.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • This is a perfect example of why Perl needs better packaging. Something like PAR [perl.org] seems to be a good idea, but people don't seem to use it. Odd given that many people are having [perl.org] problems [perl.org] with [perl.org] installing dependency-heavy Perl modules.

    The value of CPAN is greatly diminished if we don't have an easy packaging mechanism. Many PHP apps are much easier to install than equivalent Perl ones, solely due to dependencies.

    • If I wanted to release giant spaghetti-code apps that required a large standard library and reimplemented a bunch of functionality over and over again, and had no unit tests and no code reuse between applications at all, I imagine we could do installers as simple as PHP apps too.

      CPAN is complicated for a reason.

      The answer is not to make packages stupider, it's to make the CPAN tools smarter.
      • I don't like PHP and I'm not at all advocating PHP-style code and coding practices (do they have any, apart from 'hack'?). I'm just pointing out that many PHP apps, however dirty, are much easier to install.

        I don't see how bundling prerequisite modules with an application (e.g. Plagger) or large module does any harm. More disk space will be used, but in most cases that won't be a problem. Some may prefer to install modules separately, and I'm not seeking to stop that - I just want an easier way of distrib

  • I wonder if all those dependencies are really needed. I was listening to a talk by brian d foy and he made a good point. Perl modules tend to be "all inclusive" of the spec and not "just what you need". That tends to make them rather largish. Now if you are doing something specific that probably works but once you step into framework land, I am not sure if that works as well.
  • Well, I guess it is a matter of perspective since it is doing exactly what it says it does.

    "Currently the cache manager only keeps track of the build directory ($CPAN::Config->{build_dir}). It is a simple FIFO mechanism that deletes complete directories below build_dir as soon as the size of all directories there gets bigger than $CPAN::Config->{build_cache} (in MB)."

    Perhaps a better way of handling would be for it to recognize that it is removing directories that it needs for the current installation
    • Even if it does what it says it does, if what is says it does is broken, it's still wrong.
    • I think that CPAN.pm should only check the cache size after it finishes running the queue, not in the middle. CPAN doesn't guarantee anything about how far over the cache limit the build cache can go and is pretty vague about how often the cache is checked. So I think this is a bug that can be fixed pretty easily.

      It reminds me of the Uncertainty Principle in a way -- you get to exceed the cache limit until you actually go and measure how much cache you're using. As long as that time window is set appro

      • There is the config variable scan_cache that determines how often the cache is teared down to its limit. You can set this variable only to two values: "atstart" or "never". This means that (modulo bugs, of course) it is impossible that the cache manager is run in the middle of a session. The usual bug that's happening instead is that somebody (usually Module::Install) calls CPAN.pm recursively. Then "atstart" can be at any point in time.

        I hope that I soon get evidence about what really is going on on Josh

        • Jifty::DBI uses Module::Install which is apparently what kills the Jifty install. It turns out that upgrading CPAN fixes the problem. I suppose that modules with large dependency trees ought to have the non-errant versions of the installer in their pre-req lists.

          Perhaps that should be dynamic - if CPAN is asking Jifty about installing it, Jifty can tell CPAN that CPAN needs to be at least 1.90 to be able to successfully install it.
          • Or maybe Module::Install should just die.

            • That’s what I’d say. I understand why people consider it necessary, but I think the effort would have been better invested in fixing Module::Build’s bootstrap problem. (Last I heard, we’re getting there, though, so hopefully Module::Install can indeed just die, soon.)

              • That still leaves us with the problem of all those people who have bundled Module::Install with their own modules. Perhaps the solution is for CPAN.pm to include its own Module::Install which does nothing but die, and set PERL5LIB to make sure that its Module::Install killer always loads first.

                Bwahahahaha

    • Andreas hinted the new CPAN.pm 1.90 might not have this problem. I'm testing it now.