Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Alias (5735)

Alias
  (email not shown publicly)
http://ali.as/

Journal of Alias (5735)

Wednesday March 07, 2007
12:17 AM

Fixing CPAN: Part 1 - Toolchain Communication

[ #32597 ]

The current CPAN toolchain is broken.

Not in a "but it works for me just fine" and "users should upgrade" world it isn't. Competent users are doing just fine. It's the incompetent users we have a problem with.

If you want a demonstration, get a computer with a fresh untouched install of a 5-year old Perl and try to install anything of note by just doing 'sudo -H cpan -i Module'. It will almost certainly crash.

And if we ever want anyone that doesn't know Perl (who are by definition not competent with Perl) to install a Perl-based application from CPAN, we need to fix that.

Ironically, most of the things that are broken can be fixed easily, it's just that they involve multiple toolchain components, and the toolchain authors aren't as effective as they could and should be be about co-ordination.

So I'll be trying to get some of these fixed over the next little while, with a view to having a CPAN toolchain that doesn't suck ready for 5.10.1, the first long-term production release of Perl 5.10.

The first step in fixing this is really to get my (adopted) house in order and try to make Module::Install play well with others a bit better.

One of the problems has been that Module::Install contains Module::AutoInstall. This piece of embedded mitochondrial functionality will allow your Makefile.PL to launch a CPAN client inside itself to fulfil dependencies, IF AND ONLY IF it is not already running underneath a CPAN client.

Except it often doesn't work, because Module::Install has no reliable way to tell if it is being run underneath a CPAN client or being run by hand.

The CPAN.pm-detection uses an unreliable lockfile, and the CPANPLUS detection uses an environment variable PERL5_CPANPLUS_IS_RUNNING that lies. It doesn't actually mean that CPANPLUS is running, it just means that CPANPLUS is loaded.

Inside of Module::Install, I've been working on a new experimental Module::Install::With package that contains all the functions used for determining the environment Module::Install is running in.

Now this finally has a reliable way of detecting CPANPLUS. As of the most recent devel release of CPANPLUS, it now will additionally set the environment version PERL5_CPANPLUS_IS_EXECUTING to be the rel2abs path of the Makefile.PL it is running.

In this way, Module::Install can now detect not only that CPANPLUS is running, but that it is running ME and isn't around for some other reason. The next developer release will also (for future-proofing sake) add PERL5_CPANPLUS_IS_VERSION so that Module::Install can know what version of CPANPLUS is running, in case it has to implement some sort of workaround far in the future.

Unfortunately, CPAN isn't quite up with the game yet.

Due to that whole "not communicating enough" problem, the latest developer release of CPAN now for some strange reason sets PERL5_CPANPLUS_IS_RUNNING when calling Makefile.PL... which just won't do at all.

But before the next full CPAN release we should have that fixed, and it will instead set PERL5_CPAN_IS_EXECUTING and PERL5_CPAN_IS_VERSION in a similar way.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • If you want a demonstration, get a computer with a fresh untouched install of a 5-year old Perl...

    I'm not sure how to make it easy for people who don't upgrade their software to upgrade their software, when the defining characteristic of the group of people who don't upgrade their software is that they don't upgrade their software.

    There's a point at which helping people do stupid things goes beyond negligence and on toward malice. Supporting ancient versions of Perl gets dangerously close to that lin

    • You make it easy for people that don't upgrade their software to upgrade their software by making the software help/force them to upgrade their software.

      The only challenge is in doing it in a way that isn't stupid.

      In essence, if the past is stupid and broken and will hurt them, and they don't realise it, then we drag them into the future unless they actively tell us not to.
      • Is your plan then to produce a minimal amount of software that can upgrade an ancient Perl installation into something modern, at which point the entire upgraded toolchain is available?

        I'm all for that.

        Crippling all of the nice modern tools that take advantage of all of the nice bugfixes and new features developed this millennium in Perl for the sake of people who refuse to upgrade, however, bothers me a little bit.

        • My plan is to firstly make the toolchain aware of the fact it is fatally out of date, and to give it the ability to upgrade itself.

          If, in the process of doing so, an ancient Perl discovers that part of the toolchain is not longer supported on that version of Perl, it should intentionally stop and tell the user CPAN no longer is supported on Perl version $blah, rather than just crashing.
          • Provided that only a minimal amount of code has to be backwards-compatible with, say, 5.004_x, that sounds reasonable.

            • That should be the case.

              From my monitoring of heavily used modules, CPAN as a whole has had a fairly clear dependency on 5.005 for a long time.

              And there's been a trend over the previous year to 5.6.1 as a minimum.

              Plus of course the 5.8.5 unicode sanity point.

              We'll see how it goes, but I don't expect the requirements to be too onerous, and the amount of code should be fairly minimal.
    • A 5 year old perl install is 5.8.0 (well 4 years 8 months). Is that really old enough to be called ancient?
      • Yep, go try it.

        Typically what will happen is that some module (say, Catalyst) will have a dependency that uses Module::Build in the Build.PL. But dependencies don't get installed until AFTER Build.PL.

        Circular unresolvable dependency. Explode.

        Unless you first upgrade Bundle::CPAN, the standard CPAN client doesn't work.

        And anyone that isn't a Perl programmer is going to have no idea that they have to upgrade CPAN to unbreak it before they run it.
      • Sorry, I didn't realise you were replying to chromatic, not me.
      • There have been eight stable releases of Perl since 5.8.0. That number will soon be nine. It's ancient.

  • And if we ever want anyone that doesn't know Perl (who are by definition not competent with Perl) to install a Perl-based application from CPAN, we need to fix that.

    Is that really what we want to be aiming at? I mean, I think that CPAN (and CPANPLUS) is great, but the only people who know about it are Perl developers. It's great for us that the same method works (for some value of "works") on every platform that we use, but for the average user (or the average adminstrator of a computer with Perl instal

    • Yes, sysadmins that don't know Perl should be able to install the latest and greatest off CPAN.

      Yes, newbie programmers on the first day they start Perl should be able to install modules without having to deal with errors they don't yet know.

      You are correct that it would be better for users to use binary packages, and solutions for these exist.

      Unfortunately, nobody that I'm aware of has managed to find a way to package and maintain up to date a reasonably large subset of Perl. debian is at 3,000ish out of 11
    • Is that really what we want to be aiming at?

      Is that really what we want to not be aiming at? How does having a working CPAN toolchain preclude binary package repositories? Why should we not aim at having both?

    • I'd love to see a way to make it easier for CPAN authors (or perhaps CPAN itself - or another project attached to CPAN) to create packages that can be used to install CPAN modules using packaging mechanisms that are native to various operating systems - whether it's rpm, deb, sis or something else.

      Take a look at CPANPLUS::Dist::Deb [cpan.org] and debian.pkgs.cpan.org [cpan.org] and you'll see you and I are on the same page :)

      It still needs some more hacking to make it easier to write new C::D::* modules, but the concept works :)

      • one gotcha i found with using debian CPAN repo is that the installation dir is different from CPAN installation. and CPAN is not aware of what module is installed by debian repo. since CPAN module path is ahead of debian CPAN module in @INC. it may introduce problem when you accidentially installed a newer version module that is already installed by debian repo. the faulty module is however hard to spot sometimes.

        there is also dh-make-perl in debian. IIRC, cpan2dist works better than dh-make-perl.
      • Ooh. Very cool.

        Now, of course, personally I don't care at all about Debian packages. Where is CPANPLUS::Dist::RPM.

        Or have I just volunteered myself?

  • Perl packagers should install Bundle::CPAN with any new installation -- this is the approach Vanilla Perl takes and the result is quite sane for installing any module. It's reasonable to put this onus on packagers -- it's like ensuring that bugfix packages on any older piece of software are in place.

    Don't use Autoinstall -- most of the gyrations you describe are just to fix the problems this creates. The module toolchain should leave it to CPAN/CPANPLUS/etc to resolve dependencies. Makefile.PL just n

    • Unfortunately, even if everyone did bundle Bundle::CPAN, that only solves the problems that have occurred in the past. It doesn't solve the problems that will occur in the future.

      To most of the old Perl distributions out there that predate the invention of the new generation of toolchain modules, they all worked just fine at the time. It was the future that bit them.

      And so we need to try to deal with the bugs that we don't know have happened yet in the toolchain.

      As for AutoInstall, you are superficially cor
  • In the past few months, I've seen toolchain-related issues appear on perl-qa, module-build, and module-authors mailing lists (and those are just the ones to which I'm subscribed.) And here of course.

    Where's toolchain@lists.perl.org? If there's an appropriate list already, let's funnel the issues into it and collect them into one place.