Slash Boxes
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Alias (5735)

  (email not shown publicly)

Journal of Alias (5735)

Thursday May 29, 2008
10:48 PM

We need to do something about this dev release pollution

[ #36548 ]

More than just about any other factor, what I love most about the CPAN is that it solves the cross-platform problem so well.

Rather than try to factor out change by adopting one platform's concept, or going for a lowest-common-denominator approach, it embraces diversity.

Modules like File::HomeDir (which I loved even before I took it over) with it's vastly differing dependencies on different platforms make programming interesting, because they robustly solve the problem of getting the platform out the way.

I can accept that my code might be missing a feature, or have some bug. But I don't want to have to care WHERE I'm writing some bit of code. I just expect my stuff to work everywhere.

And so I tend to spend quite a bit of time keeping my CPAN Testers FAIL counts for modules at zero. And I've done a reasonably good job of it so far, despite the volume of packages I maintain.

But unfortunately, this is starting to become very very hard to achieve.

The problem started when the CPAN Testers modules started to get good enough that people were considering hooking up their bleed tests and developer-release tests and developer versions of Perl to the same reporting mechanisms.

And so now we have this situation where any legitimate production testing results are drowned out in the sea of results from developer releases.

Worse, the TOTAL pass/fail counts are reported inclusive of these developer releases as well, leading the headline testing results that people rely on when choosing modules to use to be invalid, all because some blead patch a year ago accidentally broke some minor part of Perl, causing half of CPAN to fail to build.

It may be interesting to a small cadre of people SOME of the time, but it's at best not interesting to the public, and at worst highly misleading of the stability of the CPAN.

I say that we've now passed a threshold where this extra testing resolution is causing more harm than good, and I strongly recommend that anyone with a site in the CPAN cloud comprehensively filter out non-production information by default.

Primarily this should mean no more visibility of things like Perl 5.9.1 or 5.11.0 patch 123456 by default, until someone throws some notional "include developer Perls" toggle.

I'm tempted to suggest we also filter out CPAN module developer releases by default. But I have some concerns that if we did this it will discourage awareness and use of developer releases amongst the authors as a whole.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
More | Login | Reply
Loading... please wait.
  • I've also been thinking about this recently, but have yet to think of a suitable solution. While the dev release testing is useful for authors, it isn't for users who don't expect to ever use those releases in a production environment. This applies to past developer releases 5.9.x and 5.7.x, as well as 5.11.x. Incidentally was 5.5.x a developer release?

    In the future with the newer system that's in production, it will hopefully be easier to filter these releases out. However, at the moment, the CPAN Testers

    • Personally, I'm quite happy for the statistics site to keep including developer releases, at least in the headline graphs.

      For the version/platform matrix, that's where I'd like to see you remove dev Perls
  • I see your point. This level of intensity in testing is great for the Perl core -- I would love to get it for Parrot -- but less useful for non-core distributions.

    Since the testers responsible for these dev versions are (a) our #1 tester (by volume) and (b) one of our most respected module authors, perhaps this could be raised on the CPAN Testers Discussion list -- which they both read -- and a proposal for change worked out.