Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Alias (5735)

Alias
  (email not shown publicly)
http://ali.as/

Journal of Alias (5735)

Sunday July 05, 2009
01:07 PM

Let me be clear, again, about the nature of the DarkPAN.

[ #39234 ]

The growth in the argument that the DarkPAN is somehow some form of boogieman, some wispy vaporous nothing that is unobservable and thus something able to be ignored astonishes me. And I'm not singling out one person here, I'm hearing it from a number of different people.

It's especially unnerving because a big chunk of it is sitting in plain view.

So let me repeat this again.

Until perhaps very recently, the DarkPAN was generally considered to be more or less all the Perl code that wasn't on the CPAN. Because if it wasn't on the CPAN, it was largly impossible to find. And even if you could find it, you couldn't do anything with it.

With the advent of PPI and Ohloh, the situation has now changed.

Now we know, for example, that just the small part of the DarkPAN that is visible is at least 4 times bigger than the CPAN. Because 4 times more code than CPAN is basically what Ohloh managed to uncover. Last time I looked, it was reporting about 97,000,000 lines of code, and we know that the CPAN is in the general range of 20-25,000,000 lines of code.

I've called this semi-visible part of the DarkPAN the GreyPAN, but that's just a way of categorising it. It's still a part of the DarkPAN. It's still code we can't feed into CPAN Testers or other automated systems to examine how a Perl change will impact it.

We also know that it's mostly not large codebases. Anecdotal reports from the Ohloh people themselves suggest it's made up of lots of different bits and pieces. So in fact, it serves as quite a useful analog for the consistency of the DarkPAN itself.

It's something we can suck in, process, and then run what-if scenarios for. At least at a document-level anyway.

Frankly, I'm sick and tired of this back and forth crap.

I'd like to see both sides of this argument shut up (me included) until we've actually built the thing that can suck in the visible parts of the DarkPAN and process it.

If you'd like to help, I've got most of the "Process it" part working now, but I'd dearly love some help on the "sucking it in" part.

If, after we've finished, we're able to establish that (for example) across all C code there is on average 1 line of utility Perl code for every 100 lines of C code then we've got something to work with.

Because we can take estimations about the total amount of C code in the world and extrapolate from that to make a guess at the total amount of Perl code in the world.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • No competent project manager would agree to manage something unmeasurable.

    My argument remains that attempting to do so with regard to the DarkPAN is futile.

    • Insistance is futile, you will not be assimilated?

      • If you can't observe the DarkPAN, its contents, its needs, and the consequences of any change to Perl 5 are opaque. I believe that basing design decisions on the inobservable DarkPAN is a mistake.

        Observing (at least part of) it gives the Perl community a better chance to evaluate proposed changes in documentation, training, best practices, idioms, workarounds, core library changes, and core language changes.

        We may still disagree about the nature and scope of changes as well as their most appropriate or use

        • Maybe the solution a two versions of perl.
          • That idea crossed my mind yesterday. Instead of enabling syntax features at runtime depending on the presence of a command-line flag or use feature 5.10;, use #ifdef to compile in or compile out features. Build one binary without new syntax as maintperl or something and the other as perl.

            This would increase the testing and smoking burden -- and that's a very real consideration for maintainability -- but it provides a backwards-compatible binary as well as a modern binary.

          • Or more. That seems to make sense when there are too many egos for one project. Linux does something like this, with the various branches (e.g. "-mm") coexisting with Linus's tree. Of course there's the usual drama, but they somehow manage to keep sharing patches.

  • It seems odd to lump everything not in CPAN into one amorphous category to be disregarded. Imagine if Microsoft ignored all software not distributed by Amazon, or if Apple ignored all iPhone apps not... wait, bad example. In an open development community, it seems inappropriate to require a specific distribution channel to be used in order for your feedback to be considered on what features are key.
    • It's not deliberate. It's that analyzing all code on the CPAN is so much easier than analyzing code anywhere else that the latter seems intractable.

      • With the analysis tools being made publicly available, input can be solicited from the community at large, and from a sampling of known users. As this is an old, long-standing problem, experiences from other development communities can be used. With the greater amount of interconnection today, it should be easier now than in the past to enable feedback and data to be collected.