The growth in the argument that the DarkPAN is somehow some form of boogieman, some wispy vaporous nothing that is unobservable and thus something able to be ignored astonishes me. And I'm not singling out one person here, I'm hearing it from a number of different people.
It's especially unnerving because a big chunk of it is sitting in plain view.
So let me repeat this again.
Until perhaps very recently, the DarkPAN was generally considered to be more or less all the Perl code that wasn't on the CPAN. Because if it wasn't on the CPAN, it was largly impossible to find. And even if you could find it, you couldn't do anything with it.
With the advent of PPI and Ohloh, the situation has now changed.
Now we know, for example, that just the small part of the DarkPAN that is visible is at least 4 times bigger than the CPAN. Because 4 times more code than CPAN is basically what Ohloh managed to uncover. Last time I looked, it was reporting about 97,000,000 lines of code, and we know that the CPAN is in the general range of 20-25,000,000 lines of code.
I've called this semi-visible part of the DarkPAN the GreyPAN, but that's just a way of categorising it. It's still a part of the DarkPAN. It's still code we can't feed into CPAN Testers or other automated systems to examine how a Perl change will impact it.
We also know that it's mostly not large codebases. Anecdotal reports from the Ohloh people themselves suggest it's made up of lots of different bits and pieces. So in fact, it serves as quite a useful analog for the consistency of the DarkPAN itself.
It's something we can suck in, process, and then run what-if scenarios for. At least at a document-level anyway.
Frankly, I'm sick and tired of this back and forth crap.
I'd like to see both sides of this argument shut up (me included) until we've actually built the thing that can suck in the visible parts of the DarkPAN and process it.
If you'd like to help, I've got most of the "Process it" part working now, but I'd dearly love some help on the "sucking it in" part.
If, after we've finished, we're able to establish that (for example) across all C code there is on average 1 line of utility Perl code for every 100 lines of C code then we've got something to work with.
Because we can take estimations about the total amount of C code in the world and extrapolate from that to make a guess at the total amount of Perl code in the world.