All the Perl that's Practical to Extract and Report
use Perl Log In
tsee (4409)
{smueller} {at} {cpan.org}
http://steffen-mueller.net/
You can find most of my Open Source Perl software in my CPAN directory [cpan.org].
PAR evilness to make remote auto-upgrading work
When I think about telling people about PAR internals, a reply from a colleague readily comes to mind, when he was asked about an icky detail of his analysis:
You don't want to know how sausages are made!
But then I can't resist grossing out people with some details anyway...
Two years ago, I wrote PAR::Repository::Client as an interface for loading PARs and thus arbitrary modules from a remote server. If the client is installed, all you need to do to auto-load missing modules from the server is:
use PAR { repository => 'https://foo.com/myapp' };
use Foo; # will be loaded from remote if necessary
But since this may become expensive, and caching the binaries only removes part of that, the "install" option was part of the interface almost from the start:
use PAR { repository => 'https://foo.com/myapp', install => 1 };
use Foo; # will be loaded AND INSTALLED if necessary
Back then, I also added most of the code necessary for an "upgrade" option.
use PAR { repository => 'https://foo.com/myapp', upgrade => 1 };
use Foo; # will be loaded AND INSTALLED OR UPGRADED if necessary
Unfortunately, it was missing a few critical details until today. The repository client is normally only invoked when all other sources fail. But that's a problem if you're trying to check for upgrades. Thus, repositories in upgrade-mode are now checked early in the module-loading process.
The real bummer was that in order to check for upgrades, the locally installed version has to be determined. Since this is hard to do reliably without loading the module, that's what PAR has to do. But that means require()ing module X from within an early @INC hook that ran due to a "require X;". There's so many things wrong with that idea, it's not even funny. It seems that creating an infinite recursion in an @INC hook segfaults perl 5.8.9. Regardless, it can be (and was) made to work:
- Before running the client's upgrade_module method, dynamically override the set of active (via PAR.pm) PAR::Repositories to be empty.
- Run the current repository client's upgrade_module method which will attempt to require the module for checking its version.
- Afterwards, check whether the module was loaded using %INC.
- If not, continue normally, probably ending up failing to load the module from anywhere or loading the freshly installed copy.
- If the module was loaded, prevent an additional loading with an evil trick in the @INC hook:
my $line = 1;
return \*I_AM_NOT_HERE, sub { $line ? ($_="1;",$line=0,return(1)) : ($_="",return(0)) };
Even disregarding the slight obfuscation, can you figure out how this works?
One obscure feature of @INC and the module loading is the return value(s) of a subroutine @INC hook. It normally simply returns a file handle that the module code is then read from. But if it returns a code ref as its second return value, that code ref is called repeatedly until it returns false. After each invocation, $_ is assumed to contain the next line of the module code. If the first argument was a file handle nonetheless, $_ is initialized to a new line from the file handle before calling the subroutine.
The motivation here is mostly that we want to set the file contents to "1;". Unfortunately, passing undef as the file handle resulted in the subroutine not being called. This smells like a bug in perl to me, but I'll have to check that more closely with blead. Furthermore, it's not wise to load any unnecessary modules in PAR.pm as they would have to be included verbatim in an uncompressed part of PAR::Packer created executables. Therefore, instead of simply passing a IO::Handle->new(), I'm supplying an arbitrary GLOB ref.
Finally, the subroutine itself simply sets $_ to "1;" in the first invocation and returns zero on the second to stop the evaluation, thus essentially short-circuiting require()'s loop through @INC.
After going through this considerable pain, I got the auto-upgrading feature of PAR::Repository::Client to work. There's probably still bugs and testing it as part of the test suite is no fun (but still feasible).
Stay tuned for a new release of the involved modules.
Cheers,
Steffen
Padre 0.22 highlights and checks Perl6 code
Padre version 0.22 has just been uploaded to the PAUSE. That means it will propagate to the CPAN mirrors without a few hours. Like the previous release, the list of changes is quite long, but one particular achievement is support for highlighting Perl6 code and checking its syntax if Padre::Plugin::Perl6 is installed and enabled. Christmas is close.
Once the distribution has reached your CPAN mirror, you will be able to access the full Change Log here.
The next Padre release is very preliminarily scheduled for December 28th or 29th and we're still looking for a new release engineer.
Best regards,
Steffen
On a similarity between C++ and Perl popularity
Today, I read an interesting interview with Bjarne Stroustrup, the father of C++, on DevX from August of this year. It's a good read, so if you're a C++ user, you should have a look. But even if you never touched any C++ code, there's a very interesting bit of information:
On page 6, the interviewer, Danny Kalev, asks Stroustrup:
Is C++ usage really declining, as some biased analysts and journalists have been trying to convince us for years (often not without ulterior motives), or is this complete nonsense?
Does that ring a bell? The "Perl is dead" crap that's been splashing down the gutters of the interweb waste disposal system, anyone? I urge you to read the full answer. Much of it applies to Perl as well. Please note that Stroustrup doesn't simply dismiss the issue. Here's an excerpt from his reply:
[...] C++ use appears to be declining in some areas and appears to be on an upswing in other areas. [...] Most of the popular measures basically measures noise and ought to report their findings in decibel rather than "popularity." Many of the major uses are in infrastructure (telecommunications, banking, embedded systems, etc.) where programmers don't go to conferences or describe their code in public. Many of the most interesting and important C++ applications are not noticed, they are not for sale to the public as programming products, and their implementation language is never mentioned. [...]
It's a really big world "out there" and the increase in the number of users of one language does not imply the decrease in the numbers of another. [...]
One simple thing that confuses many discussions of language use/popularity is the distinction between relative and absolute measures. For example, I say that C++ use is growing when I see user population grow by 200,000 programmers from 3.1M to 3.3M. However, somebody else may claim that "C++ is dying" because it's "popularity" has dropped from 16 percent to 10 percent of the total number of users. Both claims could be simultaneously true as the number of programmers continues to grow and especially as what is considered to be programming continues to change. [...]
Most of the popularity measures seem to measure buzz/noise, which is basically counting mentions on the web. That's potentially very misleading. Ten people learning a scripting language will make much more noise than a thousand full time programmers using C++, especially if the thousand C++ programmers are working on a project crucial to a company—such programmers typically don't post and are often not allowed to. My worry is that such measures may actually measure the number of novices and thus be an indication of a worsening shortage of C++ programmers. Worse, managers and academics may incautiously take such figures seriously (as a measure of quality) and become part of a vicious circle.
I know first hand about large C++ systems that don't produce the slightest bit of publicity for the language they're implemented in. It's what I deal with every day. Dito for large Perl code bases. Stroustrup hits the nail on the head about this issue (and C++). It's exactly what I think about Perl in the same context. There may be an issue with not generating as much noise as others (not enough new blood), but it's by no means an indication of stagnation or decline. People simply use it do what they always did. People also use it to do new stuff. But they don't blather about it all day. They earn their salary and at the end of the day, they go home to their families and spend their spare time on more interesting things than blogging about their favourite new toy language*. You have to realize: This applies to easily more than 95% of all professional programmers.
* That reminds me of something... got to go.
Padre 0.21 - More improvements than can fit in this margin!
I'm quite proud to announce the release of Padre 0.21.
It features the biggest list of changes of any Padre release so far. The menu code has received a major overhaul, the editor has become multi-threaded, and we see more "advanced" Perl-specific features like reliably finding the location of a variable declaration or the experimental feature of replacing a lexical variable. The full list of changes can be found in the Changes file of the distribution.
The list of developers has grown to thirteen, but that list is probably not even complete. The program has been translated to nine languages at this point.
With this release, we are starting to rotate the release duty so the weekly or bi-weekly releases don't block on the availability of Gabor at those rare points in time when no major refactoring or feature implementation is going on.
You can find more information and the Padre mailing list, irc information, bug and issue tracking, etc. on the Padre site as usual.
Cheers,
Steffen
Class::XSAccessor with constructors = Object::Tiny::XS
Since Class::XSAccessor now supports constructors (see previous journal entry), there's everything in place to implement Object::Tiny in XS. Hence you can find Object::Tiny::XS 1.01 on CPAN very soon. It's been uploaded to PAUSE.
The following benchmarks can be found in the distribution. They're comparing O::Tiny::XS to O::Tiny and Class::Accessor::Fast.
Benchmarking constructor plus accessors...
Rate accessor tiny tiny_xs
accessor 107325/s -- -40% -57%
tiny 177837/s 66% -- -29%
tiny_xs 248931/s 132% 40% --
Benchmarking constructor alone...
Rate accessor tiny tiny_xs
accessor 168659/s -- -49% -57%
tiny 330831/s 96% -- -17%
tiny_xs 396844/s 135% 20% --
Benchmarking accessors alone...
Rate accessor tiny tiny_xs
accessor 331/s -- -16% -54%
tiny 395/s 19% -- -45%
tiny_xs 715/s 116% 81% --
Class::XSAccessor - now doing XS constructors
So far, when using Class::XSAccessor or Class::XSAccessor::Array to generate fast accessors implemented in XS for you, you'd still have to write a simple new:
sub new {
my $class = shift;
my $self = bless { @_ } => $class;
return $self;
}
So I just added an XS implementation of the above constructor which you can optionally import into your class. The gains are modest compared to the large speedup of the accessors. You gain only something like 40% (without arguments). However, I haven't compared this with any of the heavier method generators out there.
Now, if you think that this kind of micro-optimization is pointless: You're right. Besides being an exercise in XS, this module means that you don't have to spend your time micro-optimizing your simple methods. Everybody knows it's bad, but almost all succumb to the inner voice that makes them do it regardless eventually. With this being done, you can't get much faster, so you don't even have to try!
Cheers,
Steffen
Marcel's accessor generator benchmarking
So Marcel has been benchmarking accessor generators. He used Class::Accessor::Fast::XS which is a verbatim copy of Class::XSAccessor with the Class::Accessor interfacing code added. But he missed out on the fastest ones bar XS accessor hand-rolling. It's a combination of using Object::Tiny's constructor and Class::XSAccessor's accessor generation. (Let's call it Object::Tiny::XS!) Here's the adapted code and then the results:
Benchmarking attribute access only
#!/usr/bin/env perl
use warnings;
use strict;
use Benchmark qw(cmpthese timethese:hireswallclock);
package WithMoose;
use Moose;
has myattr => ( is => 'rw' );
package WithMooseImmutable;
use Moose;
has myattr => ( is => 'rw' );
__PACKAGE__->meta->make_immutable;
package WithMouse;
use Mouse;
has myattr => ( is => 'rw' );
package WithClassAccessor;
use base qw(Class::Accessor);
__PACKAGE__->mk_accessors(qw/myattr/);
package WithClassAccessorFast;
use base qw(Class::Accessor::Fast);
__PACKAGE__->mk_accessors(qw/myattr/);
package WithClassAccessorFastXS;
use base qw(Class::Accessor::Fast::XS);
__PACKAGE__->mk_accessors(qw/myattr/);
packag e WithClassAccessorComplex;
use base qw(Class::Accessor::Complex);
__PACKAGE__->mk_new->mk_scalar_accessors(qw/myatt r/);
package WithClassAccessorConstructor;
use base qw(Class::Accessor::Constructor Class::Accessor::Complex);
__PACKAGE__->mk_constructor->mk_scalar_accessors(qw/ myattr/);
package WithMojo;
use base qw(Mojo::Base);
__PACKAGE__->attr('myattr');
package WithClassMethodMaker;
use Class::MethodMaker
[ scalar => [ qw/myattr/ ],
new => [ qw/-hash new/ ],
];
package WithAccessors;
use accessors qw(myattr);
sub new { bless {}, shift }
package WithObjectTiny;
use Object::Tiny qw/myattr/;
sub set_myattr { $_[0]->{myattr} = $_[1] }
package WithSpiffy;
use Spiffy -base;
field 'myattr';
package WithClassSpiffy;
use Class::Spiffy -base;
field 'myattr';
package WithClassXSAccessor;
use Class::XSAccessor accessors => { myattr => 'myattr' };
sub new {my $class = shift; bless {@_} => $class}
package WithClassXSAccessorArray;
use Class::XSAccessor::Array accessors=> { myattr => 0 };
sub new {my $class = shift; my %args = @_; bless [$args{myattr}] => $class}
package WithObjectTinyXS;
use Object::Tiny qw/myattr/;
use Class::XSAccessor accessors => { myattr => 'myattr' }, replace => 1;
package main;
my $moose = WithMoose->new;
my $moose_immutable = WithMooseImmutable->new;
my $mouse = WithMouse->new;
my $class_accessor = WithClassAccessor->new;
my $class_accessor_fast = WithClassAccessorFast->new;
my $class_accessor_fast_xs = WithClassAccessorFastXS->new;
my $class_accessor_complex = WithClassAccessorComplex->new;
my $class_accessor_constructor = WithClassAccessorConstructor->new;
my $mojo = WithMojo->new;
my $class_methodmaker = WithClassMethodMaker->new;
my $accessors = WithAccessors->new;
my $object_tiny = WithObjectTiny->new;
my $spiffy = WithSpiffy->new;
my $class_spiffy = WithClassSpiffy->new;
my $direct_hash = {};
my $class_xsaccessor = WithClassXSAccessor->new;
my $class_xsaccessor_array = WithClassXSAccessorArray->new;
my $object_tiny_xs = WithObjectTinyXS->new;
cmpthese(timethese(-5,{
moose => sub {
$moose->myattr(27);
my $x = $moose->myattr;
},
moose_immutable => sub {
$moose_immutable->myattr(27);
my $x = $moose_immutable->myattr;
},
mouse => sub {
$mouse->myattr(27);
my $x = $mouse->myattr;
},
class_accessor => sub {
$class_accessor->myattr(27);
my $x = $class_accessor->myattr;
},
class_accessor_fast => sub {
$class_accessor_fast->myattr(27);
my $x = $class_accessor_fast->myattr;
},
class_accessor_fast_xs => sub {
$class_accessor_fast_xs->myattr(27);
my $x = $class_accessor_fast_xs->myattr;
},
class_accessor_complex => sub {
$class_accessor_complex->myattr(27);
my $x = $class_accessor_complex->myattr;
},
class_accessor_constructor => sub {
$class_accessor_constructor->myattr(27);
my $x = $class_accessor_constructor->myattr;
},
mojo => sub {
$mojo->myattr(27);
my $x = $mojo->myattr;
},
class_methodmaker => sub {
$class_methodmaker->myattr(27);
my $x = $class_methodmaker->myattr;
},
accessors => sub {
$accessors->myattr(27);
my $x = $accessors->myattr;
},
object_tiny => sub {
$object_tiny->set_myattr(27);
my $x = $object_tiny->myattr;
},
spiffy => sub {
$spiffy->myattr(27);
my $x = $spiffy->myattr;
},
class_spiffy => sub {
$class_spiffy->myattr(27);
my $x = $class_spiffy->myattr;
},
direct_hash => sub {
$direct_hash->{myattr} = 27;
my $x = $direct_hash->{myattr};
},
object_tiny_xs => sub {
$object_tiny_xs->myattr(27);
my $x = $object_tiny_xs->myattr;
},
class_xsaccessor => sub {
$class_xsaccessor->myattr(27);
my $x = $class_xsaccessor->myattr;
},
class_xsaccessor_array => sub {
$class_xsaccessor_array->myattr(27);
my $x = $class_xsaccessor_array->myattr;
},
}));
Results:
class_accessor_constructor 2454/s
moose 4989/s
mouse 11187/s
class_methodmaker 35946/s
mojo 52058/s
class_accessor_complex 54325/s
moose_immutable 60575/s
class_accessor 75348/s
class_spiffy 79642/s
class_accessor_fast 89620/s
class_accessor_fast_xs 107349/s
spiffy 134988/s
class_xsacessor_array 155608/s
class_xsacessor 173799/s
object_tiny 234721/s
object_tiny_xs 287949/s
direct_hash 449616/s
Benchmarking object creation and attribute access
#!/usr/bin/env perl
use warnings;
use strict;
use Benchmark qw(cmpthese timethese:hireswallclock);
package WithMoose;
use Moose;
has myattr => ( is => 'rw' );
package WithMooseImmutable;
use Moose;
has myattr => ( is => 'rw' );
__PACKAGE__->meta->make_immutable;
package WithMouse;
use Mouse;
has myattr => ( is => 'rw' );
package WithClassAccessor;
use base qw(Class::Accessor);
__PACKAGE__->mk_accessors(qw/myattr/);
package WithClassAccessorFast;
use base qw(Class::Accessor::Fast);
__PACKAGE__->mk_accessors(qw/myattr/);
package WithClassAccessorFastXS;
use base qw(Class::Accessor::Fast::XS);
__PACKAGE__->mk_accessors(qw/myattr/);
packag e WithClassAccessorComplex;
use base qw(Class::Accessor::Complex);
__PACKAGE__->mk_new->mk_scalar_accessors(qw/myatt r/);
package WithClassAccessorConstructor;
use base qw(Class::Accessor::Constructor Class::Accessor::Complex);
__PACKAGE__->mk_constructor->mk_scalar_accessors(qw/ myattr/);
package WithMojo;
use base qw(Mojo::Base);
__PACKAGE__->attr('myattr');
package WithClassMethodMaker;
use Class::MethodMaker
[ scalar => [ qw/myattr/ ],
new => [ qw/-hash new/ ],
];
package WithAccessors;
use accessors qw(myattr);
sub new { bless {}, shift }
package WithObjectTiny;
use Object::Tiny qw/myattr/;
sub set_myattr { $_[0]->{myattr} = $_[1] }
package WithSpiffy;
use Spiffy -base;
field 'myattr';
package WithClassSpiffy;
use Class::Spiffy -base;
field 'myattr';
package WithClassXSAccessor;
use Class::XSAccessor accessors => { myattr => 'myattr' };
sub new {my $class = shift; bless {@_} => $class}
package WithClassXSAccessorArray;
use Class::XSAccessor::Array accessors=> { myattr => 0 };
sub new {my $class = shift; my %args = @_; bless [$args{myattr}] => $class}
package WithObjectTinyXS;
use Object::Tiny qw/myattr/;
use Class::XSAccessor accessors => { myattr => 'myattr' }, replace => 1;
package main;
cmpthese(timethese(-5,{
moose => sub {
my $obj = WithMoose->new(myattr => 27);
my $x = $obj->myattr;
},
moose_immutable => sub {
my $obj = WithMooseImmutable->new(myattr => 27);
my $x = $obj->myattr;
},
mouse => sub {
my $obj = WithMouse->new(myattr => 27);
my $x = $obj->myattr;
},
class_accessor => sub {
my $obj = WithClassAccessor->new({ myattr => 27 });
my $x = $obj->myattr;
},
class_accessor_fast => sub {
my $obj = WithClassAccessorFast->new({ myattr => 27 });
my $x = $obj->myattr;
},
class_accessor_fast_xs => sub {
my $obj = WithClassAccessorFastXS->new({ myattr => 27 });
my $x = $obj->myattr;
},
class_accessor_complex => sub {
my $obj = WithClassAccessorComplex->new(myattr => 27);
my $x = $obj->myattr;
},
class_accessor_constructor => sub {
my $obj = WithClassAccessorConstructor->new(myattr => 27);
my $x = $obj->myattr;
},
mojo => sub {
my $obj = WithMojo->new(myattr => 27);
my $x = $obj->myattr;
},
class_methodmaker => sub {
my $obj = WithClassMethodMaker->new(myattr => 27);
my $x = $obj->myattr;
},
object_tiny => sub {
my $obj = WithObjectTiny->new(myattr => 27);
my $x = $obj->myattr;
},
spiffy => sub {
my $obj = WithSpiffy->new(myattr => 27);
my $x = $obj->myattr;
},
class_spiffy => sub {
my $obj = WithClassSpiffy->new(myattr => 27);
my $x = $obj->myattr;
},
direct_hash => sub {
my $h = {};
$h->{myattr} = 27;
my $x = $h->{myattr};
},
object_tiny_xs => sub {
my $obj = WithObjectTinyXS->new(myattr => 27);
my $x = $obj->myattr;
},
class_xsacessor => sub {
my $obj = WithClassXSAccessor->new(myattr => 27);
my $x = $obj->myattr;
},
class_xsacessor_array => sub {
my $obj = WithClassXSAccessorArray->new(myattr => 27);
my $x = $obj->myattr;
},
}));
Results:
class_accessor 159366/s
mojo 240146/s
class_spiffy 250628/s
mouse 270458/s
spiffy 277170/s
moose 288516/s
moose_immutable 293403/s
class_accessor_fast 295518/s
class_accessor_constructor 302447/s
accessors 318795/s
class_accessor_complex 352492/s
class_methodmaker 389809/s
object_tiny 435876/s
class_xsaccessor 537101/s
class_accessor_fast_xs 572424/s
class_xsaccessor_array 620285/s
object_tiny_xs 638328/s
direct_hash 1183721/s
I'll leave the analysis of the results to the reader.
Gabor's post on Perl application distribution
Since Gabor has disabled comments in his blog, I'll have to write in my own.
This is about Gabor Szabo's post about Perl application distribution.
There is a whole lot of things that need taking care of with a scheme like the one that Gabor outlines. The whole notion of managing dependencies gives me headache. In my experience, there are very few distribution schemes which work well (e.g. dpkg, CPAN) and many which don't work quite as well (various CPAN-alikes for other languages, PPM). Sometimes, it's a conceptual issue, sometimes just that such a beast carries a large technical and maintenance burden.
For example, I once created a PAR::Repository with (almost) the whole of CPAN in it. Took a horribly long time to generate... and was slow as molasses because of some technical issues. (In this case, it was a combination of large DBM::Deep databases and the dynamic dependency resolution of PAR::Repository::Client. It would be somewhat faster today.)
But Gabor suggests reducing this whole nightmare to a relatively "flat" dependency tree. It shifts parts of the problem to the developer of the app (need to include all dependencies which are not part of the platform) and eliminates some.
I could go on for ages about this and Gabor has had to suffer through that already, but what I really wanted to point out is that a tiny part is solved already: Soon, I'll release a tool that can install an arbitrary
Tired of ignorant users
Warning. A bit of a frustrated rant ahead.
I have many, many distributions on the CPAN and have been struggling very hard to support them as well as I possibly can. Unsurprisingly, providing user support and fixing bugs is not the only thing I do and certainly not the only thing I like to do. In order to be a good CPAN citizen, I have adopted various unmaintained modules and applied tiny changes or made large overhauls resulting in a total of 488
One of those distributions is PAR, the Perl Archive Toolkit, and many are related. If you happen not to know what that is, it's a set of tools for packaging, managing, and distributing binary packages of Perl code and related resources and was written by Audrey Tang. It's a complex and very system dependent piece of software and I'm very, very glad that unlike many other packages in my CPAN directory, I'm not the only one who works on it. There is an active mailing list with quite a few incredibly nice, competent, and patient subscribers. In particular, there are various people who have a better overview over the code and its peculiarities on some platforms than I do. (I don't do Windows nor MacOS, for example.)
Now, each and every manual page of a PAR-related module states explicitly that support requests shall go to the PAR mailing list and bugs to the request tracker (which also sends a copy to the list, thanks to Jesse Vincent's tireless work). The PAR homepage prominently displays that same information and nowhere does it as much as mention me.
Regardless, I regularly receive support requests for the PAR modules from users to my personal e-mail address. I'm glad to hear they're using PAR, but they're usually very badly written and do not even give enough information for me to help them.
So far, I think I have answered all of these mails except maybe a handful that were either extremely impertinent or just unlucky because I was much busier than usual.
I started replying with an answer to their problem if I had one. Eventually, I grew tired of that and gave them their answer after noting that they should follow up on the mailing list.
This, however, has turned out to be highly ineffective. Those who got their answer usually didn't end up participating in the mailing list but just disappeared again.
How do you solve this dilemma? I still want to try to help wherever I can, but should I really spread my spare time thin between those who read the documentation, do some testing, and write a well-phrased mail to the mailing list and those who just send me a two-line mail saying that my shit doesn't install on their computer with an unspecified operating system? I'm exaggerating here. There's lots of shades in between.
Replying simply RTFM or Send it to the mailing list and include your system details and logs feels really rude. Forwarding the terrible thing to the list seems much worse: I'd be rude to those people who spend their spare time supporting the software. I'm actually thinking about not answering those mails at all unless they seem particularly interesting.
I can't be the only one in this situation. How do you handle this?
PAUSE on CPAN, indexing $stuff
Most likely, everybody who reads this directly or indirectly depends on the operation of the PAUSE indexer (aka mldistwatch). The PAUSE indexer scans new distributions on the CPAN (really on PAUSE at that point) for the packages/namespaces and associated versions they contain, sends the uploader a friendly message with the results, and adds the information to the metadata that's used by our toolchain when people install modules from the CPAN.
The PAUSE code was written and is still maintained by Andreas König. It's a rather large and unquestionably a rather complex piece of software.
I don't think I'm giving anybody a big surprise if I say that being able to run this same indexer on a given tarball or zip offline may be useful for some toolchain modules. One example would be generating the META.yml provides section.
At the social event of YAPC::EU 2008, Andreas and a posse of other PAUSE admins, including me, sat down to talk about the directions our tools are heading as well as policies. I don't think anybody disagreed to that it'd be great to have components of PAUSE available individually from CPAN. But that's one ambitious goal!
A long time ago, I had spent a significant amount of time on porting the PAUSE indexer code in order to be able to index PAR distributions for injection into a PAR::Repository. I could do all sorts of simplifications for that purpose. For example,
Last night, I decided to give it a shot at making the PAUSE indexer it's own CPAN module.
But I failed.
It turned out to be very, very tightly woven into the whole PAUSE code. I'm really not sure how I got the PAR file scanner to work on the basis of the PAUSE indexer. So I switched to a less ambitious goal: split the PAR indexer out of the PAR::Repository code into the PAR::Indexer module (and distribution) for general consumption.
That's where it stands today. For the future, I figure adding some code back into the mix and making a more generic indexer distribution would get it up to producing 98% of the same results as the real PAUSE indexer. I can do this, but:
Now I'd like to know, would you consider this useful?
And a challenge for all the testing gurus: How would you try to exhaustively test this thing agains the PAUSE indexer?
Cheers,
Steffen
help them think they understand.
--Larry Wall
