Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Alias (5735)

Alias
  (email not shown publicly)
http://ali.as/

Journal of Alias (5735)

Saturday September 23, 2006
11:14 PM

Module::Inspector 0.01 - Incoming Module::Madness!

[ #31104 ]

It's funny sometimes how easy it is to get distracted by things you need to make your programming life comfortable, before you can effectively move up to the next layer.

I for one spent a hell of a lot of time writing class-level utility distributions, from my first module Class::Autouse (run-time class loading) to a wide range of other modules in Class:: and a dozen other places.

I find these days I feel like I have pretty much have everything I need in order to write large volumes of clean, correct, fast (well fastish, but you can't please the hardcore optimizers who don't even like using hashref objects) distributions.

This has helped enormously in my commercial life, because it means I can churn out work and be sure that it will run on every platform, be unlikely to suffer from edge-case bugs, and generally just do what I want.

But this has led me to my next big problem.

As attractive as CPAN-like distributions are for building up large amounts of functionality in a controlled manner, they aren't very easy to work with once you have them packaged.

And I don't mean 1 at a time. One CPAN dist is easy to handle.

I mean 10 or 20 at a time, with all the usual CPAN dependencies and inter-dist dependencies. CPAN is very much an isolated system, and it can be hard to get similar benefits for your own stacks of distributions outside of CPAN.

That's not to say there aren't solutions. CPAN::Mini::Inject lets you tack some distributions onto a minicpan mirror, and it's a good approach. But it's very simplistic and written sysadmin-style, by which I mean it does one specific task from beginning to end, and doesn't break up the problem in a generalised manner, so that the individual solutions to the various parts of the overall problem can be reused later.

This is a similar situation to that which existed before I created Class::Inspector. It's not that you CAN'T do some of these things, but it isn't all prettified and been nicely encapsulated behind a simple interface.

So I plan to do something about that! (i.e. Time for another dist-writing spurt) :)

To kick things off, I've uploaded a very simple initial release of Module::Inspector. I plan this to be to distributions, what Class::Inspector is to classes.

That is, it's a module you can point at a tarball file, or an unrolled dist dir, or even something in a version control checkout (SVN and CVS are currently detected), and it will do whatever steps are necesary behind the scenes to make it easy to analyze complete modules.

Internally, I'm writing it in two layers.

The first layer is a document layer. This will look through the distribution and tag any files it can with a document class. In the first upload, this only covers "Perl files" (tagged as PPI::Document::File objects) and META.yaml (tagged as a YAML::Tiny object).

So to find the name and version from the META.yml document for a tarball, you would just do...

    my $module = Module::Inspector->new(
            dist_file => 'Config-Tiny-2.00.tar.gz',
            );

    my $meta_yml = $module->document('META.yml');

    print "This is $meta_yml->{name} version $meta_yml->{version}\n";

Granted, this doesn't give you a whole lot, but it automates away the tedium of loading files and unzipping tarballs, and detecting (and compensating for) version control, and so on.

I plan to gradually expand the number of types of files supported, initially based on the sorts of tasks I need to do.

Some of these will be fairly trivial. While there is currently an imperative library for working with MANIFEST files, there's currently no OO/document module.

So for the next 0.02 release of Module::Inspector I've also written a simple Module::Manifest distribution (to be uploaded seperately) which can parse the Perl-specific MANIFEST format.

And as I need support for more files, I'll be writing any dists needed for this, or reusing existing ones as appropriate.

The second layer I'm planning to add to Module::Inspector is a more DWIM layer.

This will allow better ease of use for simple cases, with the methods loading documents and doing whatever behind the scenes.

So things like this should just work.

    Module::Inspector->new( 'sometarball' )->dist_name;

Again, that seems fairly simplistic, but means that we have a good object model for distributions that allows the NEXT layer to be built relatively easily.

As for what this next layer will do, I have started on a Module::Collection (linked for future readers) implementation.

This will let me deal with a whole group of distributions at once.

If any of you have seen the way I structure my repositories, you'll have noticed I generally have a "/releases" directory which contains all the tarballs.

What I'd like is to be able to work with all of these at once.

So for a typical project, I'd like to (for example) take the entire releases directory, identify all the distinct distribution names in it, select the highest version of each, map their dependencies from their META.yml data, then inject those dists into the L/LO/LOCAL path of my CPAN installer, fire up CPAN.pm, and install the entire project in one hit.

This gives me a sort of "Go install that directory" functionality that then makes it extremely simple and straight forward to build commercial projects as groups of CPAN dists, and then install the the project, along with CPAN dependencies, anywhere I need to.

And of course, after THAT, I plan to start looking at CPAN::Index a bit more, so that I can take the same Module::Collection, and do an index merge into a minicpan checkout, and so on. And some of the trickier PITA problems are also removed in the process.

So... I'm I missing anything here? Thoughts? Questions?

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • You know, I had a déjà vu when I read what you wrote about the Module::Collection thing. It's slightly different but solves a similar problem. Three points if you guess what I'm thinking of!

    Steffen

    P.S.: Where have you been in the past days?
    • Can't guess off the top of my head, other than maybe CPAN::Mini::Inject...

      As for where I've been, although I've moved into my new apartment, I can't get ADSL or wireless internet (or at least, not that doesn't suck hard) so I'm getting cable internet, which means a 1 week wait for the technician to turn up and connect it.

      Which should happen on Wednesday I believe.

      So for the last week I've been working in offline mode (although a client of mine was nice enough to donate a desk at their office that I'll be us
      • PAR::Repository does some of what you describe. The goals are different, but it also deals with scanning distributions, installing bunches, etc.

        Fortunately, PAR distro as simpler than CPAN distros :)
        • Hmm... if so, perhaps we should look at recycling/sharing some of the code between the two?
          • Hi,

            thought about this again. I might have been a little too quick in my reply. Well, there *are* common ideas in between the two projects. One of the things that triggered this was what you said about selecting specifically the newest versions of a module, etc. That's something PAR::Repository::Client does by default. The code is attached, but you'll realize that it's only as much code because it also deals with arch-type and perl-version which you shouldn't need to care about. PAR::Repository::* also takes
    • I also just noticed Module::Dependency, which is somewhat similar as well, although again, it only aims to do a specific task.