Slash Boxes
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

marcus (4582)


My Modules:
GraphViz::ISA::Multi (unreleased)

Computer geek who studies computer science to become a Master of his Science, playing with Parrot, Perl and all the rest just for fun - did I mention geek?

Journal of marcus (4582)

Monday October 27, 2003
05:13 PM

The Perl Standard Library

[ #15418 ]
When I look at that what I would call the Perl library at the moment, the CPAN, I see a whole bunch of modules (more than 14.000 when I remember my last count), some of them are OO, some are not and we got several different efforts running at the moment to make the situation, which is not bad but strange, better.

Thinking about CPAN related projects that is what comes to my mind:
- The Phalanx Project
- The Core Modules

The problem of all this is that we miss a real structure, something I will call the Perl Standard Library (PSL) from now on.

See, we got all this beautiful modules which all more or less work with some version of Perl, some OS and some architecture. That is nice, as most of them work for me, on a Linux i386 box. Other platforms are hard to tell, e.g. Windows and compiling something can be really hard.

The Phalanx project choose 100 modules, as I understand it to make it more or less the testing platform for Ponie, and try to improve them to be better modules(TM). CPANPLUS is an effort to make the management of the installed modules easier. CPANTS and make it more easy for the module developers to see their modules tested on hardware and version they don't have. The Core Modules (and the bad dual life modules) make Perl pumpkining harder (I guess so) and improve the basic functionality that Perl has. The Problem is that some useful modules are not Core (and maybe some useless modules are in core :-)). The next thing that is important in connection with module management is my OS vendor, or my Distribution as it is called in the Linux world. They bring a couple of modules, some CPAN original and just build for my version of the OS, or some changed ones, because a Dead Camel did not move and they needed the lib (as was a long time, at least SuSE, RedHat and Debian had their own patches because it didn't compile with 5.8.0). This all ends up in a big BLOB. I can't really tell by heart which modules are installed on my system, I'm not sure about every version number and can't promise that something I wrote works on every system. I don't have nothing to depend on when writing software, I don't have no Library. In my opinion a language is only as strong as it's library, but the problem with the library of my favorite language (guess what) is that it's library is maintained by over 1000 people and it has nothing you can rely on.

What I want to say is that we should take a bunch of modules (as the Phalanx Project does), put the together, make it stable at mark it the PSL for 5.8.1. I don't have a list of modules I want to see in there, but I think it would be an advantage for Perl if we would take the material we got, take the material others contribute (like the modules which the distro vendors choose) and make something up we could depend on in eternity (or at least for the time Perl exists). When we have bundled it we could make another effort do restructure it, as I don't believe that the CPAN structure would fit all the time. Then we can improve documentation and make everything more homogeneous. One of the other problems with the CPAN is that there is no document I can think of that says how to do what with which module. There are many books out there that recommend many modules, but there is no single "Using CPAN" book out there that covers all of CPAN.

To make myself clear again, what I would like would be a PSL project, that provides a big, stable and maybe pure OO library for Perl, version dependent, with easy to access documentation and that has obvious responsibilies.

This is just an idea, maybe I have to restructure some parts and think over others.

Another thing that came to my life lately was that another thing I'm missing at the moment is easier to install modules. I installed today just to recognize that it didn't compile because I had no ID3Lib installed. This is rather obvious but often it's hard to tell which libraries a Perl module depends on. Maybe this is something for the META files. The thing that I really didn't like was that the lib was just a line away (on my Debian) it's just 'apt-get install libid3'. I'd like to have something like CPANPLUS::Distro::Backend that would manage such thing for me. It's not very realistic to get something like that for every system, but if you can say the developer install lib foo it's more than just a missing include file. Would be some kind of community effort, but maybe it is possible.

Enough revolutionary ideas for today :-)
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
More | Login | Reply
Loading... please wait.
  • You will want to be visiting the graves of those who have come before you [] to honor those who have fallen attempting to create anything standard out of Perl.

    Any attempt to create a list of "standard" modules for Perl will fail. Why? What's "standard"? Presumably you'll want the standard library to cover the things standard Perl users try to do. Who is a "standard" Perl user? What do people typically use Perl for? The answer is, of course, everything. Everything from one-liners to hundred thousand line applications that control billions of dollars. So you have to package everything or risk alienating one group or another.

    The other problem is backwards compatibility. Once a module is in the standard library, can it be removed? No, because its a standard. Things change, rapidly. Look at the modules we ship with 5.6.0 and pick off how many are really used and how much is just historical cruft. Even probably wouldn't make it into the core if it was introduced today, the CGI fad having worn off. Any Technology Du Jour added to a standard library cannot be removed after it becomes passe. Standard libraries can only become larger. After a certain point, size becomes more important than tracking new, useful "standard" technologies and the standard lib starts falling behind. Look at the ANSI C standard library. No graphics. No web. Very little to do with the Internet at all. No regexes. No decent memory allocation.

    Creating a big list of modules will never work. Here's what might.

    "Best of breed" lists. A common use of CPAN is a user comes along and says "I need a module to do X" and then either can't find one because its not called what he/she expects or finds five of them and doesn't know which one to use. Developing answers to those common questions is a good way to solve a real problem. The trick is finding out what the questions are. Even relatively innocent attempts like, "What's the best XML parser on CPAN?" are fraught with peril as the perl-sdk folks found out. Its too simplistic a question since there are many different types of XML parsers and many different ways they're used. You have to avoid favoring one use style over another (DOM vs SAX) or one use case over another (CPU vs memory vs programmer efficiency) or weighing portability (a pure Perl module) vs speed (a libxml based module). And then there are those of us who don't care about XML. ;)

    Even best of breed lists fail because they're lists and inevitably favor one set of module attributes over another. Program speed vs portability. OO vs functional. Finding "the best" is meaningless in a zero sum game. So an evolution of the best of breed list is module metadata. Extend the module description with all the information a person might need to make an intellegent decision of which module to use based on their requirements. And do it in a way that the user can easily translate their desires into a search query. I don't know how to do this.

    Finally, there is one list of standard modules that will work. Modules to get more modules. Modules that make working with CPAN easier. MakeMaker, Module::Build, CPANPLUS, Archive::Tar, Compress::Zlib, Test::More, etc... In the face of "Perl is used for everything" this is really the only list of core modules that makes sense. The only thing we know everyone is going to use Perl for is to get modules. This looks like its becoming the new standard for core module inclusion.

    PS There's about 5500 modules on CPAN, not 14000 (there might be 14000 including old versions, though).

  • there are two camps (Score:3, Informative)

    by hfb (74) on 2003.10.28 4:00 (#25236) Homepage Journal

    The idiots who rip on Jarkko for being the worst pumpkin in the history of perl for pumping up the core with so many modules and the others who think it's ok. These former are not so easily found since they tend to hide under rocks, but the drawback of having so many ways to do something and on CPAN is that you're screwed any way you try to do something because some asshole is going to think some other way, their way, is better. So, fuck them and the horse they rode in on...let them install their own damn modules.

    An SDK would require incredible amounts of testing and packaging for binary distributions. At one point in time Kurt Starsnick was going to set up a test lab for such things but I think the whole idea just died due to lack of enthusiasm.

    Also, you might find the CPAN FAQ [], the one noone ever reads, helpful for figuring out how to find out what modules are on your system, etc.