Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Wednesday November 17, 2004
12:04 PM

The Real CPAN Limitation

[ #21897 ]

I'll shortly be leaving my current employer to go work for Kineticode. In the year and a half I've been working here, I've learned some valuable lessons. Perhaps the most important is my first exposure to what can truly be termed "enterprise-class Perl." This isn't my first exposure to coding on such a large scale, but COBOL is such a foreign world that it doesn't count (it was successful partially because of its limitations, not in spite of them.)

What did I learn about large-scale programming with Perl? I learned that I like Ruby. I am no longer convinced that Perl is the best dynamic language for such projects. You've heard the arguments before -- I mostly focus on the poor OO model -- but the strongest counter-argument that I hear is "the CPAN." That's a pretty strong counter-argument. Yet counterintuitively, it's far less persuasive for large-scale projects because those projects are to solve a crucial business problem and that problem is not a reimplementation of CGI.pm.

If I need a general purpose tool that solves a wide range of problems, Perl is an excellent choice, due in large part to the CPAN. However, the core of a large system, the value of a large system, is not in CPAN modules. If it were, everyone could just go out and download them and instantly make money. That is not how the real world works.

The real world is also poorly represented in OO classes as taught in college. (Bear with me as this is relevant to the above.) I remember taking my very first Java class. It was taught by an instructor who seemed to know the language, but was a bit hazy on OO theory. However, this was her very first term teaching and I'll cut her some slack, but she had the problem that I see in many classes. She used examples that don't translate to the real world.

This instructor taught OO design by modeling nouns. A Dog isa Mammal isa Animal. A Firetruck isa Truck isa Vehicle. A BadExample isa DangerousExample. Of course, nouns can still be instructive as teaching tools. Showing why Programmer and Manager roles should not be subclasses of Employee is a great example of OO theory. The reality, though, is that good objects reflect abstract business requirements and as your business is more specialized, your objects are more specialized. Those classes will not be found on the CPAN. They are what truly represents your business value.

This isn't to say that the CPAN is worthless. Modules on the CPAN tend to represent more abstract concepts than Vehicle::Truck::Firetruck and, regardless of the language you use, if there is a large body of prewritten code available and it suits your needs, it may be worth reusing it. However, many of the core features necessary are available in other languages. Today, if we were to rewrite our code from scratch, I think we would reach for Ruby.

Now here's my "ace in the hole" argument: Our Perl is not what you think. Sure, it's easy for me to sit here and say Perl's CPAN isn't the end all and be all when we use it so frequently. However, most (not all) things that we use from the CPAN are either readily available in other languages are are easily cobbled together. The heavy lifting -- those things that are the core value in our code -- has all been built in-house for our needs. Case in point:

# shown here with permission but still munged to hide details
my $dataset = Dataset->new(
    [ qw/rev state title_name xtns polled/ ],
    [ qw/state/ ],
    [ qw/rev/ ],
    $request
);

That little snippet builds a 30 line SQL statement that only shows data the user is allowed to see and creates a dataset that, with a one-line command in our templates, builds a large, paged, sortable report. We can effectively treat our entire database as if everything came from a single table. We just ask for the information we want and how to group and sort it. There is no need to specify the tables we are pulling from or how to join them together. There is also no reason why this needed to be written in Perl.

Of course, we do use an OORDBMS mapper. We wrote our own because the ones that existed when we created ours were in their infancy.

We also use a home-grown xUnit style test suite because Test::Class didn't exist when we wrote our version.

Despite our inventing or reinventing many wheels, we're destroying our only serious competitor in the department I work in. Why? They can't match our productivity. There are actually few companies in the world that can do the specialized data management that we do and that's in large part because of choosing a dynamic language, not because we used Perl.

This doesn't mean that Perl is a bad choice, either. Perl is fantastic, but if you want it to scale, you really need a stronger than average programmer and those can be tough to find. Just because Sally Admin or Joe Web Designer can cobble together a few scripts does not mean that they are qualified to build big systems. Most Perl programmers, in fact, are decidedly not qualified to work on those systems due to the dangers inherent in them. That's why Java has been so successful. Sally Admin may not be qualified to work on the Java system either, but, as she learns, she's less likely to wreak havoc on the system.

This, incidentally, is why I'm bullish on Perl 6. It appears to solve most of my Perl 5 problems. Tim Bunce has already mentioned that he plans to port DBI.pm to Perl 6, a Perl 6 CGI.pm won't be far behind (it might even be first) and I suspect that a few other core tools will quickly get ported. With those in place, and given that the real value in a business is the in-house code, if Perl 6 is stable and runs as fast as we suspect, maybe I won't be eyeing Ruby with such longing.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • This, incidentally, is why I'm bullish on Perl 6. It appears to solve most of my Perl 5 problems. Tim Bunce has already mentioned that he plans to port DBI.pm to Perl 6, a Perl 6 CGI.pm won't be far behind (it might even be first) and I suspect that a few other core tools will quickly get ported. With those in place, and given that the real value in a business is the in-house code, if Perl 6 is stable and runs as fast as we suspect, maybe I won't be eyeing Ruby with such longing.

    What are you expecting to

    • I can certainly work with Ruby more, but only on a side basis. For day to day work, I'm still stuck with Perl. And still happy with Perl, to be fair. It's a great language.

    • What are you expecting to get out of Perl 6 that you won't get from Ruby, other than a speed advantage?

      An upgrade path. (Not that a speed advantage is a bad thing with the amount of traffic we have...)

      Seriously where I work we have about 3 years of Perl code accumulated. Suppose that Ruby is 30% more productive than Perl. Then (by a naive calculation) if we switched, it would be 10 years before we had caught up and passed where we are now. That's not even counting the mess of going through a period w