use Perl

Four years ago, I read about 9-block quilt patterns and I turned it into a perl program and article, purely for amusement-value. And a Christmas present.

Someone else has taken the original idea and made them into a security feature for web discussion-boards to prevent spoofing. It's a one-way hash from someone's IP address into a unique picture. Quite a clever idea, I think.

I noticed a few weeks ago when these shapes started showing up on wordpress blogs. It seems they've added it as a standard wordpress plugin option. It will take me a while before I don't do a double-take each time I see them though.

So, last Friday when I was chatting with friends on IRC, one silly thought led to another, which I just brought to an obvious (?) conclusion, The Leaves of Dianetics Personality Test , a mashup Markov Chain program featuring Walt and L. Ron, together (hopefully) for the first time.

Some sample output:

Do you consider that you read? When recounting some amusing incident can you easily pleased?

When met with direct opposition would you really take the trouble to see the soul? See, your own inferiority?

Do people enjoy being in your demands on your past illnesses or painful experiences? Do you consider the best points of most people and only rarely speak slightingly of them? Do you have little regret on past misfortunes and failures? Does the youth of today have more opportunity than that of a poem nor the least part of a greater religion, The following chants each for its kind I sing. My comrade! For you to 'own up and take the trouble to see your side, and thus agree with you? Do you openly and sincerely admire beauty in other people? Do you sometimes completely unable to enter the spirit of things? Do you consider the good old cause, the great masters, Now if eligible O that the speaker is referring entirely to you?

The code is a Markov chain program by Dave Smith.

Glad to be home.

First this morning, I went to the MVC talk by Perrin Harkins. He compared three web-app frameworks: CGI::Application, Catalyst, and Open Interact 2; his slides may cover other MVCs he didn't have time to discuss too. This talk felt like it should've been longer than 40 minutes, given the content. He was able to illustrate a bit of sample code, but wasn't really able to compare and contrast as deeply as he has in his online writings. I'm left wanting to learn more, but not knowing if I actually have any more additional knowlege than the little bit I wrote below.

So: briefly:

• Catalyst features include very flexible URL handling, which you won't get in OI2 (without hacking). There are tools to generate lots of stub methods, for easy startup. However, documentation on this is shaky; perrin had to read sourcecode for 3-4 modules to get that to work. There are good plugins available, which will illustrate ideomatic code for you. Data relationships are stored in perl modules, such as via Class::DBI::Loader::Relationships perhaps.
• CGI::Application is the most skeletal framework of the three; you end up rolling things yourself or at least, writing enough to piece together some of the (many) good plugins available.
• Open Interact is quite automatic. You turn it on (it runs its own webserver at the outset), click a few buttons, and you have a MVC. There is more abstraction, for those plusses and minuses (he described one level which he found quite irritating; I didn't catch what it was called...) OI is quite appropriate for large-scale projects, where it will be worth the effort to learn. OI uses lots of configuration files (which I believe is where data relationships can be defined).

So, I have some information to go on for my next web app project. It's frustrating; the last MVC I used was Maypole, which I never quite got working well enough to do anything beyond the examples; since then I've re-invented wheels a few times on small db-backed projects, and I'm loathe to do the same again, but at the same time it's not completely clear that the existing frameworks will satisfy either. Guess I need to dive in and try them...

I skipped the next time-slot to check out and chat with people a bit; I was unhappy about missing Casey West's talk on development tools, but luckily (?) he got bumped due to over-runs and got to give a shorter version to everybody in the afternoon. Which was an awesome talk. He ended up with nine wonderful development tools I am pretty excited to learn about:

• SQL::Translator - goes from any DBD to any other; and will even make images, too.
• HTTP::Server::Simple::Static - lets you run a no-muss, no-fuss HTTP server in your web directory, for quick testing.
• Devel::Cover - How much code are your tests covering? Not only provides answers, but spits out very pretty HTML tables of modules and functions which you can drill down into.
• podwebserver - another standalone POD database, but looks quite fully-featured. Will show you the source, and tidy/colourize it for you, too.
• perltidy - speaking of which, perltidy will allow your coding team write using each person's own preferred style, then you can run it through the (highly customizable) perltidy script to make it all standardized.
• Test::Pod::Coverage - self-evident?
• DBI::Shell - command-line tool, db equivalent of perl -e (or possibly perl -de1), with handy switches.
• Module::Refresh - nit sure what this one did
• CPAN::Mini - 600MiB of current CPAN goodness, as opposed to however many GiB are actually available.

After the morning break, I watched Ingy discurse about Data Driven Testing (so, writing tests in "0 lines of code" which actually means writing config files in a very easy to understand format) and Perldoc, his ideas for making POD suck less. I said "discurse" because he has an amazing ability to go off on tangents that are both trival and very important, and if you're not paying at least as much attention as he is, you might miss what he actually said. For example, I completely failed to take notes during them because they were so entertaining. On the other hand, he wrote the URLs for us to follow along; but I missed it. (Edit: and Autrejus told me what it is, below.)

Lunch was with other KW Mongers, down at the Bon Marche on Bay Street. We debriefed a bit about the morning, but I'm not confident enough about what people said to repeat it here. (See what I did when I echoed Allison on Monday?)

What else. Oh, lightning talks. Of particular note for me to look up [this is not meant as a slight against those I don't mention!]:

• The list of all 16 talks is here.
• Regexp::Compost does automatic Regexp decomposition to make them predict /future/ matches as well. Sounds crazy but the demo seemed cool enough that it might just work for some applications (eg., monitoring web logs). By shields@cpan.org.
• SPF was invented at YAPC::NA Florida (03)? I had no idea. Anyway, Meng Wong, the creator, is working on two new projects, Domain Keys and "The Karma Project" for improving site-to-site trust, and "RSS email" as a complete reversal of the ideas behind SMTP. URL: somewhere here and possibly here.
• annocpan.org
• Not Just Another Perl Hacker: The Perl Foundation is soliciting nominations for the best technical innovations in perl. talk to dha.
• wikis and cities: cityguides.com
• the talk of reimplementing CPAN for javascript has found a home; openjsan.org is live, today.
• there was entertaining juggling. As opposed to lame juggling, which would've happened in other fora, but never at YAPC.

  Must go eat dinner now; more later (for some value of later) when I have a chance to look for stuff I only hinted at above... Comments welcome of course; it might be a few weeks before I respond though, because I'm off to the wilds of Virginia till the 12th. [edit: fixed URLs, 6/30]

  Thanks to all the people who made YAPC possible this year (from the organizers to the speakers to...) It was a lot of fun to help out behind the scenes, such as I was able to. Looking forward to next year.

Well aren't I psyched, because I've gotten the oomph to follow day 1 with day 2. Ha, we'll see about day 3 tomorrow. Today I pretty much followed the databases track, with a detour at the beginning for 1.5 hours of perl 6 and Pugs. I'm quite impressed with how the perl 6 projects are coming along, though, as Eric/fishbot just told me, he was excited to see how the specs for Grammars have changed since he last looked at them. Now, I'll be excited to learn them when the design is finalized, and I'm excited for neat design development happening, but I'm not a language designer so I'll leave those discussions for those who are competent at such things.

In the database track, I saw a total of 4 Class::DBI talks of various sorts; I'm not convinced it's as useful as some claim, but I'm not actually warned away by the equally strong warnings I've heard from other smart people. Class::DBI seems just right for projects that can deal with a simple model. I suppose the first time I hit a wall with it, it'll be clear where the limitations are. But it seems it can handle many-to-many relationships, the thing I was most concerned about.

I need to look at Shawn Sorichetti's weblog, which will have interesting notes about IMA::DBI, and also I should track down the wiki for Class::DBI.

Casey West's talk about advanced Class::DBI included a number of modules to look at; which will shorten the amount of code you write, even further.

• Database-specific extensions: there are subclasses of Class::DBI for each of the major DBDs; each has a 'set_up_table' method which will semi-automatically create the tables you need, at compile time (for better and worse).
• Automatic Class creation: Class::DBI::Loader has a method 'find_class' to shorten class-creation code...
• Automatic Relationships: Class::DBI::Loader::Relationships will set up simple cases of relationships. Also, Loader has a 'relationships' flag to look into.
• Globally Unique Keys: ::Loader has parameter for 'additional_classes => [qw [Class::DBI::UUID]] whih uses/adds UUID field for able; it's DB independent and creates 16-character string UUIDs.
• Automatic XML Serialization: Class::DBI::AsXML (I think this was written by Casey): has 'to_xml_columns' method which will dump XML definitions for your data (non-recursively, I think; it might need more work)
• Automatic CGI Input: Class::DBI::FromCGI : the name is a misnomer; it can be used for many things besides CGI inputs. It will allow you to define columns/form-entries to untaint (as per type; you can specify integer, printable, etc... Pseudocode:

  use CGI::Untaint;
  # create handler
  my $h = CGI::Untaint->new(CGI->new->Vars):
  my $a = $camoose->create_from_cgi($h);

  Class::DBI::AsForm creates populated or empty form.

Advice: Don't: write a Class::DBI subclass; that namespace is already too populated. Don't reinvent classes already on CPAN.

Do: write a Class::DBI mixin; use exporting to provide your plugin; and use Class::DBI internals.

I need to read up in the docs, and possibly ask Casey what he meant for the last two Do's.

Dan Friedman's talk on Class::DBI::DataMigration describes a framework for basically building a better database programatically, but mostly just requiring a YAML config file that will do the translations. Which certainly sounds easy.

As for non-Class::DBI talks: I enjoyed the Yahoo tlak on high-volume databases and the high-availability talk which suggests corporate appropriateness for MySQL (hm...). Finally, I got a little bit of graph theory schoolin' with Walt Makowski's talk on Graph Theory and one instance when to use trees instead of sets to describe data so it can be combined more quickly.

Gotta go run and catch a boat-cruise. :)

[edit: back. Boy that was a long auction. Why did I think it would feel shorter if it were on a 4-hour boat-cruise? The conversations were excellent, though.]

A review of YAPC day 1.

Larry Wall's keynote was quite high-level and syncretic. He enthused about tensegrity, the engineering design principle that opposing forces properly balanced can provide tension and integrity at the same time. The talk was based on one he was invited to give in Russia, on how to build an open-source community.

Allison Randal's "state of the carrot" was a snapshot of where perl 5 and perl 6 design are right now. I didn't take notes, but there were some perl6 milestones released the day before, and everything seems to be progressing. PUGs has 80% of perl6 features, wow. [edit: This should be corrected to <30% or 10% (based on fraction of design that has test cases to cover it; see Autrijus's note below!)]

When we all split up, I started off in the "Enterprise Workflow" talk by Jim Brandt, which described how he and his group at the U of Buffalo used a few modules to save their University lots of time and money. The modules were: Class::DBI, Data::FormValidator, and CGI::Prototype. I've not actually done coding with Class::DBI, though I've read about it. Some people seem to dislike it intensely, for any sort of difficult projects, because it creates as much work as it saves. But for simple database lookups and manipulation, it seems like a timesaver.

Then, becasue we were running late and I was hungry, I ducked out and got lunch instead of going to the second talk...

For the afternoon, I took the easiest route and went for the session on Kwiki with Brian Ingerson. I probably would've decided differently if all of the talks weren't being recorded, but as it is, the other talks looked more formal and his looked more freeform. Sure enough, he started off by saying he was going to one-up Autrijus who wrote the first half of a talk on the train to the client's site, then wrote the second half during the intermission.

Ingy's talk had the first half written last night at 4am, and the second half was going to be written by us.

Sure enough, the first half he presented slides on the architecture of his freeform wiki.

The thing is entirely plugins based, and has interesting abilities to make plugins interact and do the Right Thing with the least amount of code. He invented a number of techniques from scratch, he says because he's too lazy to research how other people have done things. TIMTOWTDI, right? Yeah. The plan is for the design to be a lego set for perl hackers.

Stuff he wrote that he thinks is cool: 'spiffy': a source-filter that adds 'use warnings' and 'strict' and a trailing '1' to each package file, and 'my $self=shift' to each method (this is turned on if you 'use Spiffy qw( -Base )'. -base won't include those. mixins: anonymous class between base-class and current class. like in Ruby. allows for a simple inheritence chain. Allows you to wedge stuff to replace or enhance methods which were either defined or inherited from a parent-class. And, he wrote a program registry, which is shared between plugins, and has some funky interactions which make sense but I didn't write down how it works, sorry.

He decided, when the plugin modules started exploding in number (there are well over 100 now by over 10 main authors), that CPAN wasn't the optimal way to release, since dependencies and versions make things slow to install. Instead, he recommends checking out his svn.kwiki.org.

Interesting feature: 'kwiki -view [blah]' will create a new kwiki named blah, with its own configuration files, but whose data and metadata are symbolicly linked from the parent directory. So it's easy to set up an admin view with extra features.

Brian's not terribly excited by wikis; they were just a means to playing with these new ideas. Spiffy has universal improvements. Spoon is generic techniques that seem interesting outside Kwiki. and Kwiki is built on Spoon.

Spiffy has a function named 'xxx' which will die at that point and do a stack-trace and some sort of variable dump.

I asked about documentation, which is... sparse... and how one is to start learning this when everything is tied together so intimately. He pointed me at doolittle.kwiki.org, which is a wiki which incorporates all of the POD stripped from the live version; (and I believe, checks the POD back into the SVN for the files) when he releases to CPAN, he does a once-over of doolittle and it gets released along with. So the docs are written by the community. In theory. I haven't taken a look at it yet...

The second half was organized around a pair-programming session with him and Chris Dent, where they built a plugin to do tags, like Delicious. It seemed pretty straight-forward, though, I would have loved to have asked more questions about the bugs they worked through. However, I did learn a bunch about debugging kwiki.

Afterward, I met up with other kw.pm mongers, and agreed we'd meet at Simon's house later for beer and chat, and possibly coding.

I was going to precede that with an RT (request-tracker) BOF, which would've been useful for work, but they left before I could get myself sorted out. I'll have to meet Jesse Vincent later on. So instead, I rode around on a segway for a little bit.

So that was my afternoon. Listening to other peoples' afternoons, I'm glad I went for less-structured. Maybe I'll do the perl6 track on Tuesday. Dunno.

In the evening, Simon, Eric and I set up our laptops and started hacking on Bots and Kwiki. Shuchit, Arguile and Greg came a bit later, and we had really good pizza.

We saw a few Futurama episodes (including the Popplers one, yay) and took the train back.

OK, this will be an attempt to journal my thoughts about this year's YAPC. Non technical notes will go in my livejournal. Who knows whether I'll have time to keep up both; nor which one will get priority on a particular day.

Since taking a full-time job last month, I've had a pile of smallish project-requests fall my way. I've been directing people to the perl jobs list, but I'd like to do better than that because these are all for friends or colleagues who I'd like to do the right thing by.

Please drop me email for contact information. Here's the top two gigs:

About one month's work cleaning up a batch of 20 scripts used by an IC fabrication plant. Lots of DBI.

Hand-holding and basic CGI for a website design firm in Vermont. Likely ongoing work in 10-20 hour chunks. My most recent project was building a mySQL membership directory, moving scripts from a windows to linux host, and "cleaning up" some awful CGI, but they didn't have time to pay me to rewrite the bad stuff from scratch.

I had fun today playing with GD.pm and quilting patterns. In "nine blocks" there are 9 squares, whose contents are limited to 16 basic patterns and radial symetry.

Here's a sample output: 3x3

The idea comes from this guy who has done some amazing artistry with Flash programming. While he has open-sourced his code, I don't have access to Macromedia products so I built it from the specs on that web page.

I'm currently planning to print out a whole lot of them on a sheet of photographic paper, as a present for a quilter.

It's got problems with it, but it's a start.

#!/usr/bin/perl
# nineblocks.pl - create assorted quilt patterns using 16 graphical primitives
#   and radial symetry.

#   Copyright (C) 2004 Daniel Allen.  It is distributed under the same
#   terms as Perl itself.  See the "Artistic License" in the Perl
#   source code distribution for licensing terms.

#   inspired by http://www.complexification.net/gallery/machines/nineblock/

use warnings;
use strict;
use GD;

my $XS = 20;    # dimensions of squares in pixels
my $XN = $XS*5; # dimensions of nineblocks in pixels

my $dim = 10;    # number of squares wide/tall
my $border = $XN/2;

my @squares = (
               "",
               ".5,1 1,1 1,.5",
               ".5,.5 .5,1 1,.5",
               "0,0 .5,.5 1,0",
               "0,.5 1,.5, .5,0",
               ".5,.5 .5,1 1,1 1,.5",
               ".25,.25 .25,.75 .75,.75 .75,.25",
               "0,1 .5,1 .5,.5 1,.5 1,0",
               "0,.5 1,1 .5,0",
               "0,0 .25,.5 .5,0 1,0 .75,.5 .5,0",
              # "0,0 .25,.5 .5,0 1,0 .75,.5 .5,0 .5,1, + .25,.5 .75,.5",
               "0,0 0,.5 1,1 .5,0",
               ".5,0 .5,1 1,1 1,0",
               "0,.5 .5,1 1,.5 .5,0",
               "0,0 .5,1 1,0",
               "0,1 1,1 1,0",
               "0,0 1,0 1,1 0,1"
               );

my @centers = (
              "",
              ".25,.25 .25,.75 .75,.75 .75,.25",
              "0,.5 .5,1 1,.5 .5,0",
              "0,0 1,0 1,1 0,1"
              );

my $image = new GD::Image($dim * $XN, $dim * $XN);
my $white = $image->colorAllocate(255,255,255); # background

my $xpos = 0;
my $ypos = 0;

OUT: while (1) {
    my $block = &nineblock(&r(3), &r(3), &r(15), &r(15), &r(3));
    $image->copy($block, $xpos, $ypos, 0, 0, $XN, $XN);
    $xpos += $XN;
    if ($xpos > (($dim ) * $XN)) {
        $xpos = 0;
        $ypos += $XN;
        if ($ypos > (($dim ) * $XN)) {
            last OUT;
        }
    }
}

# nineblocks are at left-top corner of their background
# (which might be useful for colored backgrounds, but not for white)
#
# new background should be: + 2 * border - the total offset of each 9square

my $background = new GD::Image($dim*$XN + 2*$border - 2*$XN/5,
                               $dim*$XN + 2*$border - 2*$XN/5);

$white = $background->colorAllocate(255,255,255);
$background->copy($image, $border, $border, 0, 0, $dim * $XN, $dim * $XN);

&display($background);

sub r {
    my ($max) = @_;
    int rand $max;
}

sub nineblock {
    # edgeRot     = 0-3 ( times 90 degrees)
    # cornerRot   = 0-3
    # edgeShape   = 0-15
    # cornerShape = 0-15
    # centerShape = 0-3

    my ($edgeRot, $cornerRot, $edgeShape, $cornerShape, $centerShape) = @_;

    my $image = new GD::Image(3*$XS,3*$XS);
    my $xpos = 0;

    my @face = (&square($edgeRot,   $squares[$edgeShape]),
                &square($cornerRot, $squares[$cornerShape]));

    my $center = &square(0, $centers[$centerShape]);

    $image->copy($center,
                 $XS, $XS, 0, 0, $XS, $XS);

    for (@face) {
        my $square = $_;#6&square($_);
        # $destimage->copy(srcImage, destX, destY, srcX, srcY, width, height)
        #
        # $destimage->copyResized(srcImage, destX, destY, srcX, srcY,
        #                         destWidth, destHeight, srcWidth, srcHeight)

        my $max = (2*$XS - $xpos);
        $image->copy($square, $xpos, 0, 0, 0, $XS, $XS);
        $image->copy($square->copyRotate90, 2*$XS, $xpos, 0, 0, $XS, $XS);
        $image->copy($square->copyRotate180, $max, 2*$XS, 0, 0, $XS, $XS);
        $image->copy($square->copyRotate270, 0, $max, 0, 0, $XS, $XS);
        $xpos += $XS;
    }
    return $image;
}

sub square {
    my ($rot, $points) = @_;

    my $image = new GD::Image($XS,$XS);

    my $polygon1 = new GD::Polygon;

    #my $white = $image->colorAllocate(&r(255),0,&r(255));
    my $white = $image->colorAllocate(255,255,255);
    my $black = $image->colorAllocate(&r(255),0,&r(255));
    #my $black = $image->colorAllocate(0,0,0);

    &add_points($polygon1, $points);

    $image->filledPolygon($polygon1, $black);
    $image->flipVertical;
    for (1 .. $rot) {
        $image = $image->copyRotate90;
    }
    return $image;
}

sub add_points {
    my ($poly, $points) = @_;

    foreach my $pair (split /\s/, $points) {
        $pair =~ /([\d.]+),([\d.]+)/;
        $poly->addPt($1 * $XS, $2 * $XS);
    }
}

sub display {
    my ($image) = @_;

    #   'display' program provided with ImageMagick
    open OUTFILE, "| display -" or die "couldn't open display";
    print OUTFILE $image->png;
    close OUTFILE;
}

This morning, I locked my keys in the car. On a hot day. With the dog also in the car. Due to Rover, though, I had the rear window cracked, so I successfully snagged the keys from the ignition with two coat-hangers attached together.

The timing was particularly bad, because I was on the way to pick up my sweetie at the airport, and was early enough that I figured I'd pick up lunch on the way.

Oops.

So, this evening, I feel I need to breathe a huge sigh of thanks that the first time I've ever done this, I did it with the window cracked open, so that I only inconvenienced myself for half an hour, instead of inconveniencing both of us for half a day while some tow-truck driver did their magick.

The learning experience this time seems to be: have backups. Preferably, a box with a good magnet stuck to the bottom of the car. And, of course, always check for keys before leaving the car.

GD::Text::Arc is an addition to GD and GD::Text that simplifies drawing text on a curve. Sample output is here. It's not a terribly complicated module, in fact it's about 250 lines of code, but it does just what I want it to, so I'm happy. The most time-consuming part of the process was writing the 250 lines of documentation.

This was also my first foray into writing tests while I wrote code; that was just as useful as everyone says it is, so I think I'm a convert. Of course, I'm sure I need way more than 46 tests, but it's a fine first draft.

Now that I'm all puffed up with releasing my first CPAN module and stuff, I'd love for people to pick it apart and tell me what I did badly. :-)