use Perl

I am not Andy Lester. I do not have piles and piles of advice on getting hired. Every time I go through the hiring process, though, I gain a bit more wisdom to share with applicants. Here is some for you today.

If your application begins with this line, you are immediately junked:

Dear Prospective Employer:

That is all.

Alas, as they're footing the bill, I have to devote my attentions to their particular business interests, which aren't Perl (though I've been stealthily infesting it where I can). So Perl sessions/tutorials are almost completely absent from my OSCON schedule this year. Gotta learn Python (ugh), Eclipse PTP, and more on Actors. The TDD tutorial looks Perl heavy, but we'll see. The rest of my schedule is databases, databases, and more databases. I'll try to steal a couple sessions for Perl stuff (esp. Tim B.'s fancy new profiler stuff), but the few spare slots I have also have some nifty Javascript sessions, and I've recently been splashing about in that pond - there's lots of kewl and sexy stuff going on there, so I gotta get me some more.

I see the Perl monks are planning an ad hoc gathering Wednesday, hopefully I'll be able to hook up. And somehow I have to get over the river to the OBF and back before Mssr. Wall's annual Onion. Or maybe after. Or maybe both ;^}

And then some R&R with friends in Seattle, and a leisurely camping meander back to Reno.

While the price of gas is definitely gonna hurt, its a such a great road trip through the Sierras, Cascades, and Siskiyous (assuming it hasn't all burned down by now) in the topless Jeep that I can't bring myself to fly. So I'll grin and bear it...but mostly grin.

So ct, the guy behind Net::Twitter totally thought ahead when he wrote the library and set it up so that you could set the api[url|host|realm] for authentication … so in case there was ever a twitter clone that supported the same API … things would just work.

Identi.ca just rolled out support for the Twitter API. Things just work .

I sometimes go into a loop where my workflow contains pushing a distribution several times to application host. I prefer working in the editors on my local machine and not in vi/vim on the application hosts (and then I have to pull back stuff to stick it in CVS).

This takes quite a few commands since I have to push it via a jump host via SSH. So I attempt to make the commands I have in my history as one-linerish as possible so I do not have to skip through several steps.

I have been cursing at Module::Build's ./Build dist, since it kept prompting me to decide whether I wanted to overwrite the existing tar-ball with the same name, being the product of the command.

gzip even has a force flag!

Today I went back and had a look at the docs and I fell over the --gzip option.

So now my command looks like this:

% ./Build dist --gzip='/usr/bin/gzip --force'; scp somedistribution-1.00.tar.gz jonasbn@jumphost:~/

I a do not have to answer yes everytime I want to deploy a new distribution.

Next step would be to make the jump host transparent so stuff would go to the designated application server on the other site, I am sure this can be done,propably involving some serious tunneling - I just not had the time to figure out how and we do have quite a few application servers, so I have to figure out some nifty approach.

Hints are welcome...

Journey to OSCON / Module releases
It's 4:30pm in Oregon, the jetlag is pretty bad, and I'm drinking yet another coffee to perk myself up. The thing is, I'm still in Australia, where it's stupid-o'clock in the morning. I figure getting the jetlag out of the way now and sleeping on the plane is going to be better than being awake during the plane and whacked in Portland.

At least, that's the plan.

I've spent all of today, and all of yesterday, and all the day before that working on slides. I'm giving 4.5 hours worth of presentations in tutorials and conference talks, and since my presentations tend to move fairly quickly, I have more than one thousand slides prepared. I'm still adding them, or at least I was until brain crashed.

So, as a "reward" for having worked on so many slides, I'm writing code. At 2:30 in the morning. I've pushed out a new release of IPC::System::Simple. It now has capturex() and systemx(), which are variants of capture() and system() that never ever ever use the shell, even if you use one argument. Thanks to ikegami for encouraging me to implement this.

In order to make sure my Perl Security tutorial matches reality, there's also a new release of Proc::UID making its way to the CPAN. It's mainly a tidy-up of the old code, with better documentation, tests, and build system.

What I haven't got out yet is the stable release of autodie. Yes, I know it's what everyone's waiting for. Before I can release it, I want to walk through my TODO list and figure out what really needs doing, and what can be put off. Unfortunately my spell-point meter is dangerously low from the jet-lag, and casting "comprehend project" is rather challenging right now. Instead there's a good chance I'll be making a release while over the Atlantic, and pushing that to the CPAN once I touch-down.

For those hoping to meet me at OSCON, I'll be landing Saturday, but I won't be very sociable until Monday afternoon, after the stress of my tutorial is over.

api_ref_data $ svn status aggtests/test/xml/xpath/

# OK, svn says there's nothing there.  Let's put something there
api_ref_data $ mv pips3.t !$
mv pips3.t aggtests/test/xml/xpath/

# Yup.  svn say's it is not in the repository
api_ref_data $ svn status aggtests/test/xml/xpath/
?      aggtests/test/xml/xpath/pips3.t
api_ref_data $ svn add aggtests/test/xml/xpath/pips3.t
A         aggtests/test/xml/xpath/pips3.t

# OK, added just fine.  Let's commit it.
api_ref_data $ svn commit !$
svn commit aggtests/test/xml/xpath/pips3.t
Adding         aggtests/test/xml/xpath/pips3.t
svn: Commit failed (details follow):
svn: File '/svn/pips3/branches/api_ref_data/aggtests/test/xml/xpath/pips3.t' already exists
svn: Your commit message was left in a temporary file:
svn:    '/home/poec01/source/pips3/branches/api_ref_data/aggtests/test/xml/xpath/svn-com mit.tmp'

Yup. Subversion said it wasn't in my repository. I had to do a svn rm directly to the repository. That sucks.

I wasn't really expecting anyone to use it other than myself, but Paulo Custodio did, and he found bugs, and he sent patches, with tests. He suggested some other improvements too, and I handed over maintenance to him. Version 2 gone done grown up, with a real lexer/parser, proper #include files and everything.

I'm very grateful.

Bug? Feature? I report, you decide (and yes, I hate Fox News).

It didn't take me long to fix because I've seen this before, but I had to jog my memory. I wonder if PPI would catch this?

sub find_mapper_class_for {
    my ($self, $pip_obj) = @_;

    my %domain_to_mapper_map_for = map { "Pips3::Domain::$_" => $_ } qw(
        Brand
        Episode
        OnDemand
        Promotion
        Schedule
        Segment
        SegmentEvent
        Series
        Version
    );

    my $mapper = $domain_to_mapper_map_for{ ref $pip_obj } or X::Internal->throw(
        details     => "Unknown mapper (" . Dumper($pip_obj) . ")",
        stack_trace => longmess(),
    );

    return "Pips3::Persister::Mapper::DBIC::$mapper";
}

I have been tripped up by this a couple of times now and it can be a devil to figure out if you're not familiar with it. Heuristic parsing ain't fun.

More and more, I'm dealing with lots of mostly-JSON web service APIs. I enjoy this. It's simple, and tends to work well. I think JSON is a really nice format to work with. I had a discussion with Ingy and a bit of the #yaml gang about things I don't like about YAML (largely it's implicit types) and I ended up thinking that YAML wasn't quite as insane as I'd thought. Still, from the perspective of clarity, JSON blows it out of the water.

I want to add more formalized validation to our JSON APIs, so I was looking at ways to quickly write validation for data structures. I'd seen Kwalify and its Perl implementation out of the corner of my eye a number of times, and I looked into it more today. It started out looking quite reasonable, but then I started to find things that made me unhappy.

The first thing that should've made me wary was the fact that you can declare a pattern for a node, saying that the string must match a regex. That's not so crazy if you're writing a library for one language, but Kwalify seems like it's multilanguage. What regex library does it use? You can't specify it. Presumably it's the "native" library, but that means that your schemata are already not cross-language. That didn't jump out at me until later.

There's an assert, too, which is basically a string of code that gets templated and evaluated. This made me grumpy, but it's marked as "experimental," so I pretended that it would be ditched or replaced.

The mechanism for reusable schema parts is the YAML reference/anchor schema, which is equivalent to Perl references. In order to say that both "masters" and "servants" are the same kind of data, you might write a schema like this:

type: map
mapping:
  masters:
    type: seq
    sequence:
      - &person
        type: map
        mapping:
          name: { type: str }
          age:  { type: int }
  servants:
    type: seq
    sequence:
      - *person

...or something like that. I'd rather write something like:

schema: map/person
type: map
mapping:
  name: { type: str }
  age:  { type: int }
...

type: map
mapping:
  masters:
    type: seq
    sequence:
      - type: map/person
  servants:
    type: seq
    sequence:
      - type: map/person

Then I could have some better registry of types than just "references to other parts of one document." For example, the first document in the above section could actually be resolved by Some Unspecified Mechanism like "look up in filesystem." Without that, each schema is an island.

Kwalify also has a "data-binding" feature, which really means that it can generate a class from a schema and transform input into objects. That's okay, but I don't see how the fact that you might want to do that belongs in the schema. Doesn't it belong in some loader layer? Every time more features are added that do runtime things, the schema system becomes less cross-platform.

The two container types allowed in Kwalify are seq and map, which correspond to YAML's sequence and mapping. Unfortuantely, the sequence type can only define sequences of zero or more elements each matching the same constraint. In other words, you can say "this is an array of zero or more integers" but not "this is an array of between 2 and 8 integers" or "this is an array of an integer, then a string, then a map."

Validation systems always disappoint. So far my experience has been that everybody has his own particular fetishes for what he wants from a validator, and the more validation he has done, the more perverse his desires become. If you're really satisfied with your data validator (and you only use the one library for it) you're probably not validating enough.

Some time ago, I looked at BDFOY's Brick library. It looks great, but it has no non-developer releases, and the impression I got is that while it's really, really cool, no one actually uses it. I'm wary of being the first one to try to use it in production.

Beyond that, it's pretty clearly Perl. Of course, it would be possible to write something to compile simpler schema definitions into Bricks, so you could hand your compiler something like a Kwalify YAML file and get a Brick.

I've spent the last two hours or so trying to decide how I'd want to write this, with an eye toward being able to say things like, "and then you should have an array with one or more things matching the Pobox/Spam/Item schema." I'm also keen on being able to have things defined in terms of semantics that can be implemented in multiple languages, rather than being specified in regex. Maybe that just means you have to say "to be a valid Email/Address, you meet these criteria" and people are responsible for their own testing.

Of course, with a schema in a format like JSON, and a cross-platform testing system like FIT, this could be practicable. Further, it's not the worst thing in the world if parts of the schema can't be validated everywhere by everyone. It lets the client say "well, I tried to validate before sending, but I do not have all of the schema to be sure I am okay." This might not be acceptable for two-way transmission, but it's okay as long as the receiving end can validate enough to be happy.

I'm going to keep thinking about it while I clean the guinea pig cage.