rjbs's Journal

rjbs no longer found at use.perl.org journals

rjbs — 2008-12-02T22:16:47+00:00

It is very likely that I will no longer be posting to my use.perl.org journal.

I have always posted both to my Rubric at http://rjbs.manxome.org/rubric (or /journal) and to this journal. I posted here for visibility and comments.

I have added TypePad Connect to my Rubric, which will handle comments for me without needing to cross-post. If you are, for some reason, interested in my noise-making, please follow it at my Rubric.

Maybe this means I'll get back to adding features to Rubric!

the problem with console rpgs

rjbs — 2008-11-28T13:55:12+00:00

I'm playing Mass Effect now. It's pretty good, although I'm starting to find it a bit tedious, mostly because I keep having to repeat the same bits. This is especially frustrating when I have several dialog options that look very similar but end up coming out of my mouth with very different tones of voice.

That's a problem specific to Mass Effect, and it has some others. The thing that annoys me the most, though, is that it suffers particularly badly from a case of Stupid Game Economy. I wrote about the stupid Cyberpunk 2020 economy before, and it's a problem you can find all over the place if you look.

Mass Effect is a space opera. It has laser guns, super armor, space grenades, and holographic healing gauntlets. You start out with some of these, but they're bottom of the barrel and you have to acquire better ones as you play. You acquire these, sometimes, as spoils of war, but a lot of the really good stuff needs to be purchased. For example, some of the best weapons and armor I can get right now sell for six figures.

It not easy for me to acquire that kind of capital. It might be impossible without hours and hours more gameplay to even get close to it.

A minor upgrade to one of my weapons costs maybe 3000 credits. A license to preach on the main space station's promenade costs 150 credits.

So, let's say these are dollars. One hundred fifty dollars for a license from the government is not insane. Three thousand dollars for a pretty nice gun is not insane. One hundred thousand dollars for a really badass shield generator... well, let's say it's sane. Let's say all the prices make sense.

I ran a smuggling job for an alien. I used my status as a Spectre to bring experimental weapons into a tightly-controlled facility for him to resell. For my trouble, he gave me $250. I broke into an office complex and murdered something like twenty (corrupt) law enforcement officers. The guy who hired me to do that gave me about $700. What the hell is going on here?

Oh, so I mentioned I'm a Spectre? A Spectre is a super-elite ultra-secret agent who reports directly to the triumvirate that rules the galaxy. I've been sent to capture a rogue agent. They sent me as an alternative to sending an entire fleet. "Here," they said, "have a starship."

Not pictured, of course, is the conversation where I say, "Wow, thanks, that's a fantastic starship, and I know it's the best one in the whole human fleet. Do you think I could get some better kit for my team, too? Maybe one of those Ripper IV assault rifles and some shield generators?" The Council of Galactic Omnipotence then says, "Sorry, no. We know your mission is going to prevent the destruction of all life int he galaxy, but... we want you to work for it."

This is present in any game where the hero is known by all to be, you know, a hero saving the world from destruction, but he still has to pay for a glass of milk to heal.

"I need an ice hammer."

"Sure thing, Hero of Eternity. That'll be 350 gilootnis."

"Um... I need the hammer to get into the mountains to defeat the dragon that's been attacking your village."

"Yeah, I heard. Thanks!"

"So, really, can't you just give me the hammer? Or loan it to me? I'm trying to save you and, you know, your family. The Great Deity Thrumboat didn't give me a budget or anything when he imbued me with the Master Spirit so I could save you. I guess he figured people would be reasonable and want to contribute to their own salvation."

"Ho! That Thrumboat! Always figuring."

"..."

"Yeah, seriously though, tree fiddy."

awful itunes hack for album listening

rjbs — 2008-11-26T18:00:54+00:00

It's bugged me that iTunes makes it hard to listen to things as albums. Sure, it has shuffle-by-album, but smart playlists are all per-track.

After years of meaning to, this morning I wrote a (very very slow) Mac::Glue script to build a playlist of unrated or highly-rated albums that I haven't listened to lately. When I was nearly done and looked into one little bug, I found some other similar scripts. Oh well!

I'll eventually update this to avoid having it pick all albums by one artist, but for now, it's good. Thanks to it, I am re-listening to Method Man's Tical.

#!/usr/bin/perl use strict; use warnings; use Mac::Glue qw(:glue); use List::Util qw(sum); my $itunes = Mac::Glue->new('iTunes'); my $pl = $itunes->obj( playlist => whose(name => equals => 'Regular Music') )->get; my $albumen = $itunes->obj( playlist => whose(name => equals => 'Albumen') )->get; die "no albumen" unless $albumen; { my $tracks = $itunes->obj( 'track' => gAll, playlist => $albumen->prop('index')->get, ); for my $t ( $tracks->get ){ $t->delete; } } print "getting tracks\n"; my @tracks = $pl->obj('tracks')->get; my %album; while (my $track = shift @tracks) { my $trackid = $track->prop('database ID')->get; my $album = $track->prop('album')->get; my $artist = $track->prop('compilation')->get ? '-' : $track->prop('artist')->get; next unless defined $album and defined $artist; next unless length $album and length $artist; my $rec = $album{ $album, $artist } ||= []; printf "storing record of $trackid ($album/$artist); %s remain\n", scalar @tracks; push @$rec, { id => $trackid, rating => scalar $track->prop('rating')->get, played => scalar $track->prop('played date')->get, # epoch sec size => scalar $track->prop('size')->get, # in bytes }; } my $DEFAULT_TIME = time - 30 * 86_400; my %avg_age; ALBUM: for my $key (keys %album) { my ($album, $artist) = split $;, $key; printf "considering (%s/%s)\n", $album, $artist; my @tracks = @{ $album{ $key } }; unless (@tracks > 4) { printf "skipping (%s/%s); too few tracks\n", $album, $artist; delete $album{$key}; next ALBUM; } my @lp_dates = map { undef $_ if $_ eq 'msng'; $_ || $DEFAULT_TIME } map { $_->{played} } @tracks; my $avg_age = time - (sum(@lp_dates) / @lp_dates); $avg_age{ $key } = $avg_age; if ($avg_age < 86_400 * 30) { printf "skipping (%s/%s); too recent\n", $album, $artist; delete $album{$key}; next ALBUM; } my @ratings = grep { $_ > 0 } map { $_->{rating} } @tracks; my $avg_rating = sum(@ratings) / @ratings if @ratings; if ($avg_rating and $avg_rating < 60) { printf "skipping (%s/%s); too lousy\n", $album, $artist; delete $album{$key}; next ALBUM; } printf "keeping (%s/%s) @ %s\n", $album, $artist, $avg_rating || '(n/a)'; } my $total_size = 0; ADDITION: for my $key (sort { $avg_age{$b} <=> $avg_age{$a} } keys %album) { my @tracks = @{ $album{ $key } }; for my $track (@tracks) { $total_size += $track->{size}; my $t = $itunes->obj( track => whose('database id' => equals => $track->{id}) )->get; $itunes->duplicate($t, to => $albumen); } last ADDITION if $total_size > 500_000_000; }

first few xbox games

rjbs — 2008-11-24T15:04:29+00:00

My Xbox 360 came with LEGO Indiana Jones and Kung Fu Panda. I haven't tried Kung Fu Panda, but I finished the story mode of Indiana Jones. It was pretty good, although nearly the entire Last Crusade scenario was incredibly annoying. The LEGO games from Tt have a lot going for them, but they also have a lot of flaws that just don't seem to be getting fixed. They have awful cameras, terrible jumping predictability, horrible partner AI, and the vehicle chapters tend to kind of suck. Despite all that, I'd probably give it a B-. Like I said, the LEGO games have a lot going for them.

I borrowed Assassin's Creed from Bryan and played through a bit of it. It's pretty cool, but I'm not sure it's going to get its hooks into me. I'll give it another hour or so, and then call it "good but not addictive enough." I need to be picky about what games I play, since I don't have lots of gaming time to spend.

I rented Bioshock and Gears of War from Gamefly. Bioshock is pretty great. The setting is a lot of fun. If you aren't familiar with it, it's basically an Ayn-Rand-inspired community that has gone insane and self-destructed. The whole thing is very art deco looking, with slogans hung around the place like, "Altruism is the origin of all degeneracy."

The gameplay is also good, ranging between really innovative and frustratingly familiar. I supposed I should be ashamed of saying this, but I liked FPS better when you couldn't jump. Well, no, that's not true. I get frustrated when it's impossible to tell whether a surface is too high to jump or I just suck at jumping. Further, if I'm a big tough guy who can run and jump, why can't I climb? I mean, that wall is only four feet high. Do I really need to find a way around it? And why must there be constant footfalls even when there's nothing nearby? "Because it's spooky" isn't a good enough reason.

Anyway, that's mostly nitpicking. Since I set the game to Easy (so that I could defeat one of the nasty Big Daddies) I've had little reason to complain. It's really very good.

I was ready to give up on Gears of War after the first mission or so. It looked good and seemed interesting, but I kept getting slaughtered and I had trouble getting my head around how to play. Last night I gave it one last chance and ended up playing for two or three hours. I really got the hang of it, and it's fantastic. I've always liked it when shooters are more tactical than frantic, and Gears of War is very, very tactics-based. It throws in some frenzy, too. In the last bit I played last night before quitting, I was engaged in a shooting match with some aliens who had good cover, while also trying to fend off rushing little bastards who'd swarm me. I'd stand up to shoot the "wretches" only to have a "grub" take pot shots at me from behind cover.

It's very rewarding.

Next up, I'm waiting for Dead Space or Dead Rising. I played the Dead Rising demo and it seemed like it might be fun, but I'm not expecting to feel the need to buy it. I've heard mostly intense praise for Dead Space, so I'm hoping to get that next. I also have Bryan's copy of Mass Effect, and might try to give that a go this weekend if I haven't gotten a new game from Gamefly yet.

in which rjbs apologizes for being a dick

rjbs — 2008-11-17T15:36:43+00:00

A while ago, I did a bit of work applying patches to MIME::Lite. It's the most widely depended upon of the dists I maintain, and one of the oldest, with the most RT tickets. I also don't really like it very much, so it doesn't get as much work as I'd like to give it.

Anyway, I recently applied a patch from one of those tickets, which was marked as being the patch that Debian used. In the changelog, I put a snarky remark about Debian maintainers being jerks and not sending me (the upstream maintainer) their patch.

It was then pointed out to me that they had, in fact, sent me the patch, that it was in a ticket, and that I had marked it resolved.

So, hey, Debian guys who might be reading this and whom I didn't already apologize to: I'm sorry. What was I thinking? Well, clearly I wasn't.

After giving people a hard time about giving people a hard time, I should really know better. I will endeavor to be better about sticking to my own rule: don't be snarky.

i have an xbox 360 now

rjbs — 2008-11-16T05:34:08+00:00

I noticed, recently, that I had enough Amex rewards points to get an Xbox 360. I thought about it for a while and then decided to go for it. As is my habit when ordering something, I obsessively checked the status of my order every few hours, and it kept sitting at "submitted." I figured it could take quite a while, since I was buying with points.

Today, though, the Xbox showed up via FedEx around two o'clock. I had it set up before I left for my evening out, but it wasn't online. It turns out that the Xbox 360 doesn't have a built-in wifi connection. You can buy a USB wifi adapter for it, but the only one that seems to still be made that works is Microsoft's, and they want $100 for it. That is completely insane. I bought a USB adapter for my TiVo and it cost $20.

I decided to use my old Airport Extreme as an extension for my Time Capsule's network, but this ended up being a big headache. I had to use WDS, because my Airport predates the simpler "extend wireless network" option. Then I struggled with making WDS work, until I finally found that I had to set both devices to use the same channel. I thought this would be a bad idea, as there would be some kind of conflict. Goes to show what I know, I guess.

I got my Xbox Live account ported over from my old Xbox days, hooked up my Xbox Live account to my Live Messenger account, played the first level of LEGO Indiana Jones (meh) and that's that. I'll probably try renting or borrowing some games over the next few weeks. Given that the thing cost me nothing, I'm in no hurry to recoup the expense.

publishing parts of my tiddlywiki

rjbs — 2008-11-15T04:19:53+00:00

Some time ago I wrote that I had moved my D&D wiki to TiddlyWiki. This has worked pretty well, although I've mostly given up storing YAML in my TiddlyWiki -- mostly because I didn't end up using the tools that used it all that much. Maybe next time.

Anyway, I'm getting close to starting my next game, and I've been doing much more work on the wiki for that one, and I have been annoyed at all the copying and pasting I've been doing. I want to give some of the content to the players but keep most of it private.

I looked at using two TiddlyWikis, but of course I'd want a single one to edit. I thought maybe I could have the second be for the players, and I'd sync it from mine. It was going to be a pain, though, to edit the list of synced pages. I really wanted to say, "sync everything tagged Public."

Then I realized that I didn't want this, either. I want to be able to put secret data on my wiki pages, and to easily give the page to my players -- sans the internal notes.

My solution is an ugly hack that I think will work just fine. I've set up a shared folder on Dropbox where my players will save their notes, maps, and so on. I made a folder in that share where I'll put articles about house rules, mechanics, and so on. It's all stuff from my wiki, published with a script that iterates over my TiddlyWiki finding and reformatting pages with the Public tag. It strips out private notes, replaces transclusion with cross reference, and does some other stuff.

I thought I'd be able to publish HTML using some CPAN module, but the only TiddlyWiki formatter on the CPAN seems to be vaporware. In the end, I decoded that wiki markup is easy enough for the players to read. I think this will work really well.

Here is the hacky script I'm using:

use strict; use warnings; use 5.010; use HTML::TreeBuilder; use Text::Autoformat; use Text::Balanced qw(gen_extract_tagged); my $extractor = gen_extract_tagged(map quotemeta, qw( [[ ]] )); my $tree = HTML::TreeBuilder->new->parse_file($ARGV[0]); my @tiddlers = grep { ($_->attr('tags') || '') =~ /\bPublic\b/ } $tree->look_down(_tag => 'div'); sub eq_pad { my ($str) = @_; my $total = 73 - length $str; return "$str " . ('=' x $total); } sub filename { my ($title) = @_; $title =~ s/\W+/-/g; return lc "$title.txt"; } for my $tiddler (@tiddlers) { my $title = $tiddler->attr('title'); my $fn = filename($title); open my $fh, '>', $fn or die "can't open $fn to write: $!"; my $tag_str = $tiddler->attr('tags') || ''; my @tags; while (length $tag_str) { my $tag; ($tag, $tag_str) = $extractor->($tag_str); if ($tag) { push @tags, $tag; next; } else { push @tags, split /\s+/, $tag_str; last; } } my $mod_date = $tiddler->attr('modified') || ''; my (@date) = $mod_date =~ /^(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})$/; say $fh 'Title : ', $title; say $fh 'Tags : ', join ', ', sort @tags; say $fh join ' ', 'Modified:', ($mod_date ? (join('-', @date[0,1,2]), join(':', @date[3,4])) : '??'), 'by', $tiddler->attr('modifier') || '?'; say $fh ''; my $text = $tiddler->as_text; $text =~ s{<part \w+>\n}{}g; $text =~ s{</part>\n?}{}g; $text =~ s/^!!(.+)$/"\n\n== " . eq_pad($1) . "\n"/meg; my @chunks = split /\n{2,}/, $text; my @xref; for my $chunk (@chunks) { last if $chunk =~ /^----/; next if $chunk =~ /@@/; $chunk =~ s/\[\[([^\]|]+)(?:\|[^\]]+)?\]\]/$1/g; if ($chunk =~ /<<tiddler Template:Summary with: ([\s\w]+)>>/) { push @xref, $1; next; } if ($chunk =~ /^== /) { $chunk .= "\n\n"; } else { $chunk = autoformat $chunk } print $fh $chunk; } say $fh "SEE ALSO: $_" for @xref; }

astounding optimization! thanks nytprof!

rjbs — 2008-11-12T02:33:58+00:00

We've been unhappy with the performance of some code, recently. I was pretty sure I knew where the problem was, but I thought I'd run NYTProf just to see how things looked. I'm running an older NYTprof, so it's not 100% clear that my SQL-level optimization is what I need to do -- but it's the right thing to do anyway. Anyway, I figured I might see something sort of interesting, but I never expected this:

Calls InclTime ExclTime Subroutine 27,908 406 406 DBI::st::execute 543,412 79 31 Carp::caller_info

...and let's not go any further. The program took almost exactly 600s to run. Of that, nearly five percent was because the program called Carp, and it called it a half million times! What?!

I won't be coy, because I'm writing this while waiting for a test suite to run and while watching House. It turns out that it was related to this line:

Calls InclTime ExclTime Subroutine 139,340 222 6 SUPER::get_all_parents

That subroutine looks like this:

sub get_all_parents { my ($invocant, $class) = @_; my @parents = eval { $invocant->__get_parents() }; unless ( @parents ) { no strict 'refs'; @parents = @{ $class . '::ISA' }; } return 'UNIVERSAL' unless @parents; return @parents, map { get_all_parents( $_, $_ ) } @parents; }

See how it calls $invocant->__get_parents? Well, that's great, except that our internal ORM has an AUTOLOAD subroutine that looks like this:

sub AUTOLOAD { my $self = $_[0]; my $class = (ref $self) || $self; (my $method = $AUTOLOAD) =~ s/.*:://; return if $method eq "DESTROY"; unless (blessed($self)) { confess qq(AUTOLOAD: \$self for ->$method is not a blessed reference: ) . Dumper($self); } ... }

Now, to avoid hitting the database too much, we have a mixin that makes it talk to a memcached. That mixin (like many such modules) uses SUPER.pm. SUPER then calls __get_parents on our ORM, but hits the AUTOLOAD instead, and since that's only supposed to work on objects, it confesses. It does this every time we check consider using a cached copy of an object, causing us to invoke Carp::confess a half million times.

The solution? I added this line to our ORM:

sub __get_parents { return; }

Shaved two minutes off the test case. That's about 20%.

more more cpan metrics

rjbs — 2008-11-04T19:06:27+00:00

As suggested, I have run the code such that a dist's mere appearance on the CPAN is not counted. In other words: code exists if it is used. If not, it is ignored entirely. It ends up not having much effect.

author | volume | req (old) | req (new) ZOFFIX | 1 | 145 | (n/a) ADAMK | 2 | 32 | 34 RJBS | 3 | 43 | 43 MIYAGAWA | 4 | 82 | 84 NUFFIN | 5 | 85 | 91 GBARR | 114 | 1 | 1 PMQS | 221 | 2 | 2 PETDANCE | 31 | 3 | 3 MSCHWERN | 40 | 4 | 4 SAPER | 32 | 5 | 5

JROCKWAY suggested running this in reverse: see who uses the most. I think that doing both would be interesting: who gets used a lot, "despite" also using a lot of prereqs.

more cpan metrics

rjbs — 2008-11-04T15:47:44+00:00

For a while, I've been keeping track of the total usage of my code on the CPAN. It helps me see what people have found useful, and lets me decide how scared to be of introducing back-incompat changes. Sometimes people talk about the sort of catastrophe that can occur if a highly-required module is broken. For example, over 11,000 dists require the code in Getopt-Long. If it broke badly and people installed the new code, it would be a nightmare.

So, I applied my "who needs me?" script to the whole CPAN. It produces a list of every author with dists, showing how many other dists (recursively) use that dist. When an author uses his own dist, it is not counted as a prerequisite. The "total cpan-breaking power" score is more accurate that way.

This scoreboard is quite different to Simon Wistow's CPAN Leaderboard. Here's a comparison:

author | volume | requiredness ZOFFIX | 1 | 145 ADAMK | 2 | 32 RJBS | 3 | 43 MIYAGAWA | 4 | 82 NUFFIN | 5 | 85 GBARR | 114 | 1 PMQS | 221 | 2 PETDANCE | 31 | 3 MSCHWERN | 40 | 4 SAPER | 32 | 5

The program is included below. It's a quick and dirty hack, but it was fun to write and look at.

#!/usr/bin/perl use strict; use warnings; use 5.010; use DBI; use JSON::XS; my $dbh = DBI->connect('dbi:SQLite:dbname=cpants_all.db', undef, undef); my $authors = $dbh->selectall_arrayref( "SELECT id, pauseid FROM author WHERE pauseid IS NOT NULL ORDER BY pauseid" ); my @results; for my $author (@$authors) { my ($author_id, $pauseid) = @$author; my $dists = $dbh->selectall_arrayref( "SELECT id, dist FROM dist WHERE author = ? ORDER BY dist", undef, $author_id, ); my %analysis; analyze_dist(\%analysis, $author_id, $_) for @$dists; my $sum = 0; $sum += $_ for values %analysis; next unless $sum; warn "$pauseid,$sum\n"; push @results, { pauseid => $pauseid, result => $sum, dists => \%analysis, }; } my $JSON = JSON::XS->new; for my $author (sort { $b->{result} <=> $a->{result} } @results) { say $JSON->encode($author); } sub analyze_dist { my ($analysis, $author_id, $dist, $seen, $add_to) = @_; $seen ||= {}; $add_to ||= $dist->[1]; my @queue = $dist; $analysis->{ $add_to }++; my $needed_by = $dbh->selectall_arrayref( "SELECT p.dist, d.dist AS name FROM prereq p JOIN dist d ON d.id = p.dist WHERE p.in_dist = ? AND author <> ?", undef, $dist->[0], $author_id ); for my $needed (@$needed_by) { next if $seen->{ $needed->[1] }; $seen->{ $dist->[1] }++; analyze_dist($analysis, $author_id, $needed, $seen, $add_to); } }

You can find the results of the results as of last night in my drop box.

october horror recap

rjbs — 2008-11-04T03:42:21+00:00

As is our tradition, Gloria and I watched a bunch of horror movies this October. Here's a quick recap:

Saw II-IV were a good watch... well, at least the first two. We saw Saw a few years ago and both agreed it was mediocre. Saw II was better, though, and Saw III was actually a really good capstone to the trilogy. For some reason, though, they decided to keep going. Saw IV was really lousy. I expect that V and VI will be worse, but I'm in no rush to find out.

Opera, like Suspiria, was an incomprehensible and joyless horror film by Dario Argento. I suggest you avoid it.

Chopping Mall was a good old fashioned 80's horror film by Roger Corman, King of the Bs. It had everything you need: teenagers having sex, robots getting struck by lightning, and an exploding head. Would buy again.

Child's Play I-III were a disappointment. Each one of the movies had a few amusing bits, but they weren't very funny, weren't very scary, and weren't even all that memorable. I'm looking forward to Bride and Seed of Chucky, though, as they seem like they'll be a lot of fun. I'm just surprised that they got made. The first three were just lousy.

Dark Water was a movie about a girl who drowns and then terrorizes the living, based on a book by Koji Suzuki. In other words, it sounds a lot like Ring. It had a lot of cool ideas, but basically it stank. It wasn't very compelling, and the climax made nearly no sense. On the plus side, Dark Water, unlike Ringu, is unlikely to haunt my nights for years to come.

Prom Night was easily the worst of the movies we watched. Its plot was nearly incomprehensible, the construction of the narrative was confusing at best, the writing was awful, Leslie Nielsen and Jamie Lee Curtis were both totaly wasted, and the "payoff" was pointless. How on earth did that get to be a classic?

Mr. Ice Cream Man was mostly a terrible movie, not even on the B list, but the guy who played the ice cream man was creepy enough to get it above Prom Night. Still, what the hell was up with that last scene? It felt completely tacked on.

Basket Case 2 was, in a lot of ways, better than the first. Both were hard to really rate. They're strange movies. The second one was much more over the top, and I think it was better for it. There's a scene in which we meet a freak who is mostly a gigantic head, mostly mouth, who sings opera. Awesome!

Black Sheep did not impress. I'm not a big fan of the New Zealand school of horror-comedy. It had a few good gags, but mostly it didn't entertain me. I think it's the only movie that Gloria gave up on and went to bed during. I don't blame her.

Silent Night, Deadly Night was the movie on which we were the most conflicted. I thought it was pretty good, for what it was. Gloria thought that a guy dressed as Santa killing people was just Too Much. Despite that, it's now probably the most quoted of all the movies we watched this past month. "Punish!" and "Naughty!" just make great lines to yell out randomly.

We might watch one or two more movies that were left over, but I then we may be back to our normal mix of programming until next October. I wonder when we can start Martha on some horror flicks...

hogar crea flan

rjbs — 2008-11-02T00:54:11+00:00

There is a fairly well-respected charity that's fairly active in Bethlehem. It's called Hogar Crea. I'm sure they do lots of good stuff and help people. That's the impression I've gotten from various people. That's not what matters to me.

What matters to me is that sometimes they come to my door and in exchange for just a few dollars they give me a delicious flan. God bless you, Hogar Crea.

what the heck is distzilla?

rjbs — 2008-10-27T15:58:00+00:00

At the Pittsburgh Perl Workshop this year, I gave a lightning talk about Dist::Zilla, the system I am increasingly using to manage my CPAN distributions. I'm using it instead of writing a Makefile.PL, but it doesn't do the same thing as Module::Build or ExtUtils::MakeMaker. I'm using it instead of running module-starter, but it doesn't do the same things as Module::Starter. I've had some people say, "So should I stop using X and use Dist::Zilla instead?" The answer is complicated.

(Well, actually, for now the answer is simple: probably not. Dist::Zilla is a lot of fun and I really, really appreciate the amount of work it saves me, but it's really young, underbaked, and probably full of bugs that I haven't noticed yet. Still, the adventurous may enjoy it.)

The idea behind Dist::Zilla is that once you've configured it, all you need to do to build well-packaged CPAN distributions is write code and documentation. If you're thinking, "but that's what I've been doing anyway!" then first consider this: If you are writing =head1 NAME\n\nMyModule - awesome module by me then you are not just writing code and documentation. If you are adding a license to every file, again, you are not just writing code and documentation.

If you use, say, Module::Starter to get all this written for you, then you're safe from writing that boilerplate stuff. Unfortunately, if you need to change the license, or you want to add a 'BUGTRACKER' section to every module, Module::Starter can't help you. It creates a bunch of files and then its job is done. It never, ever looks at your module-started distribution and fixes up things. This also means that if you realize that your templates have failed to include use strict for your last three module-started distributions, you have to fix them by hand. The same goes for the stock templates, which until recently didn't include a license declaration in the Makefile.PL.

With Dist::Zilla this content is not created at startup. It is not stored in your repository. Instead, the files in your repo are just the code, documentation, and the Dist::Zilla configuration. When you run dzil build, your files are rebuilt every time, adding all the boilerplate content from your current setup. If you want to change the license everywhere, you change one line. If you want to start adding a VERSION header, you tweak the Pod::Weaver plugin's configuration.

So, there does exist a dzil new command for starting a new distribution. All it really does, though, is make a directory (maybe) and add a stock configuration file. Why would it add anything else? If you want any code, you would only be writing the actual code needed, not any boilerplate, so adding anything would be foolish.

There's also dzil release, which goes beyond what Module::Starter (and its competitors) do and into the realm of ShipIt or Module::Release. I'm hoping I can integrate with or steal from one of those sort of tools. Right now, it exists, but all it does is build a dist and upload it. In the future, it will have at least two more kinds of plugins to make the release phase more useful: VCS (so it can check in and tag releases) and changelog management. It has a changelog thing now, but it stinks and isn't very useful. In the future, you won't need to edit a changelog. It will be able to read changes out of your commits, or you will just tell it to record a changelog entry. Then the Changes file can be generated as needed.

For now, I am manually editing my Changes file.

So, eventually Dist::Zilla could obsolete Module::Starter for people who like what Dist::Zilla does. Then again, people might still want to have starter templates that add minimal boilerplate for using certain frameworks. We'll see what happens.

pod people versus elementalists

rjbs — 2008-10-25T18:08:05+00:00

A long time ago, I wanted to write something to let my pod (documentation) contain its own coverage hints. I gave up when I found out that it was not going to be trivial to say something like this:

my @blocks = PodParser->read_file($my_perl_module)->data_for('coverage');

In order to extract "foo\nbar" from:

sub foo { ... } =begin coverage foo bar =end coverage

I found ways, but they all bugged me. I gave up on the project for a long time, because it was a real yak, but eventually I came back to it when I realized how much pod manipulation I'd want in Dist::Zilla. I really wasn't happy with how Pod::Simple worked. Dieter had contributed a bit to Pod::Simple, and had talked about writing a more TreeBuilder-like interface. There were a number of significant blockers, though, and I didn't want to get hung up on them. Instead, while walking to McGrady's for ABE.pm, I had an idea and called Dieter to brainstorm with him. Basically, the idea can now be summarized as "I should write Pod::Eventual."

Pod is really great. It's so easy to write that I know I write much, much more documentation that I would if I had to produce, say, a chm file. It's a very, very simple format, and is complex enough to handle almost everything I've ever needed from it. My problems have been that I want to write even less and have it rewritten for me, so I can avoid boilerplate.

The root problem is that pod has both very simple and very complex parts. Here are some of the simple things:

a pod document is made up of paragraphs
paragraphs are separated by blank lines (but 'cut' commands are special)
pod can be interwoven with non-pod in a document
pod paragraphs are either:
- commands (start with =)
- verbatim (start with whitespace)
- text (start with anything else)
the non-whitespace characters after the = in a command are the command

So, knowing this is about enough to write a tolerable pod paragraph parser for most uses. Sure, it misses a lot of encoding stuff, but adding that later (I believe) is not a big issue.

It omits two very, very big things. First of all, it ignores the content of text paragraphs. That means that I've said nothing about what F<markup> means. This is a big obnoxious problem, and I have absolutely zero interest in tackling it. Hooray for punting, right?

The second problem is that it assumes that all pod documents are sequences of paragraphs. In fact, this is true. The problem is that on top of the syntax of paragraphs, there are paragraph semantics that make this, for example, an illegal document:

=pod =item * Isn't this simple? =end

We have an =item outside of an =over and an =end outside of a =begin. Wait... outside? If a pod document is just a sequence of paragraphs, how does containment work?

Well, it doesn't.

It is fairly obvious that the begin and over commands set up containment. They have start commands and end commands, and anything between the two is contained "inside" the block. Unfortunately, there's a lot of implicit containment in how many pod formatters relate the document to the reader. For example, look at how the Sub::Exporter Cookbook is presented. head2 items are presented, in table of contents, as being contained by the head1 items. You'd also like to think that the text and verbatim paragraphs that occur between two head1 paragraphs are contained by the first. Unfortunately, that isn't how it works, and it isn't really clear how it should work. What items cause the end of a container? What items can contain themselves?

Again, I originally punted. Pod::Eventual just produces the sequence of events. For the things I wanted to do, however, I needed structure. I wanted to be able to make a head1 thing and put head2 or other things inside of it. (Actually, in Pod::Weaver, the technical term for these is "thingers".)

Dieter had long since abandoned his work on pod stuff, so I stole (with his blessing) the name for my pod event-to-tree transformer: Pod::Elemental. It reads in a document that contains pod and returns a sequence of roots of trees that represent the document's pod. The logic by which they're formed into a structure is contained in the Nester, and anyone can write his own nester to use whatever nesting logic he thinks makes the most sense.

Pod::Weaver uses Pod::Elemental to turn a Perl document (using PPI) into a just-Perl document and a collection just-Pod elements. The elements are then reorganized and rewritten, in part by looking at the Perl and in part by using plugins and provided input. Dist::Zilla uses Pod::Weaver to add a name-and-abstract section, a license section, to build methods and attributes sections, and to do other stuff like that. It works very well, assuming you don't mind minor explosions while I rejigger the API every other day.

Right now, I know that I have ignored a lot of what is demanded by perlpodspec. Frankly, I intend to keep ignoring a bunch of it. My goal is to let people work with pod paragraph syntax without worrying about the syntax of paragraph content or of the semantics of paragraph ordering -- until they want to. The default Pod::Elemental::Nester, for example, will barf if you try to give it an =end outside matching =begin. Pod::Eventual, however, doesn't care.

Pod::Elemental doesn't care about other things, though, like the magic attached to =begin (data) blocks whose identifiers begin with a colon. Why? It's just about slinging around paragraphs, not around understanding meaning.

I'm definitely planning on adding quite a bit more standards-compliance to Elemental. For one thing, I want to get =encoding hashed out and improve the interface for the element tree. Even Eventual needs some help. For example, I think it gets the definition of a blank line (which divides paragraphs) wrong, and I'd like to change how it understands the lines between =cut and a blank.

Still, though, I'm very happy with what I have and how simply I got it. I definitely would not recomment writing a pod-to-text converter using any of this code, but for writing a pod preprocessor, I've found it really great.

overheard in #email

rjbs — 2008-10-20T21:40:27+00:00

Programming email can leave you a bit ... touched. Here are some recent gems from #email on irc.perl.org

<dave0> To bounce, or not to bounce <dave0> that is the question <dave0> whether 'tis nobler in the mind to suffer <dave0> the slings and arrows of outrageous NDRs <dave0> or by opposing, end them

It's a poetry contest he wants, is it?

<rjbs> what a piece of work is mime <rjbs> how nested in structure <rjbs> how infinite in quirkiness <rjbs> in form and encoding how completely execrable <rjbs> the paragon of feature creep <rjbs> and yet to me <rjbs> what is the quintessence of wtf

Have we gone too far? No.

<confound> well, writing a poem about mime is pretty wtf <dave0> I think a poem about MIME would need to be something epic and full of angry, capricious Norse gods <rjbs> multipart/epicpoem <rjbs> Das MIMErdämmerungenlied <rjbs> stanza*1*=latin-1'q'no_no'Das=20MIMErd=E4mmerungenlied <dave0> ; format=opera

Trust us. It's hilarious.

coping with solaris cron

rjbs — 2008-10-18T16:25:02+00:00

More and more, we're eliminating Linux boxes in favor of Solaris. This is generally not a huge deal, but one of the niggling details has been Sun's cron. It sucks. It sucks because it uses a constant as the subject of its alert messages. If you have a lot of servers running a lot of cron jobs, generating a lot of output, you end up with a display that looks like this:

1 N Oct17 Super-User ( 7) Output from "cron" command 2 N Oct17 Super-User ( 7) Output from "cron" command 3 N Oct17 Super-User ( 7) Output from "cron" command 4 N Oct17 Super-User ( 7) Output from "cron" command 5 N Oct18 Super-User ( 7) Output from "cron" command 6 N Oct18 Super-User ( 7) Output from "cron" command 7 N Oct18 Super-User ( 7) Output from "cron" command 8 N Oct18 Super-User ( 7) Output from "cron" command 9 N Oct18 Super-User ( 7) Output from "cron" command 10 N Oct18 Super-User ( 7) Output from "cron" command 11 N Oct18 Super-User ( 7) Output from "cron" command

Seriously?

So, I put in a change request to have this fixed. Deploying Vixie cron was going to be a massive pain (I was told) so instead we updated our use of puppet to ensure that our cronjobs were run by a wrapper script. I'm really happy with it, as it eliminates a few other crappy wrapper scripts and gets me what I wanted to begin with. There are a few internal modules used below, but it should be trivial to replace them with whatever you want. (I've removed a few constants, too.)

Maybe I'll CPANize this later.

#!/usr/bin/perl use strict; use warnings; use Digest::MD5 qw(md5_hex); use Fcntl qw(:flock); use Getopt::Long::Descriptive; use ICG::SvcLogger; use IPC::Run3 qw(run3); use String::Flogger qw(flog); use Sys::Hostname::Long; use Text::Template; use Time::HiRes (); my ($opt, $usage) = describe_options( '%c %o', [ 'command|c=s', 'command to run (passed to ``)', { required => 1 } ], [ 'subject|s=s', 'subject of mail to send (defaults to command)' ], [ 'rcpt|r=s@', 'recipient of mail; may be given many times', ], [ 'errors-only|E', 'do not mail if exit code 0, even with output', ], [ 'sender|f=s', 'sender for message', ], [ 'jobname|j=s', 'job name; used for locking, if given' ], [ 'lock!', 'lock this job (default: lock; --no-lock to not)', { default => 1 } ], ); die "illegal job name: $opt->{jobname}\n" if $opt->{jobname} and $opt->{jobname} !~ m{\A[-a-z0-9]+\z}; my $rcpts = $opt->{rcpt} || [ split /\s*,\s*/, ($ENV{MAILTO} ? $ENV{MAILTO} : '...') ]; my $host = hostname_long; my $sender = $opt->{sender} || sprintf '%s@%s', ($ENV{USER}||'cron'), $host; my $subject = $opt->{subject} || $opt->{command}; $subject =~ s{\A/\S+/([^/]+)(\s|$)}{$1$2} if $subject eq $opt->{command}; my $logger = ICG::SvcLogger->new({ program_name => 'cronjob', facility => 'cron', }); my $lockfile = sprintf '.../cronjob.%s', $opt->{jobname} || md5_hex($subject); goto LOCKED if ! $opt->{lock}; open my $lock_fh, '>', $lockfile or die "couldn't open lockfile $lockfile: $!"; flock $lock_fh, LOCK_EX | LOCK_NB or die "couldn't lock lockfile $lockfile"; printf $lock_fh "running %s\nstarted at %s\n", $opt->{command}, scalar localtime $^T; LOCKED: $logger->log([ 'trying to run %s', $opt->{command} ]); my $start = Time::HiRes::time; my $output; $logger->log_fatal([ 'run3 failed to run command: %s', $@ ]) unless eval { run3($opt->{command}, \undef, \$output, \$output); 1; }; my %waitpid = ( status => $?, exit => $? >> 8, signal => $? & 127, core => $? & 128, ); my $end = Time::HiRes::time; unlink $lockfile if -e $lockfile; my $send_mail = ($waitpid{status} != 0) || (length $output && ! $opt->{errors_only}); if ($send_mail) { require Email::Simple; require Email::Simple::Creator; require ICG::Sendmail; require Text::Template; my $template = do { local $/; <DATA> }; my $body = Text::Template->fill_this_in( $template, HASH => { command => \$opt->{command}, output => \$output, time => \(sprintf '%0.4f', $end - $start), waitpid => \%waitpid, }, ); my $subject = sprintf '%s%s', $waitpid{status} ? 'FAIL: ' : '', $subject; my $email = Email::Simple->create( body => $body, header => [ To => join(', ', @$rcpts), From => qq{"cron/$host" <$sender>}, Subject => $subject, ], ); ICG::Sendmail->sendmail( $email, { to => $rcpts, from => $sender, archive => undef, } ); } __DATA__ Command: { $command } Time : { $time }s Status : { join('', flog('%s', \%waitpid)) } Output : { $output || '(no output)' }

american express sends me mixed messages

rjbs — 2008-10-18T02:06:09+00:00

In general, I am a very, very happy cardholder. Just recently, when my EVDO modem died, American Express paid for me to replace it with nearly no questions asked. That saved me about $250, since the modem had just gone out of warranty. That pays for over half my annual membership. They also paid for some MacBook repairs earlier this year, which was a real plus.

Then again, I just got a letter that the "domestic companion airfare program" has been discontinued. This was the program that said that when flying within the country, I could get a free ticket when purchasing one at their normal prices. When going to YAPC in Chicago, this saved us about $400. It's really the reason I decided to upgrade to a platinum card.

Now, instead of being able to travel for less, I have to hope for my electronics to fail to recoup the cost of membership.

It's annoying to lose such a great program, but... I still really like my American Express.

annoying things learned about perl today

rjbs — 2008-10-17T02:19:53+00:00

App::Cmd::Tester lets you test that an App::Cmd program output the right things to standard error and standard out, and did so in the right order. It does stuff Test::Output can do, but also just a bit more.

I had a need to generalize this earlier today, and ran into a bunch of obnoxious problems. Most of these center around the "it's hard to get synchronized but separate output from a spawned program's stderr and stdout" problem. Others were less well-known, at least to me.

For example, I was doing something like this:

tie my $scalar, 'Tie::Class', { common => \$string, private => \$other };

I had a number of things like this, and I thought I'd be able to write:

tie my $scalar, $object->tie_args;

This didn't work, though, because the tie_args method gets called in scalar context, so it evaluated to the hash reference, which of course had no TIESCALAR method.

Instead, I ended up doing something possibly better anyway:

tie my $scalar, $object, $argument;

The object has a TIESCALAR method that proxies to the correct class with the correct arguments. I'm pretty happy with that.

The other problem is this:

tie my $scalar, 'Some::Tie'; open my $fh, '>', \$scalar or die "failed to open ref: $!"; print $fh "this is a test" or die "failed to print: $!;

This code just doesn't do what I mean. It doesn't raise any exceptions, but the STORE method is never called on the tie. This seems, to me, like a bug in perl. I'll need to investigate.

The code I was working on has been uploaded as IO::TieCombine.

lies, lies, lies (about curious george)

rjbs — 2008-10-16T13:01:06+00:00

From Zap2It's "premise of 'Curious George'":

Premise of Curious George
Curious George is a sweet African monkey who can't help but run into trouble. Mr. Renkins, who George calls 'The Man in the Yellow Hat' tries very hard to care for George and is always saving the day. The show's themes are about learning, forgiveness and playful curiosity.

George is not a monkey. He does not have a tail. That, however, is a mistake made long ago by the original translator of Curious George.

The more appaling mistake is identifying The Man with Mr. Renkins. Mister Renkins is The Man's neighbor at his country house. He is a farmer, and is married to Mrs. Renkins. There is no reason to believe, as far as I know, that there is any relationship between the Renkinses and The Man.

The recent movie version of Curious George assigned a name to The Man, but I will not repeat it here, as I find the very idea offensive, and do not wish for anyone else to be burdened by knowing what some moron thinks George's Friend's name is.

Time for Curious George. He's going to make a lemonade stand and fix some traffic lights.

another unproductive complain about subversion

rjbs — 2008-10-15T02:18:33+00:00

I remember in 2005 or so when I first started using Subversion, I liked it so much. It was much easier to use than CVS. Everyone said it would be make tagging and branching easier than CVS. In CVS, tagging was fine, but branching was such a pain that I never bothered.

Eventually, I found out that branching and merging were much easier, but still a real pain. Tagging, though, was completely insane. Tags were implemented as copies (just like branches). This sort of made sense as a cheap way for branches to work, but none for tags. Tags are labels for points in time in a repository. They shouldn't be mutable, unless maybe to let you remove a label from rev 1 and put it on rev 2.

Because they're implemented as copies, you can actually go in and alter the state of a tag, meaning that tags are useless as ... tags. It also means that if you have a standard Subversion repository with trunk, branches, and tags directories, and you check out the whole thing, you check out absolutely every file in every revision. "Copies are cheap" was a big Subversion mantra back in the day, because in the repository, only files that changed were new files on disk -- but that only goes for what's in the repository, not your checkout. In your checkout, every copy of readme.txt is its own file -- and it has to be, because even the tags are mutable. You can't say that ./tags/1.000/readme.txt is the same file as ./tags/2.000/readme.txt just because there was no change between the two releases, because you could go change either of them, and if you do, you'd change both. Oops!

This came up today because of a piece of automated deployment code that did something like this:

$ mkdir TEMP $ chdir TEMP $ svn co $REPO/project $ cd project/trunk $ bump-perl-version $ cd .. $ svn cp trunk tags/$NEW_VERSION $ svn ci -m "bump and tag $NEW_VERSION"

Checking out requires a whole lot of space, because it has to check out every single tag's copy of every file. Tagging the new release is also fairly space hungry. How hungry? Well... the project I'm working on right now is a web application. Let's call it New-Webapp.

If I export a copy of trunk from Subversion, getting just the files that make up the latest version of the application, it's 1.9 megabytes.

If I check a copy of the trunk out, so now there's all the extra working copy files, and it's 5.2 megabytes.

If I check out the whole repo, getting every tag and branch (for your information, there is exactly one branch), it's 207 megabytes.

Now, keep in mind that this gives me every file from every tagged release (there are 40 releases). This does not give me the entire revision history. There are many, many revisions missing. After all, what I have is basically 42 revisions: 40 releases, trunk, and one branch. That's it.

If I use git-svn to build a git repository of the project, meaning that I have absolutely every revision, every tag, and every branch, it's 249 megabytes. That's all 1149 revisions.

I am so ready to be done with Subversion.

our text editors, our secret masters

rjbs — 2008-10-08T12:33:34+00:00

Ever wonder how much of our programming style is dictated by our desire to see the right pretty colors? In Perl, I think it's a good bit.

For example, I know that the person who wrote this line wasn't using Vim's default Perl syntax:

Account->q(accountid => $self->{accountid});

...because it would interpret q(...) as a non-interpolative string. Meanwhile, the guy who wrote this was:

$logger->("setting account information to \%info");

...because syntax highlighting told him that %info was a variable, even though Perl doesn't interpolate hashes.

I always put spaces around my range operators, because otherwise in 1..2 the dots are different colors.

Other examples welcome.

hearts were entertaining june

rjbs — 2008-10-08T01:13:41+00:00

I feel like I'm living in Brazil. No, not the nation, the film.

Two lamps in my house have failed, and both are so old and poorly wired that repairing them has been a real pain (and they're both still broken as I type this). My EVDO modem has been really flaky for the past three or four weeks and it's been a crap shoot, each day, as to whether I could get online.

Today I did a lot of uninstall/reinstall of the modem drivers, but it didn't help. I've been suspecting that it's a problem with the hardware. A few minutes ago, trying to remove and reseat the modem's USB connector, I pulled the device gently from my laptop, only to have the modem come out but not the connector. Has it been failing due to some kind of physical defect? Maybe, but now I'll never know. I doubt my vendor is going to accept a return on something that looks like it was physically mistreated.

So, I called work to say I was going to have to order a replacement and then started to bring up 3GStore on my iPhone. While I typed, the keyboard froze and my music started to skip. I tried and tried to power down, but nothing happened. I thought, "I'll have to call a friend and ask how to turn it off in this situation!" but of course then I remembered that I can't call someone when my phone is locked up. Fortunately, I had the iPhone User Guide on my laptop which (knock on wood!) has not been failing. Once I reset the phone, it announced that its battery was in the red. What?!

I really hope that the next thing to break down isn't the bus or, worse, the car right in front of the bus.

At least I don't have a pacemaker.

just what the cpan was missing

rjbs — 2008-10-06T04:00:54+00:00

I work for Pobox. We provide identity management. For the most part, it's about email. You register an email address with us and we handle the mail for you. We send it to an IMAP store, or your current ISP, or some flash in the pan webmail provider like Google. We do other things, though, like web and URL redirection. It's about managing services that relate to your identity.

Now, in internet years, we're ancient. We've been operating since 1995, which basically means that we've been providing internet services to real world end users for just about as long as there have been end users in the real world. (Inhabitants of .edu and .mil don't count; I said real world!)

That means that we used to offer some identity services that aren't really relevant anymore. For example, sometimes when sifting through data in the customer configuration datastores, I come across the plan configuration. Who here rememberes the .plan file? It's the precursor to twitter. You put what you were doing in your plan file and then when you got fingered, that file was served up.

Yeah, don't ask me about the unfortunate name "finger." I didn't name it.

Anyway, the long-gone support for finger at Pobox came up recently at a planning meeting. I said that I'd be focusing my time on re-enabling it during some planned service updates. The sysadmins groaned, and it encourated me. I wrote "finger server?" in my meeting notes.

This weekend, I made another crack about it, and got more worried noises from a sysadmin. I pulled up the RFC, realized how incredibly simple it would be to implement, realized that the CPAN didn't have a finger daemon, and did wrote one. Net::Finger::Server is available for installation.

Just to show that it isn't entirely useless, I threw together a little finger daemon:

~$ finger @git.codesimply.com | head [zodiac.codesimply.com] Repository Description Acme-Canadian Canooks in your code, eh? Acme-Lingua-EN-Inflect-Modern modernize Lingua::EN::Inflect rule's Acme-ProgressBar a simple progress bar for the patient Acme-Studly convertBetween various_well_known Ide... Amce-CNA a moer tolernat verison of mehtod loc...

...and then...

~$ finger Amce-CNA@git.codesimply.com | head [zodiac.codesimply.com] Project : Amce-CNA Desc. : a moer tolernat verison of mehtod location Clone URL: git://git.codesimply.com/Amce-CNA.git

I don't really think that we'll be bringing back finger support at Pobox. That's fine, we have better things to do. Finally, though, the CPAN has a server for the only protocol whose RFCs have sections on integrating with vending machines.

email::messageid 1.400

rjbs — 2008-10-03T11:48:51+00:00

I've uploaded a new Email::MessageID, version 1.400. It has two major improvements. First, the result of its new method is now an Email::MessageID object, where it was previously an Email::Address object(!). This means that it can now have its own message-id-specific methods. It has one: in_brackets. This is a very common mistake:

my $email = Email::Simple->create( body => ..., header => [ ... 'Message-Id' => Email::MessageID->new->as_string, ], );

This would produce a bogus message. Message-Ids must be in angle brackets. Rather than fix as_string and break the code of everyone who was already dealing with this, I've added in_brackets. Use it!

new app cmd interface finally released

rjbs — 2008-09-15T12:30:58+00:00

One of the earliest libraries I wrote for Pobox is ICG::CLI. It makes it easy to write CLI programs by tying together Getopt::Long::Descriptive with some common options (like help and verbose) and a few bits of code to make them work. So, you got three routines, whisper, say, and yell, which were like printf, but respected the verbose and quiet flags. (This got refactored into Log::Speak, which I don't think was ever released. Oh well.)

Later, I pulled out Rubric's subcommand dispatch code out into App::Cmd, and that was a big success, and used GLD, but didn't get you the printing builtins for verbose operation. Also, building them in wasn't going to be easy, because App::Cmd was OO instead of just a few imported routines. I finally decided on a way to make it happen, about six months ago, and it was a lot of fun to implement. It makes writing App::Cmd programs even easier, introduces fun uses for Sub::Exporter and lets you write plugins that provide exported functions into all your commands magically. That means that this is now possible:

package MyApp::Command::purge; use MyApp -command; sub run { my ($self, $opt, $args) = @_; debug for $args->flatten; } 1;

...because you can have (for example) a debug routine and autobox imported into every command automatically. I have a number of plugins to write to make this possible, and I've basically been putting off the release for months, waiting to get them written, but I had enough people saying, "Seriously, can you release what's in your git repo?" that I finally broke down and did it.

Hopefully I'll get some cool plugins written, real soon now.

more stupid headlines

rjbs — 2008-09-12T14:02:00+00:00

Only a few weeks after reading in the Sunday Times that The Castle and the Trial were "Kafkaesque," today, CNN has this headline:

Warning: Ike may bring 'certain death'

I think I would've phrased it, "Possibility of death is certain" but there's no accounting for taste.

salt and vinegar goldfish crackers

rjbs — 2008-09-08T23:27:53+00:00

Wow. I was looking at Wikipedia's article on Goldfish crackers recently (don't ask) and I saw an amazing fact. In "some regions of Canada" one can acquire salt and vinegar goldfish crackers.

I must try these! Are they delicious? Can you get me some? Help!

rx, json schema, and uri templates

rjbs — 2008-09-08T04:04:14+00:00

I got to feeling like maybe JSON Schema was not yet firm enough to give up on, and that maybe I could help improve it in the areas where Rx's design made more sense. I made one or two very minor suggestions that were accepted, but the most important area was not addressed. You can read more about it at the Google Groups thread.

I moved on to looking at other parts of my project, and realized that I was going to need a way to templatize a URI. Maybe it was time to look at URI Templates again. When I'd first seen them, they didn't seem like they provided much. You'd say:

http://gallery.example.com/user/{username}/photo/{photoid}

...and then fill it in with:

{ username => "rjbs", photoid => 1234 }

That was it. Still, that's more or less everything I'd need. I thought I'd have another look and see if there was some reason not to use it.

Well, in the time since I first saw it, there have been a few revisions to the spec, adding features like list-value variables and operators. So you can say:

http://x.com/?valid={-opt|true|valid}&{-join|&|foo,bar}&baz={-list|&baz=|baz} { valid => 1, foo => 'x', bar => 'y', baz => [ 1, 3, 5 ] }

...to get:

http://x.com/?valid=true&foo=x&bar=y&baz=1&baz=3&baz=5

That -opt operation says "if the value for 'valid' is not empty, then evaluate to the string 'true' and otherwise the empty string." There's also an inverted form called -neg. The behavior of -join should be clear. Unfortunately you're not allowed to pass list-value items to it, so it's not very useful for building a query string that would contain multiple entries for a given name. That's a shame, since it seems to be intended for use in building query strings. Meanwhile, -list (once called -listjoin) doesn't act like -join. It doesn't include the name of the variable (or equals sign) with each entry, so I have to include it in the join string and before the expansion.

Here's another way to do it:

http://x.com/?valid={-opt|true|valid}&{-join|&|foo,bar}{-prefix|&baz=|baz}

-prefix (along with -suffix and -list) are actually shown in examples forming paths, letting you turn our baz value above into 1/2/3/ or /1/2/3.

The first thing I wonder is this: why do we need a template system for query strings, anyway? They're easy, they've been around since forever, and the existing tools already handle all the crap like multiple values for one named parameter. The more interesting problem is in the path and maybe, sometimes, the domain. The most important feature is probably just dropping in a named value.

Another useful feature mentioned (and according to the archives at least briefly implemented) but not in the spec is the ability to deal with substrings. The system described looked like this:

http://cpan.org/{-sub|0,1|author}/{-sub|0,2|author}/{author}

I think that stinks. In most cases that I've seen this kind of thing, you're just making a simple path, and it looks like one of these:

R/J/BS R/J/RJBS R/RJ/RJBS S/B/RJBS

I sure don't want to have to give every substring that I need, with those slashes. Why not:

http://cpan.org/{buildup:author,3,cumulative}

...or something more along those lines.

I think I need to look through the APIs that I'd use this on and see what features I'd want, and whether URI Templates address those well. If not, I'm not going to settle on something that gives me features I don't need but none that I do. I just don't relish trying to compete with yet another somewhat successful system.

code simply now migrated to gitosis

rjbs — 2008-09-07T02:58:08+00:00

For a few days now I've felt really unproductive, but also unenthused about any of the work I had in front of me. Today, I managed to get one thing done, though: I reconfigured Code Simply's git setup to use gitosis.

Among other things, it means I can hand out the equivalent of "commit bits" to projects at git.codesimply.com without handing out system users. Now other people can be productive for me!

why sigils are great

rjbs — 2008-09-05T00:11:25+00:00

Last night at ABE.pm, I was talking a little bit with The Gang about some of the things I came to believe while doing the same thing in multiple languages. In explaining some of the issues I have with Ruby, both the scope of variables and the resolution of methods, this example came to me:

class X def x; return 10; end def y if false x = 100 end return x end def z return x end end obj = X.new puts "result of obj.y: #{obj.y.inspect}" puts "result of obj.z: #{obj.z.inspect}"

This will print:

result of obj.y: nil result of obj.z: 10

When obj.y is called, a variable called x is not assigned to, because the assignment happens in a conditional branch that is not entered. Nonetheless, the assignment to the variable creates the variable, because variables are not block scoped and because variables need not be explicitly declared. This means that when return x is reached, x is a local variable with no assigned value. That variable, which is nil, is returned.

When obj.z is called, nothing has defined x in the method's scope, so Ruby resolves it as a method call on the current target, obj. In other words, the x in the method z is the same as self.x.

Of course, good method and variable names goes a long way to making this less of an issue, but "Can't we just agree to not introduce bugs?" isn't a great safety net.

In Perl 6, the y method would be something like:

method y { if false { my $x = 100 } return $x; }

...but that's a syntax error, because Perl has block scope. To return the variable you'd say:

method y { my $x; if $bool { $x = 100 } return $x; }

To return the result of a method on self, you'd say:

method y { if $bool { my $x = 100 } $self.x; }

...and if you have a lot of stuff operating on $self and want to avoid a bunch of that typing, you could write:

method y { if $bool { my $x = 100 } given $self { # (pretend we have lots of code here that we save typing on) return .x; } }

It's always clear whether we're dealing with a variable (there's a $ or other variable-marking sigil) or a method (there's an invocant or a lone dot, which acts something like a method-call sigil).

I think that Ruby's use of sigils is great. Its sigil-for-scope makes much more sense given Ruby's "everything is an object" variables. I just wish that they had a sigil for local scope variables.

This would actually be a huge improvement on a current problem. See, Ruby 1.9 is going to have block-level scope. Unfortunately, this will sometimes change how code worked.

value = 10; array.each { |value| do_something }

This code (in Ruby) will clobber value, because the value in the proc is not created in a distinct scope from the value to which 10 was assigned. When Ruby 1.9 is adopted, this code will change. The two value variables will be distinct, and the first value will not be clobbered.

If block scope had been introduced with a new sigil this would have been avoided. Your old code would be:

*value = 10; array.each { |*value| do_something }

Well, of course it's clear that these would be the same! They both have the (hypothetical) function-scope sigil. Ruby 1.9 wouldn't break this, because instead of changing the way sigil-free local variables worked, it would let you have a block-local variable if you asked for it:

%value = 10; array.each { |%value| do_something }

That first variable could use a star, too. The important thing is that the smaller-scoped variable would be marked as only existing in its block-local scope. Of course, running out of sigils is a risk, but I'd rather see Ruby 3.0 require twigils than have backcompat issues like this.