Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

BooK (2612)

BooK
  {book} {at} {cpan.org}
http://paris.mongueurs.net/
Yahoo! ID: philippe_bruhat (Add User, Send Message)

Obfuscation [plover.com]. Pink [axis-of-aevil.net]. HTTP::Proxy [cpan.org]. YEF [yapceurope.org]. Fishnet [perl.org]. Kapow [cpan.org]. Cog's [perl.org] bitch [cpan.org]. Invitation [perl.org]. White [perl.org] Camel [perl.org]. Nuff' said.

Journal of BooK (2612)

Friday August 20, 2010
05:17 PM

Flore Louise Apolline Bruhat-Souche

On Thursday, August 19, 2010 at 9:30, Flore Louise Apolline Bruhat-Souche was born. She weighs 3.02 kg and measures 48 cm.

Word already spread through IRC (#perlfr and #yapc mostly) and via email and telephone.

The mother is fine, the father is slightly tired and the big sister is happy.

There is one photo online.

Wednesday March 24, 2010
08:07 PM

Fixing Mailman with Perl

Mailman is useful. Mailman works. Mailman is ubiquitous. I am subscribed to over 50 mailing-lists managed by Mailman.

But Mailmand is software, and therefore hateful.

My particular Mailman hate is the nodupes parameter.

Avoid duplicate copies of messages?

When you are listed explicitly in the To: or Cc: headers of a list message, you can opt to not receive another copy from the mailing list. Select Yes to avoid receiving copies from the mailing list; select No to receive copies.

If the list has member personalized messages enabled, and you elect to receive copies, every copy will have a X-Mailman-Copy: yes header added to it.

I like duplicate email. Moreover, I like the List-Id header that makes emails sent through a list special (at least in the sense that they can be filtered automatically by more tools, and I can just delete the stuff that piles up in my Inbox). And by the way, how could Mailman be really sure that I got that other copy? Just because the headers say so? Bah.

Oh, and I also hate the fact that Set globally never worked for me with this option.

So, because I'm lazy, and I don't want to go clikety-click to first, get a reminder of the random password that was assigned to me years ago, and two, login and change that annoying option, and because I don't want to do that fifty times, over and over again...

I wrote and put on CPAN WWW::Mailman, designed to automate that kind of tedious task out of my life (and hopefully yours). Examples included, I know you're lazy too.

PS: I've been told there is a command-line interface to Mailman, but it is reserved to people managing Mailman on the server.

Monday January 04, 2010
07:35 PM

Pink stinks

I still like wearing pink T-shirts, though. And not just at Perl conferences. (My love of pink basically comes from pushing a "joke" from the YAPC Europe 2002 auction way beyond its scope.)

Some very interesting reads:

A little more than a week ago, it was the second Christmas of my daughter... I guess it shows. Raising children brings all kinds of new interesting issues and questions to one's attention.

Tuesday December 15, 2009
05:22 PM

git-move-dates

As a Perl programmer and Open Source enthousiast, you probably sometimes contribute to Open Source projects. Maybe even (gasp!) during work hours. If your employer is jealous of your time, you probably do not want your commits to look like they were done during work hours (especially in tools like the GitHub Punch Card).

On the other hand, it doesn't make sense to not commit your changes, and lose the benefits of using Git, just so that the reality of when you worked on these tiny changes is not made public. (At that point, it would probably also make more sense to have an open discussion with your boss...)

The way Git handles history makes it really easy to change the date of commits on a local branch. When I first thought about it, my idea was to write some date manipulation code (move a bunch of commits from a time range to another with all kinds of fancy nooks and crannies) and manipulate the Git trees and commits myself.

Then I discovered git filter-branch, which is all about rewriting commits. And I realized that in the situation above, moving commits a few hours in the future (like ten minutes before actually using git push or git send-email) is largely sufficient.

The problem is that the code to move a bunch of commits one hour in the future looks like this:

    git filter-branch --commit-filter '\
      GIT_AUTHOR_DATE=`echo "$GIT_AUTHOR_DATE"|perl -pe'\''s/\d+/$&+3600/e'\''`;\
      GIT_COMMITTER_DATE=`echo "$GIT_COMMITTER_DATE"|perl -pe'\''s/\d+/$&+3600/e'\''`;\
      git commit-tree "$@"' -- <rev-list>

Which is impossible to remember, and painful to write.

So, lazy as a Perl programmer should be, I just wrote git-move-dates, that writes and runs the above type of command-lines for me. Useful options include --committer and --author (to change only one of the two existing dates), and options ranging from --seconds to --weeks to define the exact timespan of your commits' time-travels.

As with my other Git gadgets, the source is available from http://github.com/book/git-gadgets.

And remember: there's nothing wrong with rewriting history, as long as it's unpublished, local history. ;-)

Thursday December 03, 2009
03:55 AM

More Git gadgets

Last week on p5p, after noticing that the new way to name version tags in perl5.git was to use v$version , I proposed to copy the old perl-$version tags to the new scheme, going back at least to 5.6.0 (which is when the Perl naming convention for version changed). I haven't done it yet for various reasons, but in order to do it, I developped a small git-copytag tool, that copies (or moves) the tag with all its annotations. Clone it from http://github.com/book/git-gadgets.

I also started to check if the existing tags in perl5.git point to the commit matching the published distribution, but my first attempt failed miserably (creating Git tree objects from the distributions and trying to find a commit pointing to the same tree id). I'll move on to a different approach (diff -r, basically).

In other news, I finally succeeded in creating the Git fractals I wanted. I'll soon post more details about it on the page dedicated to Sierpiński triangles on my personal site.

Monday November 23, 2009
07:41 AM

git-mtime

Ever complained that on a checkout Git did not reset the mtime of your files to the date when they were commited?

My home page is generated with Template Toolkit but a script stored in the post-receive hook. On a checkout Git only updates the files that have changed, so normally I can trust template.modtime to be correct and use it to show a Last mofidied date.

But I'm a perfectionist, and I want to be extra sure. So I created this little utility, that I called git-mtime:

    #!/bin/sh
    git log --name-only --date=iso --reverse --pretty=format:%at "$@" \
    | perl -00ln -e '($d,@f)=split/\n/;$d{$_}=$d for grep{-e}@f' \
                 -e '}{utime undef,$d{$_},$_ for keys%d'

Note that it passes all parameters to git log, so you can apply it on a subdirectory (using --), or even use the dates from another branch (though I'm not sure what use this can have).

And for extra bonus points, it uses the secret eskimo greeting operator!

Now that I have a few gadgets based on Git, I thought I might as well publish them somewhere. A quick look on Github ruled out git-tools, git-utils and git-extras (come on people, most of these things could be done with Git aliases!). git-aid (especially the plural) didn't seem like a good name either. So after looking around for synonyms, I settled on git-gadgets.

Clone it from http://github.com/book/git-gadgets.

Thursday November 19, 2009
07:43 PM

Git fractals

Last week (November 11) over dinner in Amsterdam, I talked with a colleague about Git as a tool for creating graphs. For some reason I started to think about a Sierpirński triangle, and we started trying to create such graphs with Git.

The basic shape is a triangle (in the UTF-8 drawing below, the arrows represent the parent → child relationship):

    ⋅
    ↓ ↘
    ⋅ → ⋅

It is quite easy to create by hand. I did it using git commit-tree, using always the same tree object (the empty tree), as we only care about the graph that represents commit lineage, not about the content.

The next step basically repeats the same shape, attached to the low-level nodes / commits of the previous graph:

    ⋅
    ↓ ↘
    ⋅ → ⋅
    ↓ ↘ ↓ ↘ 
    ⋅ → ⋅ → ⋅

I could create it in a couple of minutes, with a few more git commit-tree commands.

After that, it stops being interesting to do by hand, and one wants to program it. My goal has been to create the following shape, and larger ones, using a Perl program.

   A
    ⋅
    ↓ ↘
    ⋅ → ⋅
    ↓ ↘ ↓ ↘ 
  B ⋅ → × → ⋅ C
    ↓ ↘     ↓ ↘
    ⋅ → ⋅   ⋅ → ⋅
    ↓ ↘ ↓ ↘ ↓ ↘ ↓ ↘ 
    ⋅ → × → × → × → ⋅

Until now, I've been trying several recursive approaches, and failed miserably each time. The big issue is the merge points, showed in the graph above using ×.

In my recursive approaches, I usually first created the triangle (ABC), usually and then started again from B and then C. But the last merge of B (the × in the middle of the bottom line) can only be created once the vertical line starting from C has been started. I should probably keep state in some way, but haven't had much time to spend on this.

I also thought about using the mapping from Pascal's triangle (odd numbers as dots and even numbers as empty space, see both Wikipedia pages for details), but haven't actually tried it yet.

In other news, I've started to take care of my personal web site again. It hadn't been touched since 2002, so I gave it a facelift. Surprisingly enough, the only section yet is about Git fractals. ;-) Links to Git repositories, GraphViz images and more successful attempts with other fractals are also available there.

Thursday November 12, 2009
11:50 AM

git-dot

I'm currently experimenting with creating graphs using Git... I'm not using historical data, or even data at all (I'll soon know the SHA1 of the empty tree by heart), just nodes with %s as their label (I have yet to find a use for the rest of the metadata).

gitk is nice to look at historical information, but not so good for graphs. On the other hand, GraphViz is great for showing graphs.

What's best than Perl (and a tiny wrapping of shell on top) to produce graphs?

    #!/bin/sh
    # create a good looking graph with dot
    echo "digraph G {"
    git log --pretty=format:"%h %p" $* \
    | perl -lna  -e 'print qq("$F[0]";),map{qq("$_"->"$F[0]";)}@F[1..$#F]'
    echo "}"

The output of this is usually boring, so just but pipe it to dot -Tpng -ograph.png and watch the pretty pictures.

Also, imagine a graph that has a full filesystem attached to each node. This is exactly the kind of stuff that Git can give us.

Not that I have any idea what this could be used for...

Sunday November 08, 2009
11:34 AM

nopaste

Just a quick post to show off a small and useful script I use whenever I need to "nopaste" some text or code:

    #!/usr/bin/perl -w
    use strict;
    use WWW::Mechanize;
    use Getopt::Long;

    my %SITE = (
        snit  => 'http://nopaste.snit.ch/',
        scsys => 'http://paste.scsys.co.uk/',
    );

    my %CONF = (
        channel => '',
        nick    => '',       # use your own
        summary => '',
        paste   => '',
        site    => 'snit',
        list    => '',
    );

    GetOptions( \%CONF, 'lang=s', 'nick=s', 'summary|desc=s', 'paste|text=s',
        'list!', 'site=s' )
        or die "Bad options";

    die "No such paste site: $CONF{site}\nValid choices: @{[keys %SITE]}\n"
        if !exists $SITE{ $CONF{site} };

    my $m = WWW::Mechanize->new;
    $m->get( $SITE{ $CONF{site} } );
    die $m->res->status_line unless $m->success;

    if ( $CONF{list} ) {
        print "Possible channels for $CONF{site}:\n",
            map {"- $_\n"} grep $_,
            $m->current_form()->find_input('channel')->possible_values;
        exit;
    }

    unless ( $CONF{paste} ) {
        $CONF{summary} ||= $ARGV[0] || '-';
        $CONF{paste} = join "", <>;
    }

    delete @CONF{qw( site list )};
    $m->set_fields(%CONF);
    $m->submit;
    die $m->res->status_line unless $m->success;

    print +( $m->links )[0]->url, "\n";

Since it works has a filter, I can call it from vim or pipe to it. It also works with a file parameter, which is used to set the paste title.

Just before posting this, I looked again on CPAN, and found the follwing:

  • App::NoPaste: Seems really complete. But does much more than I need, and I like depending only on WWW::Mechanize for such tools.

  • WWW::Rafb: Well, in an earlier version my script worked with rafb.net, but the site itself is down.

  • WWW::PasteBin: Such a huge collection of distributions, I wouldn't which to install first.

  • WebService::NoPaste: Found it years ago when looking for a nopaste utility, but I preferred to write my own.

Clearly, there is no lack of modules to nopaste stuff, so I'm not going to add my own to the list. :-)

I tried publishing scripts on CPAN, but I feel the toolchain is not really targetting scripts, anyway. Anyway, I have a few utility scripts like the above lying around, and I'm thinking maybe the best way to go with them nowadays is to just publish them on GitHub. Might happen someday.

Thursday October 29, 2009
03:44 PM

buildperl.pl

I'm trying to compile a bunch of old perls to test my modules against them. I started with 5.8.9, and it went like this:

    $ git checkout -f perl-5.8.9
    [git output]
    $ git clean -xdf
    [more git output]
    $ sh Configure -Dprefix=/opt/perl/5.8.9 -des -Uinstallusrbinperl
    [Configure output]
    $ make && make test && make install
    [make output]
    [test output]
    [install output]

And, ta-da! After less than 15 minutes, Perl 5.8.9 was in /opt/perl/, ready to be used.

Encouraged by this, I went on to compile 5.8.8. I was a bit disappointed when the same procedure (after a s/5.8.8/5.8.9/g) failed with:

    $ sh Configure -Dprefix=/opt/perl/5.8.8 -des -Uinstallusrbinperl
    [Configure output]
    Run make depend now? [y]
    sh ./makedepend MAKE=make
    make[1]: Entering directory `/data/home/book/src/ext/perl'
    sh writemain lib/auto/DynaLoader/DynaLoader.a  > perlmain.c
    rm -f opmini.c
    cp op.c opmini.c
    echo  av.c scope.c op.c doop.c doio.c dump.c hv.c mg.c reentr.c perl.c perly.c pp.c pp_hot.c pp_ctl.c pp_sys.c regcomp.c regexec.c utf8.c gv.c sv.c taint.c toke.c util.c deb.c run.c universal.c xsutils.c pad.c globals.c perlio.c perlapi.c numeric.c locale.c pp_pack.c pp_sort.c miniperlmain.c perlmain.c opmini.c | tr ' ' '\n' >.clist
    make[1]: Leaving directory `/data/home/book/src/ext/perl'
    ./makedepend: 1: Syntax error: Unterminated quoted string
    make: *** [depend] Error 2

    If you compile perl5 on a different machine or from a different object
    directory, copy the Policy.sh file from this object directory to the
    new one before you run Configure -- this will help you with most of
    the policy defaults.

In the past, I had tried to compile older Perls, and Sébastien Aperghis-Tramoni had pointed me to a few patches he had made to be able to compile 5.004_05 with a more recent (3.x) gcc.

So, by looking at his patch again and the output of git diff perl-5.8.8 perl-5.8.9 -- makedepend.SH I was able to produce a patch, that looked exactaly like Sébastien's patches for 5.004_05. The Configure step now worked!

But then it's the compilation phase that failed:

    $ make
    [make output]
        Making IPC::SysV (dynamic)
    Checking if your kit is complete...
    Looks good
    Writing Makefile for IPC::SysV
    make[1]: Entering directory `/data/home/book/src/ext/perl/ext/IPC/SysV'
    make[1]: Leaving directory `/data/home/book/src/ext/perl/ext/IPC/SysV'
    make[1]: Entering directory `/data/home/book/src/ext/perl/ext/IPC/SysV'
    cp Msg.pm ../../../lib/IPC/Msg.pm
    cp Semaphore.pm ../../../lib/IPC/Semaphore.pm
    cp SysV.pm ../../../lib/IPC/SysV.pm
    ../../../miniperl "-I../../../lib" "-I../../../lib" ../../../lib/ExtUtils/xsubpp  -typemap ../../../lib/ExtUtils/typemap  SysV.xs > SysV.xsc && mv SysV.xsc SysV.c
    cc -c   -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2   -DVERSION=\"1.04\" -DXS_VERSION=\"1.04\" -fpic "-I../../.."   SysV.c
    SysV.xs:7:25: error: asm/page.h: No such file or directory
    make[1]: *** [SysV.o] Error 1
    make[1]: Leaving directory `/data/home/book/src/ext/perl/ext/IPC/SysV'
    make: *** [lib/auto/IPC/SysV/SysV.so] Error 2

This looked a bit trickier. Luckily, Google informed me that the asm/page.h file had moved in the Linux tree. Using git again, I looked for changes involving page.h. The changes were a lot bigger, and harder for me to understand.

By chance, the first interesting diff I found was for a change in ext/Devel/PPPort/devel/buildperl.pl. Wow, this looked exactly like the tool I needed to, erm, build perl.

So instead of adding complexity to my home-made perl-building script by adding more cases where a patch was necessary, I tried my hand at patching buildperl.pl to make it support newer gcc versions than the ones that existed at the time the older perls were written.

Looking for the proper patches with git is extremely easy, and I was quickly able to find the necessary patches. The longer part was to actually compile all Perls from 5.6.0 to 5.9.5. :-)

Since buildperl.pl works with perl archives, my test script ended up like this:

    #!/bin/sh

    # blead has my local patch
    git checkout blead
    git clean -xdf

    # setup temporary directories
    rm /tmp/buildperl.log
    rm -rf /tmp/perl
    mkdir /tmp/perl
    mkdir /tmp/perl/source

    # compile and test all the tags given on command-line
    for tag in $* ; do

        # get the version
        version=`echo $tag|cut -d- -f 2`

        # make a tarball
        echo "=== creating /tmp/perl/source/perl-$version.tar.gz"
        git archive --format=tar --prefix=$tag/ $tag^{tree} \
        | gzip > /tmp/perl/source/$tag.tar.gz

        perl cpan/Devel-PPPort/devel/buildperl.pl --config default --perl $version --test

        # check it was installed correctly
        if [ -d /tmp/perl/install/default/$tag/ ] ; then
            result="ok"
        else
            result="not ok"
        fi
        echo "$result - $version" >> /tmp/buildperl.log

    done

Then I just fed it with the appropriate git tags, and let it run for a while:

    $ ./compile_perl `git tag -l 'perl-5.8*'`

So within one and a half hour, I was able to compile, test and install all the 5.8.x versions of Perl.

I already have added the changes that allow 5.6.x, 5.8.x and 5.9.x to compile again in modern environments, and just sent the patches to P5P and Marcus Holland-Moritz.

Eventually, I'll work on adding the needed patches for versions 5.004, 5.005, and send another patch batch.