Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Arador (8415)

Journal of Arador (8415)

Wednesday August 26, 2009
10:30 AM

Prototypes: the good, the bad,and the ugly

This is a reply to chromatic's post The Problem with Prototypes, but his blog didn't allow me to post it there so I post it here.

Pretty much everyone agrees that there are good (such as blocks and sometimes reference) prototypes and bad ones (scalar most of all). Few discuss the third class: the ugly glob prototype.

perlsub describes them as such:

A * allows the subroutine to accept a bareword, constant, scalar expression, typeglob, or a reference to a typeglob in that slot. The value will be available to the subroutine either as a simple scalar, or (in the latter two cases) as a reference to the typeglob.

In other words, they are the same as scalar prototypes, except that they also accept globs and barewords. This is mainly used to pass filehandles, like this:

sub foo (*) {...}

foo STDIN;

but in fact it can be used to pass any bareword to function, as it leaves the interpretation of it to the function.

It's tempting to call this bad, but it offers some API possibilities that would otherwise not be possible, hence I would call it ugly rather than bad per se.

Saturday August 22, 2009
11:10 AM

use 5.011/5.012 should be progressive

As some may know, use 5.011 activates strictures. I think that's a good idea, but I don't think that's enough. I strongly feel that it should be more progressive than that. I think it should do what Modern::Perl aims to do implement: Some small, important and uncontroversial pragmas (as opposed to Perl 5i, which does do big and controversial things). My list of things would be:

  • use feature '5.011';
  • use warnings;
  • use IO::Handle;
  • no indirect;
  • use mro 'c3';

Anyway: I'd like to hear what other people think use 5.011 (and thus eventually use 5.012) should mean.

Thursday June 04, 2009
03:43 PM

CPAN2

Recently there has been a discussion about what the New CPAN of Perl6, called CPAN6 by some, is going to be like. I've heard a lot of good ideas from different people, but I'd like to take a slightly different approach. I'm all in for CPAN6, but let's have CPAN2 first.

I don't think it's helpful to reinvent all pieces at the same time. Let's keep the good pieces and get rid of the bad pieces. Module::Build and Module::Install got that right. ExtUtil::MakeMaker sucked ass, so people decide to make something better. We need more of that.

CPANPLUS.pm didn't get it right. It's a better implementation of a wrong idea. CPAN's handling of meta-data sucks. In particular, the way it doesn't deal with dependencies is problematic. We can easily come up with something better. The data exists, let's make it more accessible.

A perfect solution would require a major overhaul, but it could be fixed too by simply adding an extra tier to the CPAN ecosystem that deals with meta-data. I can think of better solutions than that for the future (where the meta-data is integrated into the whole stack), but to be honest I'm more interesting in a solution for NOW than a solution in the far future.

Wednesday February 18, 2009
08:03 PM

Reddit is lamer than I thought

Reddit is way more broken than I thought it was. Someone posted this piece of crap (see this link for why I think it's crap) and it gets +10.

Really, some people must be voting things up way too easily.

:-|

Friday February 06, 2009
04:33 PM

Casting magic against segfaults

The problem

For years, there has been the Sys::Mmap module, however, it has a few issues. For example, let's take this piece of code:

use Sys::Mmap;
open my $fd, '+>', 'filename';
mmap my $var, -s $fd, PROT_READ|PROT_WRITE, MAP_SHARED, $fd, 0;
$var = 'Foobar';
munmap $var;

First of all, it's simply not user-friendly. mmap takes 6 arguments in a weird order, and uses weird constants. Also munmap shouldn't be necessary: variables should dispose of themselves when they run out of scope.

But more importantly, this program does not do what you think it does, though the only hint of that is an Invalid argument exception when doing munmap. During the assignment, the link between the mapping and the variable is lost, so nothing is written to the file. Worse yet, this can even lead to a segfault in some circumstances.

Ouch!

Tying things up?

The documentation clearly says that you shouldn't do this (or anything else that changes the length of the variable), but IMHO this hole shouldn't be left open in the first place, if only because it is extremely counterintuitive (and thus a maintenance nightmare). Modules should fail more gracefully than this.

Sys::Mmap offers a tied interface as compensation, but this didn't work out. The tied interface indeed is safe, but it creates another problem.

Every time it is read, it copies the whole file into the variable. Every time the variable is modified, it writes the whole new value to the file, even if the change only affects a single byte.

Ouch!

Obviously, that doesn't scale at all. One user of the module reported a 10-20 times slowdown of his program after converting to ties. That's not a workable solution.

The solution

Perl has a powerful but rarely used feature called magic. (It's rare use by module authors is indicated by the fact that the prototypes of the magic virtual table as documented in perlguts aren't even complete: they lack pTHX_'s). They are used by the perl core to implement magic variables such as $! and ties (surprise, surprise). It offers 8 hooks into different stages of handling a variable, the three most important being fetching(svt_get), storing(svt_set) and destruction(svt_free).

In my case, I didn't need get magic, but I did use set and free magic. Freeing the variable is not that interesting (simply unmapping the variable), but setting it is. This function is called just after every write to the variable:

static int mmap_write(pTHX_ SV* var, MAGIC* magic) {
    struct mmap_info* info = (struct mmap_info*) magic->mg_ptr;
    if (SvPVX(var) != info->address) {
        if (ckWARN(WARN_SUBSTR))
            warn("Writing directly to a to a memory mapped file is not recommended");

        Copy(SvPVX(var), info->address, MIN(SvLEN(var) - 1, info->length), char);
        SvPV_free(var);
        reset_var(var, info);
    }
    return 0;
}

This function is called after every write to the variable to check if the variable is still linked to the map. If it isn't, it copies the new value into the map, frees the old value and restores the link. As copying is potentially expensive, it will issue a warning if warnings (or actually, 'substr' warnings) is in effect.

There is no perfect solution to this problem, but getting a friendly warning is undeniably better than getting a segmentation fault or data loss.

Anyway, you can find Sys::Mmap::Simple here. It offers more goodies, such as portability to Windows, a greatly simplified interface, and built-in thread synchronization.

Tuesday January 27, 2009
02:05 PM

Elegance in minimalism

Some time ago, I read this journal entry by Aristotle. I liked it and suspected it could be easily implemented in XS. It turned out to be the most elegant piece of XS I've ever written.

void
induce(block, var)
    SV* block;
    SV* var;
    PROTOTYPE: &$
    PPCODE:
        SAVESPTR(DEFSV);
        DEFSV = sv_mortalcopy(var);
        while (SvOK(DEFSV)) {
            PUSHMARK(SP);
            call_sv(block, G_ARRAY);
            SPAGAIN;
        }

I assume most readers don't know much C, let alone the perl API or XS, so I'll explain it piece by piece.

void
induce(block, var)
    SV* block;
    SV* var;
    PROTOTYPE: &$

This declares the xsub. It has two parameters, both scalar values. The function has the prototype &$. So far little surprises.

    PPCODE:
This declares that a piece of code follows. Unlike CODE blocks, PPCODE blocks pop the arguments off the stack at the start. This turns out to be important later on.

        SAVESPTR(DEFSV);
        DEFSV = sv_mortalcopy(var);

These lines localizes $_ and initializes it to var.

        while (SvOK(DEFSV)) {
This line is equivalent to while (defined($_)).

Now comes the interesting part:
            PUSHMARK(SP);
            call_sv(block, G_ARRAY);
            SPAGAIN;

To understand what this block does, you have to know how perl works inside.

If you've read perlxs, you may notice this function does not push any values on the stack. Are careless reader might be mistaken and think this function doesn't return anything: they couldn't be more wrong!

If you've read perlcall, you would notice a lot more is missing. For starters, the function calls SPAGAIN (pretty much equivalent to saying I accept the values you return to me), but it doesn't do anything with them.
Also, you may notive that both ENTER/LEAVE(needed to delocalize $_) and SAVETMPS/FREETMPS (needed to get rid of temporary values) are missing. The function that calls the xsub automatically surrounds it by an ENTER/LEAVE pair, so that one isn't necessary. The lack of SAVETMPS/FREETMP however is not only deliberate but also essential.

The loop calls the block without arguments (PUSHMARK & call_sv). The xsub accept the return values on the stack and leaves them there. This sequence repeated. This way it assembles the induces values on the stack. PPCODE removing the arguments at the start prevents it from returning those as first two return values. It also adds a trailer that causes all elements that have been pushed on the stack to be recognized as return values of this function. That's why a SAVETMPS/FREETMPS pair would break this code: the values must live after the code returns.

That's the elegance of this function. It doesn't even touch it's return values, it delegates everyting to the block. All the things that are missing make that it does exactly what it should do.