Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Wednesday November 11, 2009
06:39 AM

Kill Off Test::Builder?

[ #39878 ]

We've been waiting for Test::Builder 2.0 for quite some time but many of us need some of the promised benefits now. I've been giving some thought to what can be done about this.

One of the things we want, for example, is structured diagnostics. Another thing we'd like is for the existing diagnostics to be tied to the test function they're associated with. Unfortunately, here's a typical test function:

sub is_valid_json ($;$) {
    my ( $input, $test_name ) = @_;
    croak "usage: is_valid_json(input,test_name)"
      unless defined $input;
    eval { $JSON->decode($input) };
    my $test = __PACKAGE__->builder;
    if ( my $error = $@ ) {
        $test->ok( 0, $test_name );
        $test->diag("Input was not valid JSON:\n\n\t$error");
        return;
    }
    else {
        $test->ok( 1, $test_name );
        return 1;
    }
}

You'll note that the diag and ok are separate functions. This hurt me with Test::Most because I wanted the "die or bail on fail" to hit as soon as there's test failure, but that won't work because diagnostics are output after the test function. Thus, we have to wait until the next test is run to die or bail. This probably doesn't affect many people, but for those it does, it's a real pain.

I'm trying to think of a way around this and my first thought is simply sidestep the entire Test::Builder problem. Phase 1 of the attack:

package Test::Runner;

our $VERSION = '0.01';

BEGIN {
    require Test::More;

    $INC{$_} = "Replaced by Test::Runner $VERSION" foreach qw(
        lib/Test/Builder.pm
        lib/Test/Builder/Module.pm
        lib/Test/More.pm
    );
}

And in another package:

package Test::Runner::Builder;

sub Test::Builder::new  { ... }
sub Test::Builder::ok   { ... }
sub Test::Builder::diag { ... }
... and so on

And in another package:

package Test::Runner::More;

use Test::Runner qw(
  ok is isn't isnt like unlike cmp_deeply and so on
);

The idea is to create a compatible interface to the old, familiar functions, but the Test::Builder functions would record their responses but not output anything until after the test function is over. Thus, anyone who wanted to have proper diagnostics from their test program would use Test::Runner instead of Test::Builder and they would have to register every test function. This would give us the needed information to control what can and cannot be used with tests.

The reason we'd have to override much of the current Test::Builder infrastructure is because older test modules aren't going to use Test::Runner. We'd have to check what was in @EXPORT (or wrap &import in some cases?) and assume that anything which gets exported into someone's namespace from a test module is a test function.

It's an ambitious plan and fraught with error, but in the absence of Test::Builder 2.0, I think it's a worthwhile experiment.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.