Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Wednesday May 13, 2009
04:53 PM

Test::Aggregate Breaking Backwards Compatability

[ #38974 ]

The new Test::Builder is giving me fits. It incorporates some lovely changes, but there's a problem. The old Test::Builder required some very nasty internals hacks by Test::Aggregate to work. Unfortunately, Schwern's changed the internals for the first time in years. This is his right and I shouldn't have relied on them, but there wasn't much else I could do unless I bundled my own Test::More distribution and that would have introduced an entirely different set of problems (just bundling Test::Builder would fail as an updated Test::More might try to call new Test::Builder features as it does now).

The Test::Aggregate I'm working on now faces the problem of multiple calls to done_testing(). Further, there are issues with calls to any tests in BEGIN/CHECK/INIT/END blocks. I think I'm going to go ahead and bite a huge bullet (this was marked as ALPHA code, after all). I'm going to deprecate tests in those blocks and eventually call BAILOUT on them.

The problem with code in BEGIN/CHECK/INIT/END blocks is that there are timing problems associating them and associating the right test with the right test program for diagnostics. Now, however, you can do this:

use Test::Aggregate;
Test::Aggregate->new(\%options)->run;
done_testing(217);

And that will quite nicely provide an ending plan check for you. However, if any of your aggregated tests provide any tests in END blocks, those are automatically run after your call to done_testing().

As further incentive to remove BEGIN/CHECK/INIT/END blocks, we're moving to nested TAP. It's critical that tests be nested correctly and there's no way to guarantee this with tests run in those blocks. As a result, I have the following really, really nasty hack in Test::Builder::ok():

{
    my $ok;
    BEGIN { $ok = \&Test::Builder::ok }

    my %FORBIDDEN = map { $_ => 1 } qw/BEGIN CHECK INIT END/;

    sub Test::Builder::ok {
        __check_test_count(@_);
        my $level  = 1;
        while (1) {
            my ($caller) = ( ( ( caller($level) )[3] || '' ) =~ /::([[:word:]]+)\z/ );
            last unless $caller;
            if ( $FORBIDDEN{$caller} ) {
                my ( $self, $test, $name ) = @_;
                $test = $test ? "Yes" : "No";
                my ( $filename, $line ) = (caller($level))[1,2];
                $self->diag(<<"                END");
Aggregated tests must not be run in BEGIN, CHECK, INIT or END blocks.
File:  $filename
Line:  $line
Name:  $name
Pass:  $test
                END
                $self->BAILOUT('Illegal aggregated test call');
            }
            $level++;
        }
        local $Test::Builder::Level = $Test::Builder::Level + 1;
        $ok->(@_);
    }
}

I know that people aren't going to be terribly thrilled with this, but it solves many, many problems.

The main issue that people are going to have is this:

BEGIN { use_ok 'My::Module' or die }

I recommend that you stop using this meme. Simply put a 'use My::Module' at the top of your code and put all of your 'use_ok' tests into one 'load.t' test which is not aggregated.

I'll try and release the next Test::Aggregate with a strong deprecation warning for tests in those blocks instead of the BAILOUT I have in my test code, but I'm sad I have to do this. Still, the long-term benefits you'll gain from nested TAP in Test::Aggregate should be worth it.

I hope.

It should be noted that at the last Perl-QA hackathon, neither Schwern nor I found a clear way of avoiding messing with Test::Builder internals. Doing this cleanly waits on Test::Builder 2.0.

Suggestions welcome :)

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.