Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Tuesday September 16, 2008
03:03 AM

How Test::Aggregate does 'skip_all'

[ #37462 ]

Before I start, I just want to say how incredibly thankful I am that I did not take that job at Lehman Brothers. It was soooo tempting.

Test::Aggregate used to not allow 'skip_all' in plans. This was because you would "skip_all" 12,000 tests instead of just the 20 or so in the test program. Very disappointing. It even warned you if it thought you might be using skip_all. I mentioned I might fix it one day, but now I had to fix it (0.34_03 and above), but it turned out to not be easy.

Test::Aggregate uses an Apache::Registry trick to make defer loading some code. We take a small test program like this:

use strict;
use warnings;

use Test::More 'no_plan';

is 2+2, 4, 'We can add!';

And turn it into something similar to this (trimming some bits):

  1 {
  2 #################### beginning of aggtests/test.t ####################
  3     package aggteststestt;
  4
  5     sub run_the_tests {
  6         local $0 = 'aggtests/test.t';
  7
  8 # line 1 "aggtests/test.t"
  9
10         use strict;
11         use warnings;
12
13         use Test::More 'no_plan';
14
15         is 2 + 2, 4, 'We can add!';
16     }
17 #################### end of aggtests/test.t ####################
18 }

Line 3 guarantees that we're in an encapsulated namespace. Line 6 solves a problem where many tests make assumptions about the contents of $0 and line 8 is a line directive which ensures that errors are reported with the original line number and filename. There's actually a lot more, but this is the core of what we're doing.

The problem was that a "skip_all" in Test::Builder figured out an easy way to halt the tests. It just calls exit and that doesn't work for me. Since these tests are now wrapped in subs, if I wanted to simulate "skip_all" I had to call "return", but as a stand-alone test program, that looks very strange. Nonetheless, I had this at the top of a number of test programs:

if ( $ENV{FAST_TESTS} ) {
    if ( $ENV{TEST_AGGREGATE} ) {
        Test::Builder->new->skip("Skipping $0 under aggregated fast tests");
        return;
    }
    else {
        plan skip_all => 'FAST_TESTS environment variable set';
    }
}
else {
    plan tests => $num_tests;
}

I couldn't factor that out into a subroutine as that "return" would have to return from an extra stack level. Ugh! (more than once I have failed to refactor code because I need to return or eval code at a different stack level from the current one). However, even using this would throw my "skip_all" warnings in Test::Aggregate.

I was washing dishes on Sunday and thinking "damn it. I wish I had a reliable source filter or Perl 6 grammars. That would make this problem trivial because I ... I ... I..."

Oh. I'm a dumbass.

I'm not using source filters, but I'm already rewriting the damned code! Now the above test script looks a bit like this when rewritten:

  1 {
  2 #################### beginning of aggtests/test.t ####################
  3     package aggteststestt;
  4
  5     sub run_the_tests {
  6         local $0 = 'aggtests/test.t';
  7
  8         AGGTESTBLOCK: {
  9             if ( my $reason
10                 = $Test::Aggregate::Builder::SKIP_REASON_FOR{aggteststestt} )
11             {
12                 Test::Builder->new->skip($reason);
13                 last AGGTESTBLOCK;
14             }
15 # line 1 "aggtests/test.t"
16
17             use strict;
18             use warnings;
19
20             use Test::More 'no_plan';
21
22             is 2 + 2, 4, 'We can add!';
23         }
24     }
25 #################### end of aggtests/test.t ####################
26 }

Ugly, but you should never see this code anyway. Problem solved. (I wanted to put the "last AGGTESTBLOCK" in the Test::Builder::plan subroutine, but because it's called at compile-time, the AGGTESTBLOCK hasn't finished parsing, so &plan cannot "last" out of a block it can't see).

And that ugly if/else "FAST_TESTS" block above? Now it looks like this:

# If it's not in a BEGIN block, it will run too late for
# the &run_the_tests call to see it.
BEGIN {
    plan $ENV{FAST_TESTS}
        ? (skip_all => 'FAST_TESTS environment variable set')
        : (tests => $num_tests);
}

All things considered, I think this has been a good day's work.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.