Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Friday January 11, 2008
10:13 AM

More Custom Testing Goodies

[ #35363 ]

Our latest custom Test::More looks something like this:

package Our::Test::More;

use strict;
use warnings;

use Hook::LexWrap;

# XXX don't use 'base' as it can override our signal handlers
use Test::Builder::Module;
our ( @ISA, @EXPORT );

use Test::More;
use Test::Differences;
use Test::Exception;

BEGIN {
    @ISA = qw(Test::Builder::Module);
    @EXPORT = (
        @Test::More::EXPORT,
        @Test::Differences::EXPORT,
        @Test::Exception::EXPORT,
        'explain',
    );

    if ( Test::Differences->VERSION <= 0.47 ) {

        # XXX There's a bug in Test::Differences 0.47 which attempts to render
        # an AoH in a cleaner 'table' format.
        # http://rt.cpan.org/Public/Bug/Display.html?id=29732
        no warnings 'redefine';
        *Test::Differences::_isnt_HASH_of_scalars = sub {
            return 1 if ref ne "HASH";
            return scalar grep ref, values %$_;
        };
    }
}

sub import {
    for my $i (0 .. $#_) {
        if ('fail' eq $_[$i]) {
            splice @_, $i, 1;
            wrap 'Test::Builder::ok', post => sub {
                if (![ $_[0]->summary ]->[-1]) {
                    die "Test failed.  Halting";
                }
            };
            last;
        }
    }

    # 'magic' goto to avoid updating the callstack
    goto &Test::Builder::Module::import;
}

sub explain {
    return unless $ENV{TEST_VERBOSE};
    Test::More::diag(@_);
}

1;

This does several things:

  1. Imports all test functions from Test::More, Test::Differences and Test::Exception.
  2. Adds an explain() function. It's like diag(), but produces no output unless in verbose mode.
  3. Adding 'fail' to the import list will cause the current test program to die on the first failure.
  4. Fixes an annoying bug in Test::Differences.

The three test modules included are ones that we use constantly, so it makes sense to include them.

The explain() function was needed because several developers really want diag() output for times when you want to read the test narrative more clearly. However, that output is useless when running the full test suite in non-verbose mode. It requires the latest version of Test::Harness in subversion because the $ENV{TEST_VERBOSE} functionality was accidentally left out (that would be my fault, I think).

The 'fail' argument to the import list is very handy when you have several hundred tests scrolling by and you hate scrolling back up through reams of junk to find the failure. Use it like this:

use Our::Test::More 'fail', 'no_plan';

BEGIN {
    use_ok 'Some::Module', ':all' or die;
}

ok defined &foobar, 'foobar() is in the house!';
throws_ok { foobar( 5 ) }
  'My::Exception',
  "... but he don't like no arguments";
eq_or_diff foobar(), foobar(),
  "... and he repeats himself a lot"'

ok defined &barfoo, 'barfoo() has somehow shown up';
foreach my $thing (barfoo()) {
    explain("Testing $thing");
    do_some_test($thing);
}
...

As you can see, you get more testing functions, the "explain()" stuff might spew out loads of junk, but you won't see it when you run the full test suite, and it will halt as soon as the first test fails.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • Ovid,

    Thanks for posting this. In 2005 I provided a patch for an explain() equivalent to Test::More, but it was never added:

    http://rt.cpan.org/Public/Bug/Display.html?id=14764 [cpan.org]

    It's such a sensible, helpful feature, I don't understand why it remains a feature which people continue to have to rediscover and patch themselves.

    • It's getting to the point where I'm thinking about just releasing a custom Test::More::Extra (or something) to handle this stuff. The problem is figuring out which test modules are the most common. Just choosing mine seems a bit cheeky.

  • I've tried to make this approach work. Here's the problem in a nutshell:

    $ perl -wle 'use OurMore "no_plan";  is 23, 42'
    not ok 1
    #   Failed test at -e line 1.
    #          got: '23'
    #     expected: '42'
    1..1
    # Looks like you failed 1 test of 1.

    $ perl -wle 'use OurMore "fail", "no_plan";  is 23, 42'
    not ok 1
    #   Failed test at /usr/local/perl/5.8.8/lib/Test/More.pm line 329.
    Test failed.  Halting at OurMore.pm line 44.
    1..1

  • I'm curious. Why do you prefer Test::Differences vs Test::Deep? Is it better diagnostics for failing tests, speed, personal preference, or something else altogether?
    • I just added Test::Deep. Consider the code:

      use Our::Test::More 'no_plan';
      my ( $foo, $bar ) = ( [qw/1 2 3 4 5/], [qw/1 2 trois 4 cinq/] );
      eq_or_diff $foo, $bar;
      cmp_deeply $foo, $bar;

      Consider the output:

      #   Failed test at test.t line 10.
      # +----+-----+----------+
      # | Elt|Got  |Expected  |
      # +----+-----+----------+
      # |   0|1    |1         |
      # |   1|2    |2         |
      # *   2|3    |