Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

chromatic (983)

chromatic
  (email not shown publicly)
http://wgz.org/chromatic/
Blog Information [technorati.com] Profile for chr0matic [technorati.com]

Journal of chromatic (983)

Tuesday January 11, 2005
09:43 PM

Quieter Test Output

[ #22669 ]

I never really understood the point of prove ; I write my tests as normal Perl programs, launchable from the command line. perl t/testname.t does most of what I want to do.

With more than 24 tests in a file, though, I often promised myself to write or find someday a variant test harness that ignored successes and reported only failures or an all clear message.

I looked at the code of prove today to find where to add it, but again, it doesn't seem to do anything I really need and there was no way to add it. I moved on to Test::Harness::Straps. I don't know if anyone's ever actually used this module (and it's admittedly a bit of a mess inside, though the long-promised Test::Builder refactoring may make it easier to write), but it took 67 lines of code to do what I needed (and a little more) in a well-factored way.

I call it qtest:

#!/usr/bin/perl

use strict;
use warnings;

use Test::Harness::Straps;

my $strap = Test::Harness::Straps->new();

for my $file (@ARGV)
{
    next unless -f $file;

    my %results = $strap->analyze_file( $file );

    if ($results{passing})
    {
        report( sprintf('All (%d) tests passed in %s',  $results{seen}, $file));
    }
    elsif ($results{skip_all})
    {
        report( sprintf('All (%d) tests skipped in %s', $results{seen}, $file));
    }
    else
    {
        report( find_failures( $file, \%results ) );
    }
}

sub report
{
    my $message = shift;
    print "$message\n";
}

sub find_failures
{
    my ($file, $results) = @_;
    my $report           = create_header($file, @{$results}{qw( max seen ok )});
    my $count            = 0;

    for my $test ( @{ $results->{details} } )
    {
        $count++;
        next if $test->{ok};
        $report .= create_test_result( $count, @{ $test }{qw( name reason ) } );
    }

    return $report;
}

sub create_header
{
    my ($file, $expected, $seen, $passed) = @_;
    my $failed                            = $seen - $passed;
    return sprintf "File '%s'\nExpected %d / Seen %d / Okay %d / Failed %d\n",
        @_, $failed;
}

sub create_test_result
{
    my ($number, $name, $reason)   = @_;
    $name                          =~ s/^-\s*//;
    $reason                      ||= '';
    $reason                        = " ($reason)" if $reason;
    return sprintf "\tTest #%d: %s%s\n", $number, $name, $reason;
}

In practice, I may tweak the formatting somewhat. It might be nice to report skips and surprisingly-passing TODO tests, too. I also might want a flag to end after the first failure, to solve the problem of cascading failures rolling off of the screen. Still, this was a lot simpler than I thought and already makes my life easier.


List all Journal entries
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Quieter Test Output 8 Comments More | Login | Reply /

 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.

  • prove was based on a program that I wrote a long time ago to manage my tests better. I sent that off to Andy and it got warped into prove. Unfortunately, the one thing that I miss that I had code which would allow me to run sets of tests, based upon patterns on the command line. Thus, when working on stuff that touched "products", I might run a particular test program, or I might run all test programs with "prod" in the name. When I was done, I'd run the full test suite. It was very handy. Now I don'

    • I don't understand why you need prove to run sets of tests, unless it's for the nice summary at the end, which does seem like an advantage.

      (I just realized that that may be what you like as I started to type "my shell expands patterns just fine, so I don't need another program to do it for me". I think that's the conceptual part I didn't understand.)

      • I'm not sure about his reasoning... but I use it a lot.

        My entire test suite takes about 30 seconds (dual 2.8 GHz Xeon system)... and growing (since I only have about 10 percent coverage on three years of programming). That is long enough to get me out of the zone if I have to do it too much.

        So... I run prove, the setup routines, and the particular tests I'm focusing on. The setup routines take about 5 seconds and the individual tests run in about 2 seconds. Which, is short enough not to take me off task (
    • Really, prove was more based on smoke that I wrote at work. Why should you use prove? Here are some reasons! http://petdance.com/perl/use-prove-lt.pdf [petdance.com]
      --

      --
      xoa

  • You really want to look at Test::Verbose - it does exactly what prove should have been (via the "tv" command line) and has -q, -qq, and -qqq options to achieve exactly what you desire here. Moreover, it logs its full output to a file so you can scan that if you want more detail of what has happened.

    It's a shame that Test::Verbose is greatly overlooked. I find it absolutely indispensable.
  • I like to be able to run the tests for my modules without waiting to copy them into the staging area, which is pointless for a pure-Perl module. So I can simply cd into the t directory, then repeatedly say "prove -I../lib *.t", and alter that to a specific test or add -v. It's like "make test", but a dozen times more convenient.

    It's also a lot easier than typing the equivalent

    perl -MTest::Harness -e 'runtests(@ARGV)' *.t

    which I also had used in some of my early docs on testing. Thank goodness we hide

    --
    • Randal L. Schwartz
    • Stonehenge
Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.