Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Monday December 10, 2007
10:36 AM

More Stupid Testing Ideas - Generating TAP from TAP

[ #35076 ]

Test results currently look something like this:

t/foo.t................. ok
t/bar.t................. ok
t/baz.t................. 23/?
#   Failed test at t/baz.t line 9
# Looks like you failed 2 tests out of 23
t/baz.t................. Dubious, test ...

Why do we do this instead of outputting TAP (using the proposed YAML diagnostics)?

ok 1     - t/foo.t
ok 2     - t/bar.t
not ok 3 - t/baz.t
  ---
  failed:
    - 2
    - 11
  ...

And we could even add diagnostics for the non-failing tests. This could be an alternate output, but now instead of external tools having to try and parse our ad-hoc Test::Harness output, we could have an alternate machine read-able output that those tools could use. Now if only we had a useful way to read that output ...

Of course, this would be optional output, not the default. I think this could simplify a lot of "meta" testing issues we seem to have. Of course, you could say "but the new Test::Harness provides that information anyway!" The problem is that it's not easy to stream this data to a remote machine or persist it for later. Being able to persist it as TAP would be useful (TAP::Harness::Archive is a different attempt to solve a similar problem).

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • I'd wondered this myself in the past. It also occurs to me that it would be pretty keen to allow a verbose TAP-2-TAP harness to emit the contents of the individual tests as groups, so that your tests 1, 2, 3 could be expanded to three groups showing the results of the tests as a whole. If I recall correctly, groups are arbitrarily nestable, so a series of harnesses could run a very large test suite, producing a single comprehensive TAP document.

    It would also be pretty keen to attach some metadata to each
    --
    rjbs
  • I think this a neat idea worth pursuing further.