Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

petdance (2468)

petdance
  andy@petdance.com
http://www.perlbuzz.com/
AOL IM: petdance (Add Buddy, Send Message)
Yahoo! ID: petdance (Add User, Send Message)
Jabber: petdance@gmail.com
I'm Andy Lester, and I like to test stuff. I also write for the Perl Journal, and do tech edits on books. Sometimes I write code, too.

Journal of petdance (2468)

Sunday April 23, 2006
01:18 AM

Tests are a binary issue

[ #29414 ]
I just released Test::Harness 2.57_06. I removed failing test percentages from the output. It's always seemed silly to have shades of grey in what is a black/white issue: Did the tests pass or fail?

List all Journal entries
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Tests are a binary issue 8 Comments More | Login | Reply /

 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • I don't like the change myself. I'm bright enough to figure out that anything less than 100% pass is bad when developing.

    When using other peoples test suites seeing, for example, 99% ok tells me something very different from seeing 3% ok. For me the difference between "nearly there apart form this bit of functionality that I don't care about" and "completely f**ked" is useful. Yes I can figure it out from the test/pass numbers - but the percentage gives me a handy overview. Math is hard! :-)

    Not something I
  • An implied question when I do a test run is "Did I screw up something fundamental, or just some edge case?" Numbers help me get that answer.
  • I use the percentages to track progress, and it's something I can point customers to.

    Was there some other reason for this change?
    • Other than removing extra code, and not wanting to support the "correctness" of that percentage, no. I had a ticket in RT where 99.994% rendered as 100%, which was misleading. But mostly it was because it just doesn't make sense to talk about tests in a way other than pass/fail. The whole end-of-the-run reporting system is on my short list to overhaul anyway.
      --

      --
      xoa

  • Given that you are still reporting the number of failed tests, not displaying the fairly meaningless percentage is no loss at all.
Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.