Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • Personally, I don't like having tests that rely on one another. I think that tests should stand on their own, so that they can be run on their own during development.

    See my thoughts on this at http://petdance.com/perl/automated-testing/ [petdance.com]

    --

    --
    xoa

    • I guess I wasn't clear. All of the tests do run on their own. Any of those tests programs can be separated from the others and it will work just fine. However, rather than do a bunch of white box unit testing (which is what we were doing) I have found that black box integration testing, while sacrificing some fine-grained control, gets me the results I need.

      The higher number tests don't depend on the lower number tests per se, but if a higher level test fails I immediately look for a failure of a lower level test and fix that first. While that does mean my tests appear to be more tightly coupled than they should, I can still run tests individually and everything should be fine.

      The reason I've started doing this is because we were running a lot of white box unit tests with systems that have multiple tiers before you drill down to the database. I might have something like this:

      presentation -> dispatch -> handlers -> business objects -> persistence layer -> database

      We might have a small tweak in the database or persistence layer, but by writing unit tests that were isolated from those layers, the unit tests would pass like a charm and we'd miss the bugs that propagated through the system. But what happens with integration tests that fail at the dispatch layer? Maybe the bug is in that layer, maybe it's in one of the four layers below it. This drives up debugging time considerably! However, by numbering the layers and fixing test failures in the lower tiers first, we get part of the unit test benefit of narrowing down where a bug really is, but we also get the integration test benefit of knowing if we have API errors between tiers. It's been a very useful compromise.

      If you have a rebuttal, I'm all ears :)