Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • by Mutant321 (8646) on 2008.07.21 5:05 (#63971)

    I agree that you should try your hardest to at least have *some* sort of regression testing in place.

    But the problem with the above approach is that it's fragile. I know, because one of the codebases I work with has a *lot* of tests like that. And they're very noisy. So noisy that they tend to get ignored. And most cases where a test complains, it's some environmental or transient issue, or a false positive (i.e. someone has made a change that breaks the tests but hasn't introduced a bug).

    This is compounded by the fact that running these tests can be very slow, so it's usually only done nightly. You might run a few of the tests around the code you've changed, but the ones you've broken aren't obvious ones to check. Tracking down the issue can often take a long time.

    Not all of that is necessarily a symptom of using the approach you suggest (there are a lot of other issues with the test framework we're using), but I think you perhaps underestimate how hard it can be to get it right. There are some places in the codebase where the tests are worse than useless. On top of that, less senior developers are completely put off the idea of testing, since it's such a hassle for them.

    I tend to think you're better off in the long run refactoring and writing unit tests as you go.

    • Not all of that is necessarily a symptom of using the approach you suggest (there are a lot of other issues with the test framework we're using), but I think you perhaps underestimate how hard it can be to get it right.

      I can assure you that I've been down this road many, many times. For most companies I start with, I almost immediately find out that if they have a test suite, it's usually broken or limited in very fundamental ways. If they don't have a test suite, they always have a wide variety of excuses, few of which really hold water.

      You are correct that the approach I outline can be problematic at times, but the advantage that such integration tests give you that unit tests do not is twofold:

      1. You can write the te
      • Yeah, I think you're right, it's about a commitment to testing. Using integration tests is fine as an interim solution, but it must be used as a stepping stone to writing a real test suite (which to my mind is based primarily on unit tests). Otherwise you don't gain a lot.

        Of course, it's easy for the PHB to say "we've already got tests, don't we?" But that's another issue, really.