Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • I wonder if your post today was inspired by what I wrote in reply [perl.org] to your last post or we're just on the same wavelength with this. :)

    At YAPC::NA this year I did a lightning talk on a web-testing framework I built for a client*. The main topic was just that the tests were all data-driven, and because of that I gained a pile of great functionality, not least of which was the ability to turn my test suite into a benchmarking suite seamlessly.

    By running via prove or directly, each script was a set of unit
    • I think Alias addressed most of my would-be response, so I'll limit my comments to:

      All of the efforts I've personally seen to turn unit tests into benchmarks ended up being more of a benchmark of perl's compiler than anything else.  Unit tests tend to focus on tiny portions of the overall code, and test just them; they tend to not do unnecessary looping.  As such, the naive 'turn them into benchmarks' puts the loop around them, and unfortunately either re-evals them, or re-forks and evals them.

      One

      • One of the other problems we have is that because our system is enormous, with many links to external systems and pre-compiled caches, it takes two to three orders of magnitude more time to set up for a functional test than it does to run the actual tests.

        I think any sufficiently robust solution is going to need a way to factor that out, and this is a problem for the test script overloading approach.

        The test scripts would need a way to isolate the code you are interested in from the code you are running to set up for the actual test code.