All the Perl that's Practical to Extract and Report
use Perl Log In
Alias (5735)
What is the longest Perl test suite you've ever seen?
ok( Perl::Dist::Strawberry->default_machine->run );
With this release of Strawberry, I've managed to find a reasonably safe way to write tests that generate distributions without wiping out your current installation.
One of these tests the default Perl::Dist::Machine for Strawberry.
A "distribution machine" takes the options for building a distribution and generates several variants of the same thing. In this case, the machine is handling the creation of the three 5.8.8, 5.10.0 and Portable variations of Strawberry.
For completeness, this process now has a test script of it's own.
The test takes around 4-5 hours to complete (each distribution takes about an hour and a half to build) (on top of the 4-5 hours for the three tests that do the single builds on their own).
It's so long that it's now pretty much pointless to run under $ENV{AUTOMATED_TESTING} because it's going to hammer the CPAN Testers hosts (it also generates around half a gig or so of temporary data).
Currently these distribution building tests only run under RELEASE_TESTING.
So my question to you...
Am I right in thinking this is the longest test on the CPAN?
And secondly, what's the longest (Perl) test suite you've ever seen?
For me, it's my current test suite at work, which is starting to head past an hour now.
because that's exactly how much difference there is. :-)
--Larry Wall in <10209@jpl-devvax.JPL.NASA.GOV>
Longest Suite (Score:2)
The longest suite I've worked with took about an hour and a half to complete. Regrettably, it was written with a bespoke (and broken) test harness.
Why does your current test suite take so long to run? There are numerous strategies which can generally be applied to mitigate long test suite run times (or are you not bothered by this?).
Re: (Score:1)
In Perl::Dist(::Strawberry) I build Perl from scratch, about 7 times.
Across all of them, I also install about 200+ CPAN modules...
And unfortunately, none of it knows when to test in parallel on it's own, so much of my the power of my quad-core machine is wasted.
At work, much of the extra work is necessary, because the application is ENORMOUS (250k source lines of code) with lots of lots of potential variation and state.
Before every single test script, we kill and rebuilt users from scratch. Some test direct
Re: (Score:1)
For the long setup time, have you considered creating a set of general test data that can be used for many of the tests?
If you can do that, then maybe you can cache the setup of it and avoid long processing time just to get to the correct state.
So the app code is used to move the app to the correct state only once per test suite run, or even outside of the test run. Once that is done, make a snapshot of it that can be restored quicker for each test.
This is something we haven't done yet, but I'm thinking of
And growing... (Score:1)
Smolder tells me that the test suite for our biggest project at $work took 1:28:55 the last time it ran. Granted our smoke box isn't as powerful as our production or even our development machines. But it's still in active development with lots of new features planned, so I expect that to get worse...
CPAN::Reporter! (Score:1)
grep -r 'wallclock secs' .|perl -nle '/^(\S+).*=\s(\d+\.\d+)\s+CPU/ and push @d, [$1,$2]; END { print join("\n", map { "$_->[1]\t$_->[0]" }sort { $b->[1]$a->[1] } @d) }'then this shows up as the top 5, sorted by CPU seconds:
1132.65 ctr1/done/pass.CPAN-Reporter-1.1702.i386-freebsd.6.1-release-p23.1223500100.952. rpt:Files=36,932.16