NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.
All the Perl that's Practical to Extract and Report
several minutes ago. We don't need no stinkin' tactical nukes.
(By the way, do you have change for 10 million people?) --lwall
Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.
CPAN? (Score:1)
What happens if you add CPAN tests to that count? And whatever is the Ruby equivalent.
Re:CPAN? (Score:2)
Tough to say. According to Tim's slides, if we include bundled libraries, we drop significantly to a 7 to 1 advantage for tests. However, there's no clear way I can see to design a reasonable metric for non-core tests. LOC/test is silly, particularly since core Ruby seems to generate terser code than core Perl for similar functionality. Tests per function point would be difficult to analyze without static analysis of function points and agreement on said analysis.
To be fair, even the core tests might be problematic if they bundle multiple assertions into a single "test". However, they'd still have to have an average of over 50 assertions per "test" and that would suggest that their tests weren't granular enough.
Also, assessing the quality of the tests is impossible. Even 100% coverage is a pretty silly metric with only unit tests and full path coverage testing is impossible.
More importantly, the CPAN tests would be almost useless even if we did have agreement. Class::Trait has tests to assert that we didn't fall afoul of the SUPER:: bug, so if core Perl had no test for that, this merely means that we test that certain unwanted behaviors don't impact our code, not that Perl doesn't have these behaviors.
Core tests are far, far, far more important than non-core tests. You need to know that the foundation you're building on is rock-solid and without core tests, it's tough to be certain of that, particularly since a well-designed, stable test suite can almost be viewed as a contract, something Ruby currently lacks.
I know there are Ruby developers who are very frustrated with this (Daniel Berger is one, if I recall correctly) and have complained in their blogs that the core Ruby developers aren't really taking this as seriously as they should.
50 to 1 test coverage kind of makes that point without further argument, but it would be nice to know if such an important technology is coming up to speed in this area.
Reply to This
Parent
Re: (Score:1)
Was thinking also about things like CPAN Testers. 3,592,446 public test reports (as of last night) vs ???.
At least as evidence of a "testing culture". :-)
-- dagolden
Re: (Score:2)
Heh :)
I'm unaware of any other language community having anything remotely comparable.
Re: (Score:1)
... yet.
Many of the repositories are within a few years of us, so it's important to not get too proud about what we have, and keep pushing ahead.
Re: (Score:1)
First, those language communities have to get serious about including systematic automated testing throughout their repositories. I hope they do, but I haven't seen it.
Re: (Score:1)
That doesn’t really affect Adam’s point. Most of the infrastructure we have was built by individual efforts, so other communities with enough sufficiently determined members could clearly duplicate it in a reasonable time frame. (My gut estimate is about 3 years all told.)
Even if other communities show no signs of such an effort and seem to be parsecs behind, that doesn’t mean we shouldn’t still keep pushing forward as if the would-be competition were breathing down our necks.
Even if
Re: (Score:1)
I don't believe it is. I want to see a comprehensive and freely distributable and reusable test suite for every widely used language. That's the only way we can port them to Parrot, for example. (There's also the code quality issue. Can I assume everyone understands that by now?)
Re: (Score:1)
Re: (Score:1)
Oh, I’m not talking about the size of the language core test suite. I’m talking about the things Adam appeared to be referring to, by virtue of the comment which he replied to – namely the CPAN Testers and all the attendant infrastructure.