Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Saturday September 13, 2008
05:36 AM

Popular Test Modules (updated)

[ #37433 ]

A while ago I needed to know which test modules were the most popular so I could bundle them in Test::Most. I was asked how I figured that out, so I figured I would post the code. I've updated the code to make it a bit more robust, but it's still heuristically based.

First, I need to extract the tests from my minicpan:

#!/usr/bin/perl

use strict;
use warnings;
use CPAN::Mini::Extract;
use Getopt::Long;

GetOptions(
    offline => \my $offline,
    force   => \my $force,
) or exit 1;

my $cpan = CPAN::Mini::Extract->new(
    remote         => 'http://ftp.osuosl.org/pub/CPAN/',
    local          => "$ENV{HOME}/code/minicpan",
    trace          => 1,
    extract        => '/Users/ovid/code/cpanextracted',
    extract_filter => sub { /\.(pm|t)$/ and !/\b(inc)\b/ },
    extract_check  => 1,
    extract_force  => $force,
    offline        => $offline,
);

my $changes = $cpan->run;

I'm also extracting .pm files, but they could be skipped if you don't need them. Then, I run the following, passing it the location of the extracted directory. ($ENV{HOME}/code/cpanextracted/authors/id/). This takes about five minutes to run on my machine.

Note that due to the generic nature of this code, if you have a large project, you could point this at your project directory (assuming it has .t tests) and find out what test modules you're using.

#!/usr/bin/perl

use strict;
use warnings;

use File::Find;
my $target_dir = shift or die "Usage: $0 path/to/modules";

my @tests;

my $time = scalar localtime;
print STDERR "Finding tests.  This may take a while [$time]\n";
find(
    {
        wanted => sub {
            if ( /\.t\z/) {
                push @tests => $File::Find::name;
                warn $File::Find::name if /test.pl\z/;
            }
        },
        nochdir => 0
    },
    $target_dir,
);

my %times_found;

my ( $current, $total ) = ( 1, scalar @tests );

$time = scalar localtime;
print STDERR "Beginning analysis of $total tests [$time]\n";

# test_re must be a star because they might 'use Test;'
my $test_re = qr/\bTest(?:::\w+)*\b/;
foreach my $test (@tests) {
    open my $fh, '<', $test
      or die "Cannot open ($test) for reading: $!";

    if ( !( $current % 1000 ) ) {
        my $time = scalar localtime;
        print STDERR "Processing $current out of $total test programs at [$time]\n";
    }
    $current++;
    while ( my $line = <$fh> ) {
        next if $line =~ /^\s*#/;    # skip comments
        if ( $line =~ /^\s*(?:use|require)\s+($test_re)/ ) {
            $times_found{$1} ||= 0;
            $times_found{$1}++;
        }
    }
    close $fh or die "Cannot close ($test): $!";
}

for my $module ( sort { $times_found{$b} <=> $times_found{$a} }
    keys %times_found )
{
    printf "%-70s  %6d\n", $module, $times_found{$module};
}

The results are a bit different then before because I've updated my regex to include "required" modules. Here's the new top 20, along with the number of times each was seen.

  1. 69396 - Test::More
  2. 10912 - Test
  3. 2314 - Test::Exception
  4. 962 - Test::Simple
  5. 610 - Test::Base
  6. 386 - Test::NoWarnings
  7. 308 - Test::Builder::Tester
  8. 254 - Test::Deep
  9. 223 - Test::Pod
  10. 217 - Test::Warn
  11. 213 - Test::Differences
  12. 187 - Test::MockObject
  13. 157 - Test::Pod::Coverage
  14. 131 - Test::Builder
  15. 118 - Test::WWW::Mechanize::Catalyst
  16. 113 - Test::XML
  17. 112 - Test::Block
  18. 110 - Test::Perl::Critic
  19. 107 - Test::Distribution
  20. 101 - Test::SQL::Translator

All told, I found 373 test modules used.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • I am currently evaluating use of Test::Class.

    Your mentioning of our violations of good coding practices in test suites at YAPC::Europe in Copenhagen, have made me think a lot about the general layout my test code.

    Test::Class seem to offer something a have been looking for, namely test reuse.

    But I do find it quite concerning that is does not appear on your list. Neither do I see Test::Unit, which I evaluated a long time ago - I went with Test::(More|Simple) at that time, but Test::Class does seem to combine

    • Test::Class is number 52 out of 373. It should be much higher. It's a fantastic module. There are bits and pieces which could be changed but out of the box it's great and I highly recommend it. The test organization, test inheritance and general test code reuse gives it top marks in my book.

      Test::Unit is basically abandonware and does not play well with Test::Builder. This means that if you want to use standard Perl testing modules, you can't. You'd have to port any functionality you need if you wan

      • Ah my bad I did not notice that it was a top-20.

        Well I ported a 100% covering test suite to Test::Class yesterday and it does feel good. Next step is to have to other applications reuse the test suite - I can hardly wait.

  • On the plane today I made this a feature of my BackPAN indexer. I have all the bits to take a list of distributions and look inside them, so I just needed a worker task to count test modules instead of doing all the other stuff. It mostly works now, and I've pushed it to github. It's not like you need to care about that because I'll do the work to make the reports. :)

    If you really wanted to, you could also look through the output files [perl.org] I've made previously and extract the modules used in each of the test fi

    • Thanks for doing this. That's great!

      The other day, Schwern mentioned on the Test::More development list the other day that CPANTs also does the "once per distribution" thing. Counting the test module once per distribution as opposed to once per test program is good, but offers a different use case. It shows how widespread the dependency is, but not how widespread the usage is (both numbers being valuable, of course).

      For example, if you have one distribution with five .t programs, if one uses Test::Pod an

  • That surprises me.... a lot....