Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Journal of jjore (6662)

Tuesday April 10, 2007
12:30 PM

Test::Pod::Coverage is harmful

[ #32958 ]

Its preventing File::Slurp, a prerequisite of Jifty, from installing cleanly just because its detecting some imported POSIX constants. WTF?!

This kind of test should be run only when an environment variable like AUTHOR is true, not for every user. I didn't want my Jifty install to fail just because some deep prerequisite was being too picky about making sure its own documentation covered everything it imported into itself.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • (1) Upgrade to CPAN 1.90

    (2) configure distroprefs: 'o conf prefs_dir ...'

    (3) copy URI.File-Slurp.yml from the CPAN distribution to your prefs directory

    (4) Relax and never ever care again about File::Slurp or Pod::Coverage.

    • Er, no. That's a losing proposition. That just means the code remains broken on CPAN. Hopefully none of the people responsible for maintaining that code are using this to ignore failures.

      When I'm testing for what a "regular" user sees, a raw configuration is better.
    • It's not just a problem for a single person, but for every user.

      I recently outgrown Test::Pod::Coverage because Pod::Coverage is just really dumb. It seems to handle only the simple solution: a single package per file, and that package has to named after the file path. That just doesn't work for me anymore.
      • From the t/pod-coverage.t of my last module:

        #!perl -T
        use Test::More tests => 1;
        0 or eval 'use Test::Pod::Coverage';
        ok( 1, 'This file is just here as a sop for CPANTS' ) or all_pod_coverage_ok();
        # vim:ft=perl:
  • If your TPC is 1.04 or prior, upgrade it first to the latest version and it would be less harmful. The same thing can be said for every Test:: modules concerned.

    Besides, skipping pod coverage test with environmental variables (TEST_POD or AUTHOR or anything) is not a good idea, as it tends to be ignored even by module authors (especially those who copy the test from someone else's modules, and thus, don't know it should be enabled somehow). Actually there're modules that have pod coverage test that fail if
    • I have no problem upgrading my own installation to the latest stuff. It's my own dev stuff, y'know? What I'm testing now is that a plain, fresh, ordinary perl can still install stuff. If modules are going to require testing infrastructure that doesn't work properly until a high enough version then modules are going to have to include those "high enough" versions in their PREREQ_PM lists.

      Without proper PREREQ_PM, modules just fail to install and that sucks.
  • Reading some of the comments, it's nice to see more people are realising what a stupid methodology Test::Pod::Coverage has. I tried it once, was curious how it could do what it said without using PPI, looked at how it worked, and ran screaming.

    As for the AUTHOR flag, pending some newly created standard, the one you want to use is probably AUTOMATED_TESTING.

    The tests will fail in CPAN Testers, but pass when a user installs it.

    Here's the current version of 99_author.t from my repository (which is bundled with
    • No, AUTOMATED_TESTING is checking for the wrong thing. There's a distinction between things that should be run by the developers and things that should be run by the "mere" users.

      Smoke vs interactive is a different distinction. The QA thread that couldn't settle on a name centered around AUTHOR_TESTS, I think. I'm happy considering that variable standard-enough.
    • Reading some of the comments, it's nice to see more people are realising what a stupid methodology Test::Pod::Coverage has.

      I'm starting to think that it's stupid because the Pod::Coverage metric is stupid.

      • Pod coverage is not a bad metric iff you use the right one. P::C is a bit too arbitrary about what symbols need coverage. Requiring the code to execute to check the docs is plenty scary too. And slow.

        I've found that looking for a /=head2 $1\b/ to match every /^sub (\w+)/ actually catches places where I renamed the sub and missed the pod. usefulness++, noise-- and all that. Yeah, there's no PPI involved there. Honestly, does it need to be? Not for *my* coding policy.

        Maybe somebody with a stricter polic