Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • We use Test::Harness and its supporting tools to test everything. The initiation of the test isn't automatic, but everything else is.

    We have a script called smoke that

    • Runs *.t files
    • Runs *.phpt files, which are PHP versions of *.t files for PHP libraries, and that emits the same style output
    • Syntax checks all *.pl, *.pm and *.php files

    All this is done by creating my own subclass of Test::Harness::Straps which can handle all those other non-*.t files.

    Also, some of the test files are meta-test files

    --

    --
    xoa

    • Cool stuff. What I'm trying to sort out for myself is if there are additional benefits to be found from doing a build on a nightly basis. That is, doing a cvs checkout into a fresh directory, firing up an Apache process which points to that directory, running the automated tests, then (assuming everything passes) tar'ing up the directory and storing it away.

      The tests that you are doing seem to capture a lot of the benefits of a nightly build:
      • Syntax checking of all of your scripts is the equivalent of checking to see that they compile correctly.
      • Smoke testing by running unit and customized tests
      • Forcing programmers to write tests, since modules without a .t file will break the smoke test
      I guess what I'm wondering is whether there are additional benefits that could be achieved by pulling the sources out of cvs and tar'ing them up. There seem like there might be two benefits:
      • If this is the core of your release system (i.e. you untar the resulting tarball on your production server(s) and kick the web server), you have tested that your release system works.
      • You have separated the decision to release functionality from the decision to commit changes to source code.
      That last one may or may not make sense. Here's the problem that I'm concerned about: One of member of the team starts to update some code on existing production functionality (call it project A). The project isn't top priority, so they commit their (incomplete) project A code to cvs, and begin work on project B. Now, suppose that we had a nightly (or even weekly) automated release cycle which simply performs a cvs update from the staging server to the production server. We have now launched unfinished code to the production server. Even without an automated cycle, there's a problem -- you need an automated way to check whether committed files are ready for production. Doing a nightly build does not solve this problem -- but it ensures that programmers will be comfortable commiting unfinished code, since committed code is not automatically a candidate for release.