Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

schwern (1528)

schwern
  (email not shown publicly)
http://schwern.net/
AOL IM: MichaelSchwern (Add Buddy, Send Message)
Jabber: schwern@gmail.com

Schwern can destroy CPAN at his whim.

Journal of schwern (1528)

Thursday September 09, 2010
03:33 AM

Finally, some Test::Builder2 examples!

For my PDX.pm presentation tonight on Test::Builder2 I threw together some quick examples of some of its killer features, in particular demonstrating changing how Test::Builder2 behaves using method modifiers and applying object roles.

First, demonstrating end-of-assert actions, there's die on fail but even cooler is DEBUG on fail! That's right, run your test in the debugger and have it automatically set a breakpoint on a failure. How cool is that?

I'm sure somebody with better debugger foo than I can make it even cooler and stop at the top of the assert stack rather than inside DebugOnFail.

The second is reimplementing Test::NoWarnings safely. TB2::NoWarnings demonstrates hooking into the start and end of the test as well as safely altering the number of tests planned by trapping the call to set_plan.

You can safely use them all together, though its a crap shoot if DebugOnFail or DieOnFail will trigger first.

While roles and method modifiers are relatively new to the Perl community, using them in lieu of designing my own event system for TB2 has two great advantages. First, I didn't have to design and debug my own event system. :) Second, rather than having to learn the quirks of a one-off system, you learn the quirks of Mo[uo]se and then can apply that knowledge all over the place.

There's a pile of stuff to be done in TB2, a lot of them are fairly small and self contained. Have a look. Patches welcome.

03:15 AM

Test::Builder2 at 10k Feet

Here's a diagram of the "flow" of assert results through Test::Builder version 1.

                     .-------.
                     | foo.t |
                     '-------'
                         |
                         |
     .-------------.     |     .----------------.
     | Test::More  |<--------->| Test::Whatever |
     '-------------'           '----------------'
            |                           |
            |                           |
            |                           |
            |     .---------------.     |
            '---->| Test::Builder |<----'
                  '---------------'
                          |
                          v
                       .-----.
                       | TAP |
                       '-----'
                          |
                          v
                  .---------------.
                  | Test::Harness |
                  '---------------'

You write foo.t using Test::More and Test::Whatever. These both
use the same Test::Builder object. It spits out TAP which
Test::Harness converts into something human readable.

The big problem there is Test::Builder is monolithic. There's no
further breakdown of responsibilities. It only spits out TAP, and
only one version of TAP.

Here's what Test::Builder2 looks like:

                                  .-------.
                 .----------------| foo.t |-------------------.
                 |                '-------'                   |
                 |                    |                       |
                 |                    |                       |
                 v                    v                       v
          .------------.     .----------------.     .------------------.
          | Test::More |     | Test::Whatever |     | Test::NotUpdated |
          '------------'     '----------------'     '------------------'
                 |                    |                       |
                 |                    v                       v
                 |           .----------------.       .---------------.
                 '---------->| Test::Builder2 |<------| Test::Builder |
                             '----------------'       '---------------'
                                      |
                                      v
           .--------------.    .-------------.
           | TB2::History |<---| TB2::Result |
           '--------------'    '-------------'
                                      |
                                      |
    .--------------------------.      |       .---------------------.
    | TB2::Formatter::TAP::v13 |<-----'------>| TB2::Formatter::GUI |
    '--------------------------'              '---------------------'
                  |                                      |
                  v                                      |
  .-------------------------------.                      |
  | TB2::Formatter::Streamer::TAP |                      |
  '-------------------------------'                      |
                  |                                      |
                  v                                      |
               .-----.                                   |
               | TAP |                                   |
               '-----'                                   |
                  |                                      |
                  v                                      v
          .---------------.                     .-----------------.
          | Test::Harness |                     | Pretty Pictures |
          '---------------'                     '-----------------'

It starts out the same, foo.t uses a bunch of test modules
including Test::More and Test::Whatever using the same Test::Builder2
object, but it also uses Test::NotUpdated which is still using
Test::Builder. That's ok because Test::Builder has been rewritten in
terms of Test::Builder2 (more on that below).

Test::Builder2, rather than being a monolith, produces a
Test::Builder2::Result object for each assert run. This gets stored
in a Test::Builder2::History object for possible later use. It also
gets handed to a Test::Builder2::Formatter object, the default is
Test::Builder2::TAP::v13 which produces TAP version 13. This is fed
to a Streamer that prints it to STDOUT and STDERR which is read by
Test::Harness and made human readable.

Because Test::Builder2 is not monolithic, you can swap out parts. For
example, instead of outputting TAP it could instead hand results to a
formatter that produced a simple GUI representation, maybe a green
bar, or something that hooks into a larger GUI. Or maybe one that
produces JUnit XML.

Here's how Test::Builder and Test::Builder2 Relate.

        .-----.                                         .-----.
        | TB2 |                                         | TB1 |
        '-----'                                         '-----'
           |                                               |
           |                                               |
           |                                               |
           |                                               |
           v                                               v
    .-------------.        .--------------.         .-------------.
    | TB2::Result |------->| TB2::History |<--------| TB2::Result |
    '-------------'        '--------------'         '-------------'
           |                                               |
           |                                               |
           |                                               |
           |              .----------------.               |
           '------------->| TB2::Formatter |<--------------'
                          '----------------'
                                   |
                                   v
                              .--------.
                              | Output |
                              '--------'

Test::Builder and Test::Builder2 coordinate their actions by sharing
the same History and Formatter objects. If you call TB1->ok() it
produces a Result object which it hands to the History singleton and
the Formatter singleton. If you call TB2->ok() it produces a Result
object which it hands to the same History and Formatter objects.

This allows most of the Test::Builder code to remain the same while
still coordinating with Test::Builder2. It also allows radically
different builders to be made without Test::Builder2 dictating how
they're to work.

The downside is that roles applied to Test::Builder2 will not effect
Test::Builder. Because of this, Test::Builder may become more closely
coupled with Test::Builder2 in the future.

Diagrams by App::Asciio.

Friday August 27, 2010
11:08 PM

A month of Test::Builder2

I've had a grant open for Test::Builder2 for, oh god over two years now. Since I started it, Perl 6 has had a release! I think its the second oldest running dev grant.

I've cleared the decks of other responsibilities and can dedicate September to, if not finishing, then at least releasing something people can poke at. First alpha release was supposed to be "two weeks after the start" ha ha ha! oh god. The design has evolved and simplified greatly in the intervening two years, but its time to get something the hell out the door. At least a Test::Builder2 Star if you will.

There's critical components missing. There's no diagnostics, YAML or otherwise. The issues with nested asserts are still congealing. Plans are not enforced. Result objects are in the middle of being remodeled... again. But Test::Builder is using what parts of Test::Builder2 are usable. Multiple output formats and streams work. Asserts can be nested in the common, simple cases without having to fiddle with $Level. And you can hook into various events.

Step one is I'm going to seal up what's there, write docs where they're missing, and release something.

A release before October or the grant dies.

Wednesday August 18, 2010
05:33 PM

Alien::SVN - new release, new management

Those of you still stuck using Subversion will be happy to find a new release of Alien::SVN. It drags it forward to 1.6.12, doesn't do much else.

Also, Alien::SVN has finally found a new manager! From out of the blue comes Matthew Lanier with a patch and the will and a PAUSE ID. He'll be taking care of things from now on. Its his first CPAN module, be gentle. Godspeed, Matthew.

Monday August 09, 2010
09:31 PM

Test::Builder2::Design

In an effort to shed some light on what Test::Builder2 is about, I took a few hours and performed a brain dump about its goals and design. You can see the result in the new Test::Builder2::Design document.

The key design goals are 1) that it has to work, 2) that it has to work everywhere and 3) that it has to test everything. This throws out a lot of 98% solutions.

Friday July 30, 2010
07:49 PM

Method::Signatures returns! 5.12, func() and fast!

Chip submitted a minor performance patch to Method::Signatures today. That drove me to push out a new release making it friendly to 5.12 and adding func() for non methods!

        func hello(:$greeting = "Hello", :$place = "World") {
                print "$greeting, $place!\n";
        }

        hello( place => "Earth" );

For those who don't know, one of the neato features of Method::Signatures is that it can alias references to make working with references less of a trial:

        func popn(\@array, $howmany) {
                return splice @array, -$howmany;
        }

        my @stuff = (1,2,3,4,5);
        my @last_three = popn(\@stuff, 3); # 3,4,5
        print @last_three;

It does this with the amazing Data::Alias module. Unfortunately, 5.12 broke its black magic and its non-trivial to fix. Method::Signatures now makes Devel::Alias an optional dependency. If its available, it'll use it. Otherwise, no aliasing for you.

But that's ok, because perl5i makes working with references enjoyable. And while perl5i is adding its own simple signatures, they're forward compatible with Method::Signatures! They play together, so if you want perl5i and the full power of Method::Signatures you can have them.

        use perl5i::2;
        use Method::Signatures;

        func echo($message is ro) {
                say $message;
        }

Just make sure you load MS after perl5i. The last one loaded wins.

Finally, I was comparing Method::Signatures with MooseX::Method::Signatures and made a disturbing discovery. I always new MooseX::Method::Signatures would have a performance penalty, it does more checks than Method::Signatures, I just didn't realize how bad it was.

Here's comparing an empty signature: method foo() {}.

         Rate    MMS     MS    Std
MMS    3207/s     --  -100%  -100%
MS  1498875/s 46644%     --    -1%
Std 1508351/s 46940%     1%     --

That's showing MooseX::Method::Signatures is 450x slower than either Method::Signatures or a normal method call creaking out a mere 3500 method calls per second as compared to the 1.5 million it should be doing. And that's for a method with an empty signature!

To be clear, that's the speed of calling a method, not compiling them.

Here's one comparing a simple signature that requires a check, so MS can't optimize it away: method foo($arg!) { return $arg + 1 } That's a required positional argument.

         Rate    MMS     MS    Std
MMS    2928/s     --  -100%  -100%
MS   983127/s 33481%     --    -2%
Std 1005357/s 34240%     2%     --

3000 method calls instead of a million.

Now I'm the first to counter arguments bemoaning method call overhead. Usually it doesn't matter. Usually the extra cost of calling a method and checking arguments is insignificant compared to what that method actually does. And MooseX::Method::Signatures has features Method::Signatures does not, most significantly type checking. But my god! Three orders of magnitude of performance lost! And its not even using the extra MMS features. That's just too much.

Thursday July 29, 2010
02:24 PM

Perl 6 Is The Language Your Language Could Smell Like

Hello programmers. Look at your code, now at Perl 6, now back at your code, now back at Perl 6! Sadly, your code is not written in Perl 6. But if you use Rakudo Star then Perl 6 is the language your code could be written in!

Now Microsoft scented!

Monday July 12, 2010
07:06 PM

"def" or "func"?

perl5i 2.3.0_01 now has basic methods and subroutine signatures with code basically lifted straight from Method::Signatures::Simple. MooseX::Declare got me addicted, now I want them everywhere.

use perl5i::2;
 
def add($this, $that) {
    return $this + $that;
}
 
method new($class: %args) {
    return bless \%args, $class;
}
 
my $echo = def($arg) { return $arg };

Its alpha for two reasons. First, I don't have time right now to really thoroughly test it, but I really want it.

Second, overriding "sub" is hard. Its been done but its a bit twitchy. Defining a new keyword is easy(er). So what should that keyword be? I've come up with two that have good arguments. "def" and "func". Both are short. "def" has the benefit of being used by other programming languages a Perl programmer is likely to encounter and not hate (Python, Ruby, Scala, Groovy). "func" is nice because it pretty clearly means "function" whereas "define" is a bit ambiguous.

perl5i currently does both. Only one will survive in version 3 (the other will be deprecated). Before you comment on which is your favorite, try it for a little bit. I found a difference between what I thought I like and what I actually use.

Sunday June 27, 2010
02:16 PM

Where The Hell Is Test::Builder2?

My progress and communication about the Test::Builder2 grant has been nothing short of appalling. There is a sort of herky-jerky progress where I figure out a design problem, push the code forward, then remember a use-case that throws a wrench in the whole design and the whole thing comes to a screeching halt again.

At the QA hackathon we elegantly solved the problem of things like die-on-fail and Test::NoWarnings but then ran afoul of things like Test::Warn and Test::Exception which runs tests inside of tests but those aren't actually part of the test stack.

Confused? I'll post more about it another time. Point is, TB2 continues to move forward, its just that there's long periods of rumination between sprints of development. And I get distracted by other projects. At this rate I'll be collecting Social Security before I collect the second half of the grant. I really want TB2 to happen, but something decisive has to be done. I work best with hard deadlines, so the plan is to clear a month for working mostly on TB2. A lot of the wibbling is trying to come up with the most elegant solution, but I usually have a less than elegant way to solve it and move forward. If its between an elegant TB2 that doesn't exist and a less elegant TB2 that does, well, go with the one that exists. With the way my schedule is looking, that will probably be mid-August to mid-September. If that doesn't produce an alpha, then I'll kill the grant.

That doesn't mean TPF gets nothing for your money. Chunks of TB2 can be harvested to improve TB1. Specifically, the TB2 formatting and history objects. The TB2 formatter makes the guts of TB1 cleaner, and it also allows it to produce something other than TAP. Used together, history and formatter allows non-Test::Builder based test frameworks to work together with Test::Builder providing even more flexibility. This is already done in the TB2 branch.

Looking at the grant deliverables, most of it is done:

* Split up global shared Test resources into individual objects
    * The test counter
    * The output filehandles
    * The plan
 
* Allow hooks for global beginning and end of test functions.
    * Ensure multiple hooks "stack"
    * die on fail
    * debug on fail
 
* Hooks for global beginning and end of test actions
    * Example: A safer Test::NoWarnings
    * Example: Don't cleanup temp files on failure so they can be debugged
 
* Allow for test output other than TAP
 
* Allow another Test::Builder-like module to work in the same process
  as Test::Builder (for example, sharing the counter).
 
* Rewrite Test::Builder in terms of Test::Builder2.

Here's what's not complete:

* Split up localizable behaviors into objects
 
* Allow individual test modules to locally override Test::Builder2 behaviors
 
* Allow test modules to globally override Test::Builder2 behaviors
    * How the plan works

Since I'm not writing to the letter of the law, there's more than that to be done before release, but the project does move.

01:34 PM

The Post-YAPC Plan

I spent most of YAPC::NA mildly sick, sleep deprived and writing talks. Each of those things alone isn't so bad, but put all together meant I had time and energy enough to do my talks, discuss with people after, and that's about it. As a result, I was kind of dead in the head most of the time and didn't do a whole lot of interaction with people. I didn't feel like I got the most out of the one opportunity a year I get to hang out with huge gobs of Perl folk.

One of the things which I wanted to do at YAPC was get gitpan restarted. It can run right now, but the code is a mess and needs to be babied. It needs a rewrite. That rewrite was supposed to happen at YAPC but see above. I'm doing that now, using MooseX::Declare, perl5i and Path::Class just to mess around with them seriously. Also log4perl, which I'm finally learning a decade late. Its fun, far more pleasant than knocking it together without, once you learn to cope with Moose's idiosyncrasies. Better to learn the quirks of one complete system than nine incomplete ones.

That's what's absorbing my time right now. After that I want to add subroutine signatures to perl5i and the über file and directory objects. They were supposed to be in, at least as a prototype, by YAPC but that didn't work out. Using MooseX::Declare, Path::Class and perl5i together has me drooling for them.

Will Coleda, representing the TPF, found me at YAPC and mercifully did not break my legs. We hashed out a plan to make a last stab at Test::Builder2 before calling the grant done. That's not going to happen until August, I'll post about that later.

Oh, and I have a talk to do at OSCON about how the world is going to end in 2038 assuming 2012 doesn't claim the prize first. And a two part Git tutorial for Drupal programmers that I'm developing into a commercial class. And two paid clients to keep happy.

One thing I *don't* have to worry about is MakeMaker. Gird your loins, MakeMaker has provisionally been handed off to Matt Trout. Maybe I need to worry about it more...