MarkyMoon's Friends' Journals

Finally, some Test::Builder2 examples!

schwern — 2010-09-09T08:33:42+00:00

For my PDX.pm presentation tonight on Test::Builder2 I threw together some quick examples of some of its killer features, in particular demonstrating changing how Test::Builder2 behaves using method modifiers and applying object roles.

First, demonstrating end-of-assert actions, there's die on fail but even cooler is DEBUG on fail! That's right, run your test in the debugger and have it automatically set a breakpoint on a failure. How cool is that?

I'm sure somebody with better debugger foo than I can make it even cooler and stop at the top of the assert stack rather than inside DebugOnFail.

The second is reimplementing Test::NoWarnings safely. TB2::NoWarnings demonstrates hooking into the start and end of the test as well as safely altering the number of tests planned by trapping the call to set_plan.

You can safely use them all together, though its a crap shoot if DebugOnFail or DieOnFail will trigger first.

While roles and method modifiers are relatively new to the Perl community, using them in lieu of designing my own event system for TB2 has two great advantages. First, I didn't have to design and debug my own event system. :) Second, rather than having to learn the quirks of a one-off system, you learn the quirks of Mo[uo]se and then can apply that knowledge all over the place.

There's a pile of stuff to be done in TB2, a lot of them are fairly small and self contained. Have a look. Patches welcome.

Test::Builder2 at 10k Feet

schwern — 2010-09-09T08:15:22+00:00

Here's a diagram of the "flow" of assert results through Test::Builder version 1.

.-------. | foo.t | '-------' | | .-------------. | .----------------. | Test::More |<--------->| Test::Whatever | '-------------' '----------------' | | | | | | | .---------------. | '---->| Test::Builder |<----' '---------------' | v .-----. | TAP | '-----' | v .---------------. | Test::Harness | '---------------'

You write foo.t using Test::More and Test::Whatever. These both
use the same Test::Builder object. It spits out TAP which
Test::Harness converts into something human readable.

The big problem there is Test::Builder is monolithic. There's no
further breakdown of responsibilities. It only spits out TAP, and
only one version of TAP.

Here's what Test::Builder2 looks like:

.-------. .----------------| foo.t |-------------------. | '-------' | | | | | | | v v v .------------. .----------------. .------------------. | Test::More | | Test::Whatever | | Test::NotUpdated | '------------' '----------------' '------------------' | | | | v v | .----------------. .---------------. '---------->| Test::Builder2 |<------| Test::Builder | '----------------' '---------------' | v .--------------. .-------------. | TB2::History |<---| TB2::Result | '--------------' '-------------' | | .--------------------------. | .---------------------. | TB2::Formatter::TAP::v13 |<-----'------>| TB2::Formatter::GUI | '--------------------------' '---------------------' | | v | .-------------------------------. | | TB2::Formatter::Streamer::TAP | | '-------------------------------' | | | v | .-----. | | TAP | | '-----' | | | v v .---------------. .-----------------. | Test::Harness | | Pretty Pictures | '---------------' '-----------------'

It starts out the same, foo.t uses a bunch of test modules
including Test::More and Test::Whatever using the same Test::Builder2
object, but it also uses Test::NotUpdated which is still using
Test::Builder. That's ok because Test::Builder has been rewritten in
terms of Test::Builder2 (more on that below).

Test::Builder2, rather than being a monolith, produces a
Test::Builder2::Result object for each assert run. This gets stored
in a Test::Builder2::History object for possible later use. It also
gets handed to a Test::Builder2::Formatter object, the default is
Test::Builder2::TAP::v13 which produces TAP version 13. This is fed
to a Streamer that prints it to STDOUT and STDERR which is read by
Test::Harness and made human readable.

Because Test::Builder2 is not monolithic, you can swap out parts. For
example, instead of outputting TAP it could instead hand results to a
formatter that produced a simple GUI representation, maybe a green
bar, or something that hooks into a larger GUI. Or maybe one that
produces JUnit XML.

Here's how Test::Builder and Test::Builder2 Relate.

.-----. .-----. | TB2 | | TB1 | '-----' '-----' | | | | | | | | v v .-------------. .--------------. .-------------. | TB2::Result |------->| TB2::History |<--------| TB2::Result | '-------------' '--------------' '-------------' | | | | | | | .----------------. | '------------->| TB2::Formatter |<--------------' '----------------' | v .--------. | Output | '--------'

Test::Builder and Test::Builder2 coordinate their actions by sharing
the same History and Formatter objects. If you call TB1->ok() it
produces a Result object which it hands to the History singleton and
the Formatter singleton. If you call TB2->ok() it produces a Result
object which it hands to the same History and Formatter objects.

This allows most of the Test::Builder code to remain the same while
still coordinating with Test::Builder2. It also allows radically
different builders to be made without Test::Builder2 dictating how
they're to work.

The downside is that roles applied to Test::Builder2 will not effect
Test::Builder. Because of this, Test::Builder may become more closely
coupled with Test::Builder2 in the future.

Diagrams by App::Asciio.

use Perl; Shutting Down Indefinitely

pudge — 2010-09-08T22:07:47+00:00

See here.

A month of Test::Builder2

schwern — 2010-08-28T04:08:39+00:00

I've had a grant open for Test::Builder2 for, oh god over two years now. Since I started it, Perl 6 has had a release! I think its the second oldest running dev grant.

I've cleared the decks of other responsibilities and can dedicate September to, if not finishing, then at least releasing something people can poke at. First alpha release was supposed to be "two weeks after the start" ha ha ha! oh god. The design has evolved and simplified greatly in the intervening two years, but its time to get something the hell out the door. At least a Test::Builder2 Star if you will.

There's critical components missing. There's no diagnostics, YAML or otherwise. The issues with nested asserts are still congealing. Plans are not enforced. Result objects are in the middle of being remodeled... again. But Test::Builder is using what parts of Test::Builder2 are usable. Multiple output formats and streams work. Asserts can be nested in the common, simple cases without having to fiddle with $Level. And you can hook into various events.

Step one is I'm going to seal up what's there, write docs where they're missing, and release something.

A release before October or the grant dies.

Alien::SVN - new release, new management

schwern — 2010-08-18T22:33:29+00:00

Those of you still stuck using Subversion will be happy to find a new release of Alien::SVN. It drags it forward to 1.6.12, doesn't do much else.

Also, Alien::SVN has finally found a new manager! From out of the blue comes Matthew Lanier with a patch and the will and a PAUSE ID. He'll be taking care of things from now on. Its his first CPAN module, be gentle. Godspeed, Matthew.

Comments re The Modern Perl book

Ron Savage — 2010-08-14T00:58:29+00:00

Hi Folks

OK, so modernperlbooks.com and Moveable Type have disabled my password for the 3rd time. I get the hint. I won't be back (there).

And no, I didn't forget it. I keep it in an encrypted file along with all the other passwords I've collected...

For the record, in the past I pretended I'd lost my password, and re-registered, but I'm sick of doing that.

As for the book, I have some comments, hoping chromatic stumbles across them :-):

1) Chapter 7: 'grovel': I don't think that's the correct usage of that word. Try 'fossick'.

2) Chapter 9: 'Alternately, use Moose and don't worry about the details' - under DOES(). Hmmmm.

This statement is far too brief. I think it should spell out exactly what a programmer needs to do in this situation when using Moose.

Cheers

use Perl;

pudge — 2010-08-11T23:34:11+00:00

I am no longer working for Slashdot/Geeknet as of September 30. I am actively seeking new employment. Unless you want to hire me, you don't need to care, unless you also care about use Perl;, because it has been generously hosted by Geeknet since I started the site over 10 years ago, shortly after I was hired by Andover.Net to work on Slashdot.

Long story short, I have not done much with the site in recent years, so my options at this point are to do nothing; migrate the site to a new server and keep it running as-is; or take the data and do something with it on a new site. Or something I haven't thought of.

I am hereby accepting proposals for what to do with use Perl;. In short, I would like to donate it to someone who will give it a good home. If you're interested, give me your best pitch.

Cross-posted on <pudge/*>.

Matt Trout, aka mst, is insane

pudge — 2010-08-11T16:41:54+00:00

Wow. I occasionally, but not too often, go into #perl. Very busy with family and life. So I go in today, and for no reason, mst bans me and tells me to not come back.

What's up with him being such an irrational dick?

Upcoming Events with Perl content

gabor — 2010-08-10T17:46:55+00:00

I have not posted here for ages but according to the recent Perl Survey it seems there are still a lot of people who prefer reading use.perl.org than blogs.perl.org or Iron Man

So just to let these readers to know, there are going to be a number of tech-events with Perl content where you could help out.

CPAN Testers 2.0: The death of via email is wrong.

jk2addict — 2010-08-10T02:49:45+00:00

I finally got around to installing perlbrew, 4 flavors of perl and updating some modules for recent perls. I've been sending a lot of email test reports lately.

Last night I got the email from bitbucket about CPAN Testers 2.0 and the death of sending reports via email with wiki links/instructions on how to setup .cpanreports/config.ini for a metabase id/transport via HTTP.

This is all well and good, but it feels wrong to me. I use CPAN::Mini. The point of which is partly to have CPAN when you're not online. The same reason I use git. With SMTP emails I could just queue up test reports in the local postifx and they would get delivered later when I'm online.

Now, that's is no more. I'm forced to be online to send HTTP reports. Seems like a bad idea. NOw, the only thing I can do is toggle off reporting when I'm offline, which really means I'll hate doing that and just turn off reporting all together.

Why did reports via email have to go away?

Test::Builder2::Design

schwern — 2010-08-10T02:31:05+00:00

In an effort to shed some light on what Test::Builder2 is about, I took a few hours and performed a brain dump about its goals and design. You can see the result in the new Test::Builder2::Design document.

The key design goals are 1) that it has to work, 2) that it has to work everywhere and 3) that it has to test everything. This throws out a lot of 98% solutions.

App::Office::CMS - A Canny, Microlight and Simple CMS

Ron Savage — 2010-08-03T08:15:12+00:00

Hi Folks

It's slow work, but I am making progress on App::Office::CMS.

It's based on CGI::Application and friends, to help sustain the microlight claim.

Which Javascript library, I hear you all cry!

YUI. You should have guessed.

Only a few weeks now...

Cheers
Ron

Method::Signatures returns! 5.12, func() and fast!

schwern — 2010-07-31T00:49:55+00:00

Chip submitted a minor performance patch to Method::Signatures today. That drove me to push out a new release making it friendly to 5.12 and adding func() for non methods!

func hello(:$greeting = "Hello", :$place = "World") {
print "$greeting, $place!\n";
}

hello( place => "Earth" );

For those who don't know, one of the neato features of Method::Signatures is that it can alias references to make working with references less of a trial:

func popn(\@array, $howmany) {
return splice @array, -$howmany;
}

my @stuff = (1,2,3,4,5);
my @last_three = popn(\@stuff, 3); # 3,4,5
print @last_three;

It does this with the amazing Data::Alias module. Unfortunately, 5.12 broke its black magic and its non-trivial to fix. Method::Signatures now makes Devel::Alias an optional dependency. If its available, it'll use it. Otherwise, no aliasing for you.

But that's ok, because perl5i makes working with references enjoyable. And while perl5i is adding its own simple signatures, they're forward compatible with Method::Signatures! They play together, so if you want perl5i and the full power of Method::Signatures you can have them.

use perl5i::2;
use Method::Signatures;

func echo($message is ro) {
say $message;
}

Just make sure you load MS after perl5i. The last one loaded wins.

Finally, I was comparing Method::Signatures with MooseX::Method::Signatures and made a disturbing discovery. I always new MooseX::Method::Signatures would have a performance penalty, it does more checks than Method::Signatures, I just didn't realize how bad it was.

Here's comparing an empty signature: method foo() {}.

Rate MMS MS Std MMS 3207/s -- -100% -100% MS 1498875/s 46644% -- -1% Std 1508351/s 46940% 1% --

That's showing MooseX::Method::Signatures is 450x slower than either Method::Signatures or a normal method call creaking out a mere 3500 method calls per second as compared to the 1.5 million it should be doing. And that's for a method with an empty signature!

To be clear, that's the speed of calling a method, not compiling them.

Here's one comparing a simple signature that requires a check, so MS can't optimize it away: method foo($arg!) { return $arg + 1 } That's a required positional argument.

Rate MMS MS Std MMS 2928/s -- -100% -100% MS 983127/s 33481% -- -2% Std 1005357/s 34240% 2% --

3000 method calls instead of a million.

Now I'm the first to counter arguments bemoaning method call overhead. Usually it doesn't matter. Usually the extra cost of calling a method and checking arguments is insignificant compared to what that method actually does. And MooseX::Method::Signatures has features Method::Signatures does not, most significantly type checking. But my god! Three orders of magnitude of performance lost! And its not even using the extra MMS features. That's just too much.

Perl 6 Is The Language Your Language Could Smell Like

schwern — 2010-07-29T19:24:09+00:00

Hello programmers. Look at your code, now at Perl 6, now back at your code, now back at Perl 6! Sadly, your code is not written in Perl 6. But if you use Rakudo Star then Perl 6 is the language your code could be written in!

Now Microsoft scented!

"def" or "func"?

schwern — 2010-07-13T00:06:57+00:00

perl5i 2.3.0_01 now has basic methods and subroutine signatures with code basically lifted straight from Method::Signatures::Simple. MooseX::Declare got me addicted, now I want them everywhere.

use perl5i::2; def add($this, $that) { return $this + $that; } method new($class: %args) { return bless \%args, $class; } my $echo = def($arg) { return $arg };

Its alpha for two reasons. First, I don't have time right now to really thoroughly test it, but I really want it.

Second, overriding "sub" is hard. Its been done but its a bit twitchy. Defining a new keyword is easy(er). So what should that keyword be? I've come up with two that have good arguments. "def" and "func". Both are short. "def" has the benefit of being used by other programming languages a Perl programmer is likely to encounter and not hate (Python, Ruby, Scala, Groovy). "func" is nice because it pretty clearly means "function" whereas "define" is a bit ambiguous.

perl5i currently does both. Only one will survive in version 3 (the other will be deprecated). Before you comment on which is your favorite, try it for a little bit. I found a difference between what I thought I like and what I actually use.

Perl 6 Design Minutes for 30 June 2010

chromatic — 2010-07-03T08:13:30+00:00

The Perl 6 design team met by phone on 30 June 2010. Allison, Patrick, and chromatic attended.

Allison:

working on Parrot packages for Debian experimental
seems like a good idea to do that before the 2.6 supported release
there was also a request for Rakudo packages
not sure if I'm the best person to do it

Patrick:

I'm sure we should package Rakudo Star

Allison:

Debian had a packager for those, but I haven't looked at the packages
this'd be an early run of what we'll do with Rakudo Star

Patrick:

we're not quite ready for packaging that yet
maybe a couple of weeks
finished the List and Iterator types for the #30 release
adjusted Rakudo's Associative and Positional roles
much cleaner implementation now
that'll require a few small spec changes
redid Rakudo's container types
more robust
preparing for autovivification of hashes and arrays
expect to finish those in the next couple of days
there was no container model previously; the code was consequently crufty
lots of cleanup of incorrect assumptions
Rakudo lists are now properly lazy
comment syntax fixed
ROADMAP updated
fixed the meaning of Nil; it's defined, not undefined
added the sink prefix (?)
fixed setting of $!
started fixing bugs and closing tickets on Monday, did 15 or 20
mostly already fixed in the previous couple of weeks
looking at the implementation of the series operator
spec is self-contradictory or ambiguous or both
waiting for Larry's clarification
fixed a bug in $*ARGFILES
had a nice contribution of that implementation last week
that behavior works on any set of files, not just those on the command line
working on autoviv
have some regex backtracking bugs to fix
will work on closures after that
put together three new YAPC presentations
the Rakudo Star presentation will become a video cast or a blog post or both

worked on a slew of Parrot optimizations for Rakudo
have a few more to go
might have to create a Rakudo branch temporarily
will try to help merge the new GC
working on a metamodel for Parrot objects, informed by Perl 6 and Moose

Modern Perl: The (Draft) Book

chromatic — 2010-06-28T23:43:33+00:00

This took longer than I expected, but the draft of the Modern Perl book is available for review. I'm especially interested in hearing from people who don't consider themselves expert Perl 5 programmers. The goal of the book is to explain how Perl 5 works (and how to write Perl 5 effectively) to help novices become adepts.

Where The Hell Is Test::Builder2?

schwern — 2010-06-27T19:16:09+00:00

My progress and communication about the Test::Builder2 grant has been nothing short of appalling. There is a sort of herky-jerky progress where I figure out a design problem, push the code forward, then remember a use-case that throws a wrench in the whole design and the whole thing comes to a screeching halt again.

At the QA hackathon we elegantly solved the problem of things like die-on-fail and Test::NoWarnings but then ran afoul of things like Test::Warn and Test::Exception which runs tests inside of tests but those aren't actually part of the test stack.

Confused? I'll post more about it another time. Point is, TB2 continues to move forward, its just that there's long periods of rumination between sprints of development. And I get distracted by other projects. At this rate I'll be collecting Social Security before I collect the second half of the grant. I really want TB2 to happen, but something decisive has to be done. I work best with hard deadlines, so the plan is to clear a month for working mostly on TB2. A lot of the wibbling is trying to come up with the most elegant solution, but I usually have a less than elegant way to solve it and move forward. If its between an elegant TB2 that doesn't exist and a less elegant TB2 that does, well, go with the one that exists. With the way my schedule is looking, that will probably be mid-August to mid-September. If that doesn't produce an alpha, then I'll kill the grant.

That doesn't mean TPF gets nothing for your money. Chunks of TB2 can be harvested to improve TB1. Specifically, the TB2 formatting and history objects. The TB2 formatter makes the guts of TB1 cleaner, and it also allows it to produce something other than TAP. Used together, history and formatter allows non-Test::Builder based test frameworks to work together with Test::Builder providing even more flexibility. This is already done in the TB2 branch.

Looking at the grant deliverables, most of it is done:

* Split up global shared Test resources into individual objects * The test counter * The output filehandles * The plan * Allow hooks for global beginning and end of test functions. * Ensure multiple hooks "stack" * die on fail * debug on fail * Hooks for global beginning and end of test actions * Example: A safer Test::NoWarnings * Example: Don't cleanup temp files on failure so they can be debugged * Allow for test output other than TAP * Allow another Test::Builder-like module to work in the same process as Test::Builder (for example, sharing the counter). * Rewrite Test::Builder in terms of Test::Builder2.

Here's what's not complete:

* Split up localizable behaviors into objects * Allow individual test modules to locally override Test::Builder2 behaviors * Allow test modules to globally override Test::Builder2 behaviors * How the plan works

Since I'm not writing to the letter of the law, there's more than that to be done before release, but the project does move.

The Post-YAPC Plan

schwern — 2010-06-27T18:34:30+00:00

I spent most of YAPC::NA mildly sick, sleep deprived and writing talks. Each of those things alone isn't so bad, but put all together meant I had time and energy enough to do my talks, discuss with people after, and that's about it. As a result, I was kind of dead in the head most of the time and didn't do a whole lot of interaction with people. I didn't feel like I got the most out of the one opportunity a year I get to hang out with huge gobs of Perl folk.

One of the things which I wanted to do at YAPC was get gitpan restarted. It can run right now, but the code is a mess and needs to be babied. It needs a rewrite. That rewrite was supposed to happen at YAPC but see above. I'm doing that now, using MooseX::Declare, perl5i and Path::Class just to mess around with them seriously. Also log4perl, which I'm finally learning a decade late. Its fun, far more pleasant than knocking it together without, once you learn to cope with Moose's idiosyncrasies. Better to learn the quirks of one complete system than nine incomplete ones.

That's what's absorbing my time right now. After that I want to add subroutine signatures to perl5i and the über file and directory objects. They were supposed to be in, at least as a prototype, by YAPC but that didn't work out. Using MooseX::Declare, Path::Class and perl5i together has me drooling for them.

Will Coleda, representing the TPF, found me at YAPC and mercifully did not break my legs. We hashed out a plan to make a last stab at Test::Builder2 before calling the grant done. That's not going to happen until August, I'll post about that later.

Oh, and I have a talk to do at OSCON about how the world is going to end in 2038 assuming 2012 doesn't claim the prize first. And a two part Git tutorial for Drupal programmers that I'm developing into a commercial class. And two paid clients to keep happy.

One thing I *don't* have to worry about is MakeMaker. Gird your loins, MakeMaker has provisionally been handed off to Matt Trout. Maybe I need to worry about it more...

Perl 6 Design Minutes for 16 June 2010

chromatic — 2010-06-26T17:07:30+00:00

The Perl 6 design team met by phone on 16 June 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

documented TOP (again), and explained how parsing is initiated and how it actually works
series operator (...) now picks a monotonic function when using single characters as endpoints
STD can now catch duplicates involving protos as well as onlys
STD no longer advises removal of parens on spaceless sub() declaration
mostly advised sorear and pmichaud
Stefan is finishing the boostrap of the STD parser
also working on adding a parallel NFA and DFA engine
no, he doesn't want to generate all the states in advance
it works faster lazily

Allison:

working on chroot environments with something more secure than chroot
relevant to building Parrot packages
looking at some bugs for Will

Patrick:

Rakudo developers decided not to make extra special effort to make a June release of Rakudo Star
the calendar works against us
the new date for the release is July 29
we're I comfortable with hitting that target
we won't be happy with the results of moving heaven and earth to release in June
there are lots of advantages
one disadvantage is not having Rakudo Star at YAPC::NA
one big advantage is using the supported Parrot 2.6 release as the basis
I'll write a post outlining the plan in the next couple of days
otherwise working on lists and interators in Perl 6 and Rakudo
after deciding to make iterators immutable, Larry and I realized that solves many problems
everything works out as plain as day after that
very happy with that design
the incorrect assumptions of the old model were pervasive
replacing the old pieces is taking a while, which is no surprise
this approach feels right though
the new branch does things no previous version could do
slices work much better, for example
metaoperators work properly
map is lazy
slurpy arguments in lists are lazy by default
no weird binding or action at a distance problems
plenty of changes to Associative and Positional roles
those are now super clean and may be lazy
more features work
~30 failing tests (not test files, just tests) now, ~500 last night
most of the current failures are minor
will try to merge the branch before the release
replacing lots of ugly code with fewer lines of elegant code
Jonathan and others have worked on lots of other pieces
adding plenty of new features
looking forward to tomorrow's release

editing the Rakudo book
moving the Rakudo release date may let us have a printed book available about the same time
depends on how much there is left to write

Perl 6 Design Minutes for 09 June 2010

chromatic — 2010-06-24T12:24:33+00:00

The Perl 6 design team met by phone on 09 June 2010. Larry, Allison, Patrick, and chromatic attended.

Larry:

not much spec change this week
figured out a syntax for a regex block to return more than one cursor
based on gather/take
in STD hacking, continued to assist Stefan O'Rear in getting STD bootstrapped via viv
now that it's bootstrapped, we're refactoring things that make sense now
we're now starting to move bits of Cursor code from Perl 5 into Perl 6
refactoring the grammar for sanity of design
started upgrading STD to normal Perl 6 syntax where it previously catered to gimme5's limitations
for example, switched STD's old .<_from> and .<_pos> hash lookups to using .from and .pos accessors
started the prep work for moving EXPR out of STD to make it generally available to any grammar wanting operator precedence
in STD parsing, made Perl 5 $< detection have a longer token to avoid confusion with match variables
STD no longer attempts two-terms detection on infix_circumfix_meta_operator
STD now parses >>R~<< correctly, or at least dwimmily
STD doesn't complain about P5isms in printf formats like "%{$count}s"
STD was parsing /m and /s with the opposite semantics
termish now localizes $*MULTINESS in its scope so that inner declarations aren't accidentally multified
STD now carps about package Foo; as a Perl 5 construct

Allison:

talked to Chris Shiflett, a PHP developer, on someone from the PHP community to sit on the Parrot board
will be in the US for a few weeks

Patrick:

working on list simplification
had a couple of breakthrough ideas on Monday
working on the implementation now
worked out inversion lists for character class matching in regexes
will make them faster, especially with long ranges of character classes
fixed a half-dozen tickets in RT
fixed Rakudo hash constructors
fixed an intermittent bug with colon-pair signatures
two possible parses exist in STD, but we removed an unneeded one in Rakudo
fixed a bug with Parrot's exit opcode
NQP and PAST needed an update not to cheat with PASM constants
I fixed that too
Vasily added multisub and multimethod support to NQP, that was a big plus
fixed the ** quantifier in regexes to understand surrounding whitespace
regex engine tried to match beyond the end of a string, so I added guards for that
will work on lists furiously before the next release
I don't think it'll take long
closures are next, hope to have those in place by the weekend

released a new version of Pod::PseudoPod::LaTeX to support the various books in progress

Perl 6 Design Minutes for 02 June 2010

chromatic — 2010-06-22T01:12:29+00:00

The Perl 6 design team met by phone on 02 June 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

mostly, I supported sorear in bootstrapping STD to use viv instead of gimme5
his stage 2 and stage 3 now output identical Perl 5 versions of STD
produces a huge amount of warnings
appears to require Perl 5.12 at the moment
working on both of those
S03 refines hyper dwimminess to be more like APL, with modular semantics
S02 refines Blobs to simply be immutable Bufs, with similar generic characteristics
S02 now describes native blob types
implemented post-declaration checks for BEGIN and use, since those can't wait for end of file
STD no longer loses existing bindings when we go to a sublanguage
STD now uses $*GOAL variable only as informative, never as a "stopper"
instead, we create a <stopper> rule for $*GOAL if necessary
can check for that only, instead of that or $*GOAL
answering lots of questions on how STD and viv work besides that

Allison:

did a lot of research on graph color algorithms for register usage algorithms
will finish my finals on Monday

Will:

trying to herd the discussion of dynop libraries
a recent branch to close an old ticket broke a lot of assumptions
some bugs have become more visible because of these changes
hope to get that cleaned up this week

Allison:

I liked your suggestion of bringing back the getstderr and related opcodes

Will:

trying to resurrect Partcl
stuck on a TT #389 closing issue
not sure how to fix that, the way things are now

Patrick:

working on the iterator and list design
brainstorming the implementation
will implement somethine one way or another this week
people keep implementing workarounds for the current system
they'll bite us eventually
Moritz and I worked on making the regex engine returning real Perl 6 objects
that mostly works
exposes some places where lists don't work exactly right
the workarounds there made me replan the list and iterator implementation
answered some questions online
Jonathan added a better backtrace algorithm for Rakudo
reports Perl 6 source lines instead of PIR lines
I'll review his code
think I can borrow it for NQP for all HLLs
Jonathan reports that it was a lot easier in NQP than PIR

trying to answer a few Parrot design questions
looking at the continuation of design from Perl 1 - 4 to Perl 5 and Perl 6
hope to have coding time soon

The Environment under 5 web servers

Ron Savage — 2010-06-21T03:53:44+00:00

Hi Folks

A comparison of env vars provided by 5 web servers is available here.

Perl 6 Design Minutes for 26 May 2010

chromatic — 2010-06-20T19:40:02+00:00

The Perl 6 design team met by phone on 26 May 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

:() syntax is now always signature
we now use foofix:[...] as the general op form instead of foofix:(...)
refactored the sematics of :nth and :x
:nth() now only ever takes a monotonically increasing list
S03 now explains how "not-raising" works on != and ne
it now basically matches the intuitions of an English speaker via HOP definition of negate metaop
STD sometimes didn't require semi between statements
statement modifiers are expression terminators but not valid statement terminators
an unexpected statement modifier word like if could terminate one statement and start another
fixed up backslashes in character classes to allow \s etc and reject \u etc
STD was accidentally using the same lexpad for different multis
Cursor now treats :() on name extension as a signature always, never as a categorical
we shouldn't introduce the stopper for circumfix until we're in the circumfix, or we can't use the same char on both ends
placeholder messages error messages are now much more informative and correct
we now disallow use of placeholder after same variable has been used as a non-placeholder, even for an outer reference
renamed add_macro (which it doesn't) to add_categorical (which it does)
participating frequently in discussions on semantics both on irc and p6l
working closely with sorear++ as he brings viv closer to bootstrapping, yay!
soon can bootstrap past gimme5

Allison:

worked on Pynie this week in my limited spare time
one goal is to generate the parser directly from the Python grammar
wrote a small, lightweight PEG parser which generates a match tree from the Python 3 grammar
can generate a lexer directly
right now it creates a parse tree
looks similar to the match nodes of NQP-rx
dumps out a tree to the PIR parser
working on PaFo elections for next year, but trying to delegate those
will have more time after June 7

Will:

working on Perl 6 advent tests
many more people are doing more work than me
liasing with Rakudo folks for any important Parrot bugs before the Rakudo Star release
my current direction there is "don't break anything"

Patrick:

sorear added hash flattening to NQP
lots of work on closures in PAST and NQP
they properly clone
fixes some lexical problems
need to get that to work in Rakudo
that's tougher; Rakudo has to wrap Parrot subs
wrapper object needs cloning as well, along with its attributes
we'll add a new PAST node type to help
that node understands contexts
essentially a way to add void context optimizations to your AST
that solves many problems in Rakudo beyond closures
added a setting into NQP along with its test suite
not automatically loaded, but available
contains standard hash and array methods
Parrot's ops2c project uses those
other people can update and enhance that setting as necessary
NQP also has the ability to parse type names
NQP doesn't do anything with them yet
eventually they'll allow the use of multis
cleaning up some NQP bugs regarding lexicals and package storage of subs
Bruce Keeler enabled variable interpolations in regexes
working on some refactorings to simplify that approach
works in NQP and Rakudo now
that's a feature we've never had before
Rakudo's REPL now works better, thanks to sorear
HLLCompiler now written more in NQP as part of that
NQP now can do eval
NQP remembers lexicals in interactive mode now
adding that to Rakudo is more complex
working on that
pleased with the progress on #perl6

reviewing long term plans for GC and Lorito
should have more time free soon

Perl 6 Design Minutes for 19 May 2010

chromatic — 2010-06-16T21:38:09+00:00

The Perl 6 design team met by phone on 19 May 2010. Larry, Will, and chromatic attended. Patrick added his notes later.

Larry:

S03 makes more explicit that doctrine that ~~ topicalizes, and removes smartmatch table fossils that automatically fall out from that
S05 renames 'accent' to 'mark' for better Unicode conformance
:a and :aa changed to :m and :mm
S05 disrequires retroactive semantics on :samecase and :samemark
the method form must now explicitly add case or mark modifiers to the pattern
regularized mm// to ms// to avoid confusion with new :m ignoremark option
STD now does a bit better at diagnosing bogus ??!! constructs of various sorts
STD now correctly adds operators to symbol tables as subs
CORE.setting now has protos of all the operators so they can be recognized as subs too
Cursor now canonicalize operator names in the symbol table
btw, not quite like specced
STD now reads user's mind on 'Str $toto' to intuit missing declarator
STD now properly diagnoses a typename between routine declarator and sub name

Will:

working on code for Carl Masak, trying to get his poker code example running on Rakudo
both fun and frustrating
some stuff doesn't quite work yet
going through the Advent examples
adding them to spectests
make sure we won't regress on such public examples
other people are helping with that now

will get back to editing the Rakudo book soon
hope to have it in print by YAPC, but no guarantee

Patrick:

fixed closures in NQP, as a precursor for fixing them in Rakudo
worked with sorear on REPL in Rakudo and PCT in general
ported the NQP "standard library" done by japhb++, bacek++, and many others into the nqp-rx repository and made it part of the standard build sequence for nqp and Parrot
decided we need a new "context sensitive" node type in PAST, will be used to create proper closures and to handle sink context
worked with bacek on adding better multimethod support to PAST and nqp-rx
discovered a problem with lexical subs in NQP being automatically entered into the package namespace (and some existing code relying on this behavior)
did some initial fixes to at least get things entered properly, but a complete fix may require a deprecation cycle
plan to review others' patches this week
plan to fix REPL, closures, and sink context in Rakudo (since those are currently large pain points)
plan to work on loops and iterators after that

Perl 6 Design Minutes for 12 May 2010

chromatic — 2010-06-16T01:47:02+00:00

The Perl 6 design team met by phone on 12 May 2010. Larry, Allison, Patrick, and Will attended.

Larry:

clarified usage of brackets around infixes
added various 128-bit types to the spec; we might make them arbitrarily extensible via role
at least LLVM could support this, even to non-powers-of-two sizes
modernized the paleolithic grammatical category description in S02
STD now uses double-quote rules for interpolating @foo[] into regex
STD now gives better message on 1__3
added the specced 128-bit types to CORE.setting
added minmax function to CORE.setting
implemented circumfix:«X Y» as grammar derivation
currently only allows a >> inside
now also recognizes foofix:("\x[face]") and foofix:("\c[YOUR CHARACTER HERE]") without actually evaluating
playing with factoring yaml out of gimme5, since viv is not likely to go that route.
mostly just answered a lot of questions on irc
egged people on about concurrency issues

Patrick:

thought on handling closures properly
have a solution, just need some time to implement
discussion on changes to CodeString
work on compiler toolkit to avoid CodeString, using StringBuilder instead where possible, in PCT, NQP, and rakudo. Pretty easy, no downstream projects block on a deprecation issue
after that, lists
also been answering questions on interactive mode (REPL) for rakudo et al. (the issue with losing lexicals)

Allison:

resolved the git conversation pretty well (for Parrot's repo migration)
worked on a pure PEG parser (following the paper), straight PIR, single day; now self-parsing. Interesting project, is lightweight. currently has memoization, but that might not be right for us because of backtracking. With some more effort, could probably handle EBNF form (useful for python)
could be setup for developer status for Debian which will improve our packaging status for Debian and Ubuntu

Will:

Parrot CodeString performance improvements
we're definitely faster in branch, but some feedback from pmichaud should help us clean up the API a bit as well, look for those to hit trunk in the next few days
Parrot makefile deps cleanup

Repost: Berkeley ICSI Internet/ISP connexion survey

Ron Savage — 2010-06-05T06:46:43+00:00

Hi Folks

Don't you just hate that?

After posting this the first time, I posted another article soon after, and due to a design fault on use.perl.org's home page, only the last shows up, under 'Recent Journals'.

I emailed pudge about it, so perhaps something will happen...

The Survey (requires Java)

New Scientist's explanation

BCVI 3.0 Hits CPAN

grantm — 2010-06-03T21:08:19+00:00

My big news today is that Version 3.0 of bcvi is now available on CPAN. If you have no idea what bcvi is then you'll be pleased to know I've written this article to answer all your questions - it even has pictures!

Way back in 2007 I knocked up a proof-of-concept implementation of a somewhat crazy idea. I used it for a while and found it useful so I shared it on PerlMonks to see if other people found it useful too. That was version 1.0.

Over time I fleshed out the core to make something more robust, more secure and more useful. But I never released it publically because it always seemed like the sort of tool that would be most useful to a person who hacked it into a form that suited them. I really had no idea how to release and maintain a tool that was meant to be hacked. So I kept it to myself and continued to find it useful. That was version 2.0.

Eventually I realised that what I needed was a solid core of infrastructure code and a plugin API that would allow people to hack the core without editing the script itself. Quite why it took me so long to realise this, I couldn't say. After all it's exactly what I did with SSHMenu and that has worked out very well.

The problem was that I didn't have a solid core of infrastructure code. What I had was a proof-of-concept script that had some of the rough edges knocked off and some new shiny bits bolted on. So earlier this year I set about rewriting bcvi in a more extensible way. Then to prove the new plugin API, I wrote a few simple plugins. Now it seems to be in a stable enough form for me to call it version 3.0.

And so it's released. There's even some documentation for the plugin API. It'll be interesting to what, if anything, people make of it.

Cross posted from my blog.

Volunteering thru http://goodcompany.com.au

Ron Savage — 2010-06-03T01:16:10+00:00

When I'm between contracts, I do some volunteer work thru
goodcompany, where small community groups post requests ('wishes') for people ('angels') to help ('grant the wish').

I only respond to Web/IT requests, but there are a range of categories into which requests can be put.

In Australia, there are between 600,000 and 700,000 such community groups, and many share the same sorts of problems, e.g. managing lists of contacts and donors.

So, I thought I'd report of on a few projects I've helped with, most of which are for charities:

o The East West Overseas Aid Foundation

They donate money to India for property and school classes, for example.

At first, I just helped with their list of contacts. But I'd seen a number of groups struggling with Excel for this, and ended up writing App::Office::Contacts and *::Donations, although they're not yet ready to use my code.

Now, we're working on bringing their web site up-to-date.

It'd be nice if there was a very-light-weight CMS in Perl we could host of a VPS.

On CPAN I found Miril, but sometimes it used MS-DOS slashes, (in config files) and I didn't feel like debugging it. And I've have to redesign the interface, to start with.

o Association for Children with a Disablity

I met a senior manager, who quickly got serious, making a small but revealing derogatory comment about my clothes!

We had a long talk anyway. She'd been quoted $16,000 by a local MS-oriented computer shop to do this project.

I offered to do it for %10 of that, or even for free. But no, a couple of weeks later she emailed saying my help wasn't wanted. Very strange.

o Windermere Child and Family Services

This was my first project thru goodcompany.

This charity had bought an Australian donation manager program - imaginatively called DonMan -, but had never used it.

So I wrote a bit of Perl to reformat several Excel spreadsheets into a CSV file acceptable to DonMan, and that was it.

o Court Network

This is the latest project. Attending court can be a bewildering experience for people who are most likely going to court due to some disaster in their lives anyway.

So, Court Network has a stable of about 450 volunteers who accompany the court user thru the process, both here in (the state of) Victoria, and in Queensland.

They had a paper-based system of matching up court users with volunteers, so I've written a classic web app to replace that.

When it goes live, the public will be able to submit details themselves, rather than go thru Court Network's office staff.

And the latter have extra features (Search, Update, Reports).

They're a non-technical group (a common situation), and so far haven't been able to tell me who hosts their web site (some other volunteer process), so we still have to deal with getting CGI and database features working.

And my contact there is going to get DreamWeaver training, so at least I won't be dealing with web site content.

o Life's Little Treasures Inc

This is a support group for women who've had premature babies.

Indeed, some premmies are still in hospital a year after birth, so their problems must be major.

In the end, this group chose Joomla, since they wanted to expand their on-line forum network's power, now that they're handing out brochures to all mums with new-borns, in some hospitals, not just to mums with premmies.

News flash: I checked their outstanding wish, and they're now asking for a WordPress export. I guess they just couldn't find a volunteer to help with Joomla. Or perhaps the Joomla side of things has been sorted out.

I'm thinking of getting East West (above) off TYPO3 and onto WordPress, but that's a coincidence (perhaps).

o Australian Karen Foundation

The Karen are an ethnic group in Myanmar, who are being genetically exterminated by the psychopaths who run Myanmar.

Some Karen have fled into a refugee camp in Thailand, where the Thai government gives them some support, and has just a few weeks ago even connected them to the internet, albeit only a couple of hours a day (I think).

Someone from a town near the camp visits occasionally to repair their PCs. They have about 10 machines, most of which work most of the time. The machines run MS Windows, but are not networked.

The A. Karen Foundation collects 2nd hand laptops from Australian companies and someone carries them over to Thailand once a year or so.

I've been collecting info on freeware directories, and on universities where free courses on learning English, maths, etc, can be downloaded.

I put the info into a TiddlyWiki - a brilliant 1-page-wiki manager.

o Lastly, Cottage by the Sea

This is a holiday/respite home for children, on the other side of Port Phillip Bay from where I live.

They are 120 years old this year, and are very slowly digitising their records (photos, annual reports, list of children who've attended).

I've offered to design a database to hold these records, and to write a search engine for people researching the historical record.

We'll see what happens.

o Summary

I've skipped a few places where I went and just had one meeting.

Nevertheless, you can see there is no way of predicting what sort of work comes up, or what it will entail, exactly.

Berkeley ICSI Internet/ISP connexion survey

Ron Savage — 2010-06-02T23:08:50+00:00

Hi Folks

The Survey (requires Java)

New Scientist's explanation