use Perl

I started out the day by looking through the RT queue for Rakudo. Two tickets were already dealt with, so I just closed those. Another was a bug report concerning assigning undef to typed variables. Doing:

my Int $x = undef;


Would give a type check failure. This is now resolved. Furthermore, if your type is a class name, then assigning undef at any point to it will result in it holding the protoobject for that class again. I also took a moment to post to Perl 6 Language to get some clarifications on what "my TypeName $x;" left $x being when $x was a role, subset type or a junction of types.

Last week I started getting grammars in place. I got so far as having the regex live in the correct namespaces, but that didn't make grammars at all class-like, which is how they should be. This week I set out to fix that. Grammars now get protoobjects too, which you can call .WHAT and so forth on. Furthermore, I got inheritance working and also smart-matching against a grammar, which runs it's TOP rule. Therefore you can now run the following example in Rakudo.

grammar Loads { regex Lots { \d+s } };
grammar Many is Loads { rule TOP { <Lots> of <Lots> } };
if "100s of 1000s" ~~ Many {
    say $/; # 100s of 1000s
}


Having closed four RT tickets so far, I took a look through there to see what else there was. There was one that did most of what was needed to implement the .perl method on Junctions (which you can call, in theory anyway, on anything to get a Perl representation of it). I did the required fixes and applied it, but realized in the process that we were missing .perl for any of the really fundamental types, so I added it for Num, Int and Str.

With that done, I spent a little time on the S12 tests. I added fudge directives to get one of the that failed to parse to do so, and added an extra test. I plan to add much more here and flesh out the tests quite a bit over time.

Turning back to the OO support, I did some updates to the grammar that both brought us closer to STD.pm - the official grammar - and added the ability to parse a range of extra things. The first grammar changes were related to method calling. Normally you call a method just with ".", but private methods are called with "!". Additionally, there are ways to call sets of methods with quantifiers (.?, .+ and .*, with meanings analogous to those in regexes). Finally, I added the ability to scope declarations of routines too, so we can now parse lexical subs and private methods. I stubbed in conditionals for all of these cases that throw unimplemented exceptions, so people didn't use them expecting that because they parse, they will also work.

So, now there's a bunch of stubs in there for another bunch of OO features and, if nobody beats me to it, I'll be filling some of those out on my next Rakudo day, or maybe before then if time allows (though I'm moving apartment - and country - over the coming week, so I'm not expecting to have much time). A big thanks to Vienna.pm for funding today's work.

Today's work has been a mixture of refactoring and clean-ups that had been on the want list for a while, but just hadn't happened, as well as making some new things work.

First, the initial work I did on types attached them to variables, but what we really needed was a more general way to attach properties. Therefore, there is now a hash of properties instead, where we can stash other stuff.

Next up, I had based pairs on the Parrot Pair PMC, though as Patrick pointed out it's so far off being right for Perl 6 (for example, it's mutable, the Perl 6 one isn't) that we might as well just have our own. Dropping the Parrot Pair PMC and doing that took me all of ten minutes of work, and we get the semantics of pairs a bit more correct too. So that's much cleaner now.

A few days back, dakkar sent in a bug report regarding inheritance. It was almost correct code, but didn't work on Rakudo, since initialization of parent attributes was not yet implemented. I've now implemented this, and I'll borrow the example from the bug report to demonstrate it.

class Foo {
    has $.x;
    method boo { say $.x }
}

class Bar is Foo {
    method set($v) { $.x = $v }
}

my Foo $u .= new(:x(5));
$u.boo;                    # 5

$u= Bar.new(Foo{ :x(12) }); # This is what now works
$u.boo;                    # 12
$u.set(9);
$u.boo;                    # 9


This is not some magical hacky syntax just to make constructors work; you can use it in the general case to associate some vivification data with a proto-object, which gives you a copy of it back with the data attached. It's a bit like currying the object instantiation. So after making the above work, it wasn't much more work to get the following working.

class Foo { has $.x }
my $foo42 = Foo{ :x(42) };
my $test = $foo42.new();
say $test.x; # 42


Note that the original Foo itself isn't changed. We'll have to revisit this again later, because the way I've done it now doesn't have the lazy semantics it's eventually meant to have. It makes the common use case work, though.

With some time spent on objects, I moved onto some improvements to regex stuff. The upshot of this is that you can now use grammar to group regexes into a namespace.

grammar Test {
    regex Load { \d+s };
    rule Loads { <Load> of <Load> };
}
if "100s of 1000s" ~~ Test::Loads { say "yes" }
yes


Note that this is just the start of grammars; inheritance doesn't yet work and you can't smart-match against them yet. It's a stop forward, though.

If you're an avid Rakudo follower, you'll have noted that regex, rule and token all (wrongly) did the same thing before today. I've fixed that too now (there was some behind the scenes work in being able to pass options to the compiler that will be useful elsewhere, as to users of the Parrot Compiler Toolkit in general). In a nutshell, token and rule don't backtrack, where as regex does, and additionally rule translates spaces to the rule, whereas normally they have no effect on the match.

# Demonstrating :ratchet semantics (rule like token here).
regex WillBT { a*a }
token WontBT { a*a }
if "aaa" ~~ WillBT { say "yes" } else { say "no" }
yes
if "aaa" ~~ WontBT { say "yes" } else { say "no" }
no

# Demonstrating :sigspace semantics.
regex Test1 { \d \d };
rule Test2 { \d \d };
if "12" ~~ Test1 { say "yes" } else { say "no" }
yes
if "1 2" ~~ Test1 { say "yes" } else { say "no" }
no
if "12" ~~ Test2 { say "yes" } else { say "no" }
no
if "1 2" ~~ Test2 { say "yes" } else { say "no" }
yes


So, that's what got done today. I'd like to thank Vienna.pm for funding this work, and hope you'll have fun playing with it, breaking it and reporting bugs. :-)

First of all, before I dig into what my recent Rakudo hackings have been, I'd like to thank Vienna.pm for funding me to work on Rakudo. I will be working one full day a week on Rakudo from now on, at least for the next three months and, hopefully, longer. Today is the first day I'm working under this funding, so I'll be posting again later on today about what I got done. This post is just to update you on little bits that I've been doing, but didn't get written up yet.

First of all, you can now use the .= operator.

class Foo { }
my Foo $x .= new();


Here we call the 'new' method on $x, which we know is of type Foo thanks to the type declaration, and assign what it returns - namely, an instance of Foo - to $x. I did initially put this in a while ago, but it was a tad buggy and I wanted to get those worked out before posting it. That's been done, so happy playing. (And note you can use it in places other than declarations too.)

Additionally, some very basic multi-method dispatch based upon types is now in place. You can only use class names, not constraints or role names at the moment for the types, and certainly not more complex types than that. However, it's a start and allows us to run the following example.

class Thing {}
class Rock is Thing {}
class Paper is Thing {}
class Scissors is Thing {}

multi sub defeats(Thing $t1, Thing $t2) { 0 };
multi sub defeats(Paper $t1, Rock $t2) { 1 };
multi sub defeats(Rock $t1, Scissors $t2) { 1 };
multi sub defeats(Scissors $t1, Paper $t2) { 1 };

my $paper = Paper.new;
my $rock = Rock.new;

say defeats($paper, $rock); # 1
say defeats($rock, $paper); # 0


Finally, I put in a small optimization to avoid having to run some runtime type-checks when we can statically determine they're not needed. This should help performance a little.

On the train over to Stockholm after the hackathon and on the plane back to Spain a day later, I implemented various cases of 'handles' (not all of them, since the wildcard ones are trickier - they will get done probably along with a whole load of other work on attributes that needs doing). The 'handles' trait verb is the thingy that lets you auto-generate methods that delegate to a methods on an attribute. Here's some examples of what you can do with this:

class Bar { method a { say "a" }; method b { say "b" } }
class Foo1 { has $x handles 'a' } # one method
my $test = Foo1.new(x => Bar.new()); $test.a()
a
class Foo2 { has $x handles <a b> } # several
my $test = Foo2.new(x => Bar.new()); $test.a(); $test.b()
a
b
class Foo3 { has $x handles :mya('a') } # rename one
my $test = Foo3.new(x => Bar.new()); $test.mya();
a
class Foo4 { has $x handles (:mya('a'), :myb('b')) } # rename many
my $test = Foo4.new(x => Bar.new()); $test.mya(); $test.myb();
a
b


So, that's another small piece of the Perl 6 implementation puzzle put into place.

I spent the weekend and most of Monday at the Oslo QA Hackathon. While I'm not a QA expert, it did provide me with time and a relative lack of distractions to get some work done on Rakudo. It was also nice seeing lots of Perl folks, some of whom I've not seen in quite a while. The Hackathon has had a very productive atmosphere with lots getting done; in this post I'll describe some of my Rakudo exploits.

The biggest thing I have been working on is type annotations. This now works with both variable declarations and in the signatures of subroutines and methods, although writing them in a multi-method doesn't lead to any type-based MMD just yet. Attributes of classes can't have types yet either. There are some quirks with various built-in types (including Str and Code types), though many others work fine (Int, Num, Pair, etc). With your own classes, roles and subset types, it should work just fine.

So, for example, if you declare a variable as having type Int, then try and assign a string to it, you'll get a type check failure (we'll get a nicer error message in the end).

my Int $x = 42; # this is ok
my Int $x = "hello"; # this is not
Type check failed


Here's a class example.

class Foo { }
class Bar is Foo { }
class Baz { }
my Foo $x; $x = Foo.new(); # ok, Foo is a Foo
my Foo $x; $x = Bar.new(); # ok, Bar is a Foo
my Foo $x; $x = Baz.new(); # not OK
Type check failed
my Bar $x; $x = Foo.new(); # not OK; Foo is not a Bar
Type check failed


Refinement types also work with this.

subset EvenInt of Int where { $_ % 2 == 0 };
my EvenInt $x = 4; say $x;
4
my EvenInt $x = 4; say $x; $x = 3; say $x;
4
Type check failed


You can do all of this with parameters too, as well as declaring an anonymous refinement.

sub Test(Int where { 0 < $_ <= 100 } $x) { say $x }
Test(50)
50
Test(0) # fails constraint
Parameter type check failed
Test("50") # not an Int
Parameter type check failed


Of course, if we drop the Int from the above, then we get coercion:

sub Test(where { 0 < $_ <= 100 } $x) { say $x }
Test(50)
50
Test("50") # matches constraint when numified
50
Test(0) # still fails it, as expected
Parameter type check failed


Getting this to work forced me to refactor the type hierarchy somewhat - a job that needed doing, but that I'd been putting off (partly out of not trusting myself to do it right). It was a tad nasty, and needed some fixes inside Parrot too. However, the net result is that the majority of built-in object types, such as Int, now inherit from Any (which we didn't have before), and that in turn inherits from Object. The Any distinction will be needed to get junction auto-threading of arguments to work properly; my initial attempt was inefficient and broken. We'll most likely need to do HLL type mapping and similar before that will really work too, though. But anyway, now the following things give what you'd expect.

if 42 ~~ Any { say "yes" }
yes
if 42 ~~ Object { say "yes" }
yes
if 42 | 43 ~~ Any { say "yes" }
if 42 | 43 ~~ Object { say "yes" }
yes
if 42 | 43 ~~ Junction { say "yes" }
yes

I've done a few other things, but I've got a work meeting here tomorrow, so I'm going to sleep now and write about them soon.

Reading through the transcript of the Perl 6 call last week, I noticed that I/O was mentioned as something that Rakudo was lacking and that people were really missing. I flew to the UK on Friday, was at a wedding on the Saturday weekend and meeting up with a friend in London on the Sunday and flew back today. However, I managed to grab a few spare hours on the train on the way there, and on the Sunday evening while Normal People slept and the morning before my flight on the Monday to get some Rakudo hacking done. As a result, we now have the beginings of IO in Rakudo.

Here's a quickly hacked up random number game.

my $answer = int(rand(100)) + 1;
my $guesses = 0;

say "Guess the number (between 1 and 100)";

for =$*IN -> $guess {
    $guesses++;
    if $guess == $answer {
        say "You got it right in $guesses guesses!";
        exit
    }
    elsif $guess < $answer {
        say "Too low";
    }
    else {
        say "Too high";
    }
}

Here's an example run of the game.

C:\Hacking\parrot\languages\perl6>..\..\parrot perl6.pbc guess.p6
Guess the number (between 1 and 100)
50
Too low
75
Too low
87
Too low
95
Too high
91
Too low
93
Too high
92
You got it right in 7 guesses!

You'll note that here I'm using $*IN. This is an instance of the IO class, so you can do:

say $*IN.WHAT;

And it will report "IO". Writing =$*IN means "get me the iterator to read from $*IN", which reads a line at a time. $*OUT and $*ERR are also available, so we can output to STDERR now by doing:

$*ERR.say("OH NOES! IT DID EXPLOSHUN!");

So we have those three file handles, but what about getting one to a file itself? That's what open is for. Here's a program that takes a file name and then prints each line in that file with the line number at the start of it.

my $fh = open(@*ARGS[0], :r); # :r = read mode
my $line = 1;
for =$fh {
    say $line++ ~ " $_";
}
$fh.close();

I ran it on the readme; I'll just quote the top bit of the output.

C:\Hacking\parrot\languages\perl6>..\..\parrot perl6.pbc ln.p6 README
1 ## $Id: README 25155 2008-01-22 19:25:25Z pmichaud $
2
3 =head1 Rakudo Perl 6
4
5 This is the Perl 6 compiler for Parrot, called "Rakudo Perl 6,"

You'll notice the use of the new pair syntax for specifying the mode to open the file in, which I talked about in my last post. So already we're building stuff on top of that syntax. :-)

I'm not sure how busy the coming week will be, but I do know that next weekend and Monday are set aside for doing Perl 6 related things: I will be joining the Oslo QA Hackathon. So I look forward to seeing any of you that will be attending there, and hopefully bringing us another few steps closer to Perl 6 on Parrot.

First off, sorry it's been a while since I last posted. It's mostly been that I've just not done a lot between the Ukrainian and Dutch Perl Workshops and now, which was thanks to being busy with $REAL_LIFE and lacking either time or brain cycles for hacking Rakudo. Thankfully, I'm back into the swing of things now, and it's time to catch up on the blogging.

I'd like to give a belated thanks to everyone at the UPW and DPW for such a good time. I greatly enjoyed attending, meeting people and speaking at both workshops, and Kiev was a more beautiful city than I had imagined it being. The cathedrals and churches there are incredible - I'll surely be coming back to Ukraine to explore some more!

At UPW, some deficiencies in smart matching of arrays was discovered in Rakudo and I promised a fix. I also said I'd write a note about it here "soon", which I failed to do - sorry. I thought I had fixed array comparison with smart match, but that appears not to be working right now. :-( Also, the Perl 5 and Perl 6 way of checking if a list contains an element are different, and the Perl 6 way was dependent on something else that wasn't implemented (though I've started work on it now, so hopefully we have that soon too). Anyway, it's on the horizon.

I'm hazy on exactly what I got done at DPW and around there, but I know I started on pairs. Thanks to some extra contributions from cognominal++, quite a lot of this is now implemented. As specifying named parameters in a sub was already done, happily it means you can now do things like:

sub sayit (:$what) { say $what; }
sayit(what => "Oo minya pivo!");
Oo minya pivo!

It also works with object instantiation too.

class Foo { has $.a }
my $x = Foo.new(a => 42); say $x.a;
42

As well as the fat arrow syntax, there's also the colon pair syntax, in various forms (we ain't got 'em all yet, but the most useful ones are there).

sayit(:what);
1
sayit(:!what);
0
sayit(:what('privyet'));
privyet
my $what = 'nyam'; sayit(:$what);
nyam

Another thing I've hacked in some basic support for is the subset keyword, which allows you to write refinement types. You can smart-match against the type name to test if the variable meets that type.

subset Guess where { $_ >= 1 && $_ <= 100 };
if 42 ~~ Guess { say "yes" } else { say "no" }
yes
if 142 ~~ Guess { say "yes" } else { say "no" }
no
if 97 ~~ Guess { say "yes" } else { say "no" }
yes
subset LowGuess of Guess where { $_ <= 50; };
if 42 ~~ LowGuess { say "yes" } else { say "no" }
yes
if 97 ~~ LowGuess { say "yes" } else { say "no" }
no
if 0 ~~ LowGuess { say "yes" } else { say "no" }
no

Hopefully more types related stuff coming soon, amongst various other things. I'll be at the Oslo QA Hackathon, and hope to get plenty of free time there, as well as generally while traveling over the coming weekends.

I've been at the German Perl Workshop for the last few days. While I was sat in a bunch of talks being presented in a language I don't understand, I had some free time to hack on Rakudo. That means there's been a few fixes and some new features.

First, I set about doing some re-factoring to the objects work I had done so far. It had wound up with a lot of inline PIR in actions.pm, and that in turn fixed us to one class system, which was fine for a first-cut implementation to get a few things working, but far from what S12 calls for. Now all the inline PIR I added is gone, replaced by calls to methods written in PIR.

With the refactoring done, it was then far easier to add what is needed to have methods in classes resolving conflicts and taking precedence during role composition. Of course, exactly how well the roles implementation matches S12 is yet to be seen, and that calls for lots of tests. I'll be drawing on the Moose ones for that.

With some basic OO support in and ready for people to play with, I moved on to looking at the regex support. There was already some there, but I've put in a bit more. $/ (the match variable) wasn't being passed into inner scopes, so if you did a pattern match in the condition of an if block then $/ would not be set inside the block. That is now fixed. Additionally, I implemented $0, $1, $2, etc numbered captures, as well as $ and $ named captures. There is also the "regex" keyword for introducing named regexes, so you can say stuff like:

regex Year {\d\d\d\d};
regex Location {German|French|Italian|London|Dutch|Ukrainian};
regex PerlConference {<Location>\sPerl\sWorkshop[\s<Year>]?};


I've got rule and token parsing, but they don't pass along the :ratchet and :sigspace modifiers yet. I hope to have that resolved soon.

A natural fall-out of adding support for $ is that %hash is now also supported for using constant hash keys. It's not much, but it's another small bit done.

It's been something of a mixed bag being at the GPW. While many people are interested and supportive of the work that is being done on Perl 6, others seem to prefer to use me as their latest target for snipey comments about how it will never be completed. And one person, after making a series of comments about code and papers he'd never read, decided to make an uncomplimentary comment about what I do with my wife in bed (by the way, I'm single). While I can put up with people being ignorant, I find personal attacks completely unacceptable, and not what I've experienced anywhere else in the Perl community. And now I'm left wondering whether I really want to spend any more of my time - which it costs me to take away from $DAYJOB - coming to GPW again.

I've stayed in the same country for a whole month now, and am feeling restless. Thankfully, the Perl community has provided a fix for this problem. In fact, it's provided three of them: I'll be attending (and speaking at) the German Perl Workshop, the Ukrainian Perl Workshop and the Dutch Perl Workshop. So if you're going to any of these, looking forward to seeing you there! :-)

Since I last wrote, I've been continuing my work on Rakudo's OO support. Part of this has been reading and comprehending more of S12, part of it exchanging emails with Stevan Little and looking over some Moose stuff and a lot of it has been just grinding out the code. I'm happy to say that things are now a tad further on.

First of all, my initial work on attributes was some way out of line with S12 (mostly because it was just to get *something* working). With the latest work, attributes are now all stored as $!foo. If you write $.foo, you will get an accessor/mutator method generated. Note that the default eventually will be accessor only and you will write "is rw" to get a mutator too, but I think it's easier and more useful to people playing with Rakudo to be more liberal and allow both until that is in place. If you declare the attribute as $x, then it's $!x really but you get a lexical alias named $x so you can refer to it either way. This work brings the implementation somewhat closer to S12 than before.

Next up was inheritance. Helpfully, while I looked away from Rakudo for a little while due to $DAYJOB and entertaining a guest, someone put the parsing side of traits in. That meant I could dig straight into the semantics. The implementation here has already been through a few iterations, getting increasingly less hacky each time. What we have now actually calls trait_auxiliary:is, and applying a trait that is a class is just a standard case of applying a trait. No special cases - it's all decided by multiple dispatch. This also open the way to implementing any other traits that can be applied to a class. A final tweak allowed subclassing of built-in types such as Bool.

I've also started on roles. There is a long, long way to go here. However, composition is started. It doesn't do conflict resolution correctly yet (just haven't had time to do what it needs to make it work yet; I will do so very soon).

And finally, it's a small thing, but I added parsing and code-gen for self too.

So, want an example?

role Wob {
    method be_tired() {
        say "I'm wobbed.";
    }
}

class Person {
    method species() {
        "human";
    }
    method describe_self() {
        say "I am a " ~ self.species() ~ ".";
    }
}

class Geck is Person does Wob {
    method species() {
        "geck"
    }
}

my $p1 = Person.new();
$p1.describe_self();     # I am a human.

my $p2 = Geck.new();
$p2.describe_self();     # I am a geck.
$p2.be_tired();          # I'm wobbed.