I'm back from a nice break in Italy and have been digging back in to Perl 6 stuff again. Today I've been doing a Vienna.pm-funded Rakudo day, and here's what I got up to.
First off, I went for a look through our RT queue. We now have over 400 tickets that are either new or open. While on the one hand that means we've a lot of work to do, it's also a sign that people, more and more, are playing with and exercising Rakudo. In just browsing through it, I found a bunch of things I could work on and hopefully resolve fairly easily during the day, and also another bunch of things that were already resolved. Just spotting the latter allowed me to mark 3 tickets resolved.
A couple of the things I worked on related to subtyping. Of note, the standard grammar accepted:
subset Foo of Int;
Without requiring a where clause. Rakudo now also accepts this, and our parsing is a little closer to STD.pm too (we parse traits on subtypes, but we don't do anything with them just yet). Next, I got Rakudo to support a neater syntax for declaring anonymous subtypes in signatures. If you just want to match a specific value, you can write the value in the signature, and that's it. For example, here is yet another way to do factorial (a recursive version).
multi factorial(0) { 1 }
multi factorial(Int $n) { $n * factorial($n - 1) }
say factorial(5); # 120
A signature
Lyle++ had sent in a patch a while back for $*CWD and chdir. I took a look at these today. The $*CWD one looked pretty good, so I applied that with just a minor tweak. The chdir one needed some more attention and fixing up first, but I got that applied and extended the tests to better exercise it. Then I got both test files added to spectest.data. So now chdir and $*CWD are both functional. Here's some play with them in the REPL.
> my $fh = open("spectest.data",
Unable to open filehandle from path 'spectest.data'
in Main (:1)
> say $*CWD;
C:\Consulting\parrot\trunk\languages\rakudo
> chdir "t";
> say $*CWD;
C:\Consulting\parrot\trunk\languages\rakudo\t
> my $fh = open("spectest.data",
>
We had a couple of tickets relating to the interaction of
Finally, in preparation to improve type check failure error reporting and resolve at least one ticket in that area, I factored all type check error generation out to one routine, which we now call consistently. That means errors that previously missed out mentioning the expected and received types now do so, and the other issues I can fix - on some future Rakudo day - in one place, and everywhere that reports such errors will benefit.
In the course of the day, I also discovered a couple of other tickets that I had opened up to investigate at the start of the day were also already-fixed issues, so I made sure we had proper test coverage and got them closed up.
So, a pretty productive day. Thanks to Vienna.pm for funding!
A while back, I did a Rakudo day and didn't get around to writing a report (probably because I hacked until the point where all I wanted to do was sleep). The main thing I did on that day was implement
The
Here is a simple example.
sub do_some_work() { say "doing stuff" }
do_some_work();
say "--";
my $handle = &do_some_work.wrap(sub () {
say "About to do some work";
callsame();
say "Finished doing some work";
});
do_some_work();
say "--";
&do_some_work.unwrap($handle);
do_some_work();
So, first we take a sub and call it. The result is just as you would expect. Then we wrap it with another sub - an anonymous one. We could have used a closure or pointy block here too. This some uses callsame() to call the original sub. If we had any arguments passed in, it would have passed the same arguments along. You can use "callwith" instead to supply a different set of arguments, if you had done some pre-processing on the arguments and wanted to pass along the modifications. Calling
doing stuff
--
About to do some work
doing stuff
Finished doing some work
--
doing stuff
Let's take a look at a second example.
sub foo() { say 1 }
foo();
say "--";
my $h1 = &foo.wrap({ say 2; nextsame; say "not here"; });
foo();
say "--";
my $h2 = &foo.wrap({ say 3; nextsame; say "nor here"; });
foo();
say "--";
&foo.unwrap($h1);
foo();
say "--";
&foo.unwrap($h2);
foo();
This time, we do a couple of things differently. We use the nextsame function instead of callsame. nextsame defers rather than calls, so after calling nextsame we are never inside the current routine again. Thus the "not here"/"nor here" say statements will never be executed. Secondly, we unwrap in a different order than we wrapped. This demonstrates the use of out-of-order unwrapping, meaning you can add and remove behaviors as you wish without having to worry where in the list of wrappers they are. The output of this program is:
1
--
2
1
--
3
2
1
--
3
1
--
1
We also have tests passing where we wrap and unwrap in loops, applying the same closure as a wrapper many times, which also works.
Since doing the initial cut of wrapping on a Rakudo Day, I have further improved and refactored
Thanks to Vienna.pm for sponsoring the Rakudo Day that saw us get wrap support.
Aurora swims in the ether
Emerald fire scars the night sky
Amber streams from Sol
Are not unlike the waves of the sea
Nor the endless horizon of ice
Not Unlike The Waves - Agalloch
After the successful completion of my previous Hague Grant, I applied for another one. Happily, it was approved a little while ago, and I've been busily working away on it. In fact, I have already made significant progress towards most of the deliverables (though with plenty of work left to do) and one of them can be considered pretty much completed already. I plan to write several blog posts about the various things I achieve under the grant, and in this one I am going to talk about parallel dispatch, which is the topic of the completed deliverable.
I enjoy masak++'s blogging a lot for the extra little bits as well as the main content, and am taking a leaf from his book and augmenting each of my posts about my work under this grant with a quote. I listen to a lot of music while working, so I've chosen to quote some of the lyrics from what I'm listening to while working on this grant. And yes, I know, it sure doesn't beat the lolcat bible. So, on with the show...
I've talked about hyper operators before. They're the ones that let you perform an action element wise over arrays or lists. For example, we can add two arrays or lists element wise like this:
say ((1,2,3) >>+<< (4,5,6)).perl; # Output: [5, 7, 9]
The same syntax can also be used to dispatch methods too, by placing the >> before the . operator. This means that you can dispatch a method on every item of a list or array and get an array of the return values.
my @a = -1, 2, -3;
my @b = @a>>.abs;
say @b.perl; # [1, 2, 3]
Happily, the various other method call forms can be parallelized too. For example, we can modify an array in place:
my @words = <foo bar baz>;
@words>>.=uc;
say @words.perl; # ["FOO", "BAR", "BAZ"]
Or use the
@pets>>.?lick($guest); # The dog will, the fish won't
You can also chain them together. Here's a quickie that prints out for you all of signatures of the candidates of a multi sub:
multi double(Num $n) { return 2 * $n }
multi double(Str $s) { return "$s $s" }
&double.candidates>>.signature>>.perl>>.say;
The output of this is:
:(Num $n)
Like with other hyper-operators, you're not just saying, "I want this method called on everything in the array", but also telling the compiler, "you can run them in different threads and in any order you want". Of course, even if it runs them out of order it will give you back the results in the correct order. For now Rakudo does not do anything like this, but one day when we have a threading implementation that works and a good way of deciding when to parallelize (or maybe pragmas to give hints) then this will be able to happen.
So, that's parallel dispatch. Have fun, and in the next grant post I'll talk about my work to overhaul Rakudo's method dispatcher, winning performance and shiny new features.
I scheduled this week's Rakudo day to take place just before this month's release, and used most of it for fixing bugs. Before I dug into that, I spent some time looking over the change log and adding a few things that it was missing. As in April, we're going to have a really quite impressive release this month with lots of new features and improvements. It's great to be part of such a productive team, and I'm quite proud of what we're achieving together.
My first fix was to typed attributes. They worked for the scalar case, but not for the array and hash cases (just about all the pieces were there, just the very final bit to apply the constraint to the attribute was missing). moritz++ had already written a bunch of tests, so after the fix I was able to unfudge them, giving us another 19 passing tests and resolving an RT ticket. Not a bad start to the day's bug fixing.
Next up, I wanted to sort out attribute initializers. If you write:
class Foo { has $.r = rand }
Then you should (probably
class Foo { has $.a = 1; has $.b = 2; has $.c = $.a + $.b;}
> say Foo.new.c # output 3
That is, initializing one attribute based upon the already initialized values of those that appear before it.
The Rakudo multi-dispatcher is pretty stable, but there have been a couple of issues that have come up where it doesn't get things right. Today I dug into solving a couple of bug reports that turned out to boil down to the same fix in the end, which affected multi-variants that had a slurpy positional parameter and named parameters.
There were a couple of tickets about attributes declared with the & sigil not working as expected. These have now been fixed, so you can declare attributes with this sigil and expect them to work, or mostly work anyway.
As I mentioned last time around, pmichaud++ has got operator overloading and user-defined operators in place. This is great, but we weren't generating the various meta-operator forms (so you couldn't use your user-defined operators in a reduction, assignment, hyper, cross or reverse meta-operator). After some hacking tonight, you now can. Thus, taking a pointless but easy to understand example operator, we now can do things like this:
sub infix:<wtf>($a, $b) { $a ~ "WTF" ~ $b }
say 'OMG' wtf 'BBQ'; # OMGWTFBBQ - this already worked
my $a = 'OMG';
$a wtf= 'BBQ';
say $a; # OMGWTFBBQ
say [wtf] <OMG BBQ PONIES>; # OMGWTFBBQWTFPONIES
say 'BBQ' Rwtf 'OMG'; # OMGWTFBBQ
say ~('OMG','BBQ' Xwtf 'OMG','BBQ'); # OMGWTFOMG OMGWTFBBQ BBQWTFOMG BBQWTFBBQ
Finding a better use for this new power is left as an exercise for the reader.
Up until today in Rakudo, when your program failed in some way, the errors were not too helpful because they missed a line number and file name that related to the original source code; instead, it told you about a position in the generated intermediate code. Various of us (pmichaud++, Tene++, me++) have been doing bits to get us towards being able to show line numbers at the Perl level rather than in the Parrot Intermediate Code. Today I coded up one of the final bits, and things are looking better. If you have for example this program:
class A {
method b {
die "OH NOES I HAS A FAIL"
}
}
sub bar {
A.new.b;
}
sub foo {
bar()
}
foo();
Will now fail with:
OH NOES I HAS A FAIL
in method A::b (crash.p6:3)
called from sub bar (crash.p6:7)
called from sub foo (crash.p6:10)
called from Main (crash.p6:12)
Which is rather more helpful. I was about to commit it just with line numbers, when pmichaud flew in with a quick patch to emit annotation directives for filenames too. So now we report both. Yay.
While I was working in the area, I also dealt with a ticket about our previously less than wonderful error if you did a return outside of a routine. We now say that, rather than the more cryptic "no message and no handler" that we gave before.
One of the most exciting happenings in Rakudo over the last few days is that pmichaud++ added support for overloading and defining new operators. Hopefully he'll have a moment to blog a little about that. Tene++ then allowed for defining custom circumfix operators. Today ruoso tried to define a (non-custom - I didn't figure out how to do custom ones of those yet) method postcircumfix:<{ }> { }, but that didn't work out too well. I added support for the postcircumfix category, but it still didn't work, at which point I discovered that methods hadn't quite caught up with routines yet in the world of op definition. A couple of lines later and then I had this running:
class Foo {
method postcircumfix:<{ }> ($name) { say $name }
}
Foo.new<abc>; # abc
One of the things we're noticing more and more with Rakudo as we add more features and need to run more tests is that it is, well, slow. Part of it is the parsing speed for sure, but a decent bit of it is at runtime too. Today I wrote and checked in some micro-benchmarks for some very specific language features that are heavily used in Perl 6 (multi-dispatch, method dispatch) as well as checking startup time. I'm thinking that I may set these up to run regularly and graph the results over time so we can track - just as we do with passing spectests - when we do stuff that hurts performance and when we do stuff that improves it (not just when we're expecting it, but when something has an unexpected effect).
Along the way, I managed to do a small tweak to signature binding that got us a 7% performance win. However, it's a drop in the ocean compared to the gains we need. Signature binding right now is, bluntly, a pig - and Parrot's not exactly stellar calling performance doesn't do us any favors either. I was fairly happy to see that multi-sub dispatch was only slightly less performant (10% worse) compared to single-sub dispatch. And that's using a dumb multi-dispatch type cache that I threw together in an afternoon.
Rakudo is now living in its own HLL-space in Parrot, which means there's now opportunity to fix a few things up. Some things - like conflicts with Parrot built-in classes - just went away as a result of this, so I closed those tickets in RT. The first one I want to take on is avoiding re-blessing Parrot multis into Perl 6 multis to get the correct dispatch semantics. However, that has the upshot that we need to consistently use Perl 6 multi-dispatch everywhere now. This is going to take some work, so I created a branch, ripped out all of the re-blessing stuff, added the HLL-mapping. So far we don't even make it through the compile, let alone any tests. But it's started, and by the end of it we'll have got all of the operators running off proper Perl 6 dispatch semantics, fix some of the overloading problems we have, get to rip out the special-case junction operator code and shave a bit off our startup time too. So, some effort, but an overall win once it's done.
I finished up the day trying and failing to track down a really weird issue. mberends++ has done some great work on Temporal, the date/time handling support in Perl 6. He'd got something running just fine under Rakudo, but trying to add it to the setting led to the stage 1 compiler hanging. Even more odd, however, was that compiling that file alone with the stage 1 compiler worked - it was just when we tried it added into the concatented setting that it failed. Removing some things seemed to help, but I didn't find a pattern, nor did I see anything that looked like it could obviously cause such an issue. So, color me confused. Very confused. Will be interesting - and probably very annoying - to get to the bottom of this one, but a couple of hours of trying later I'm stumped. Such are some problems.
Thanks to Vienna.pm for funding this Rakudo Day.
Rakudo day was a little but non-day-ish this week, and instead ended up spread over a couple of days. This was mostly as a result of me accidentally scheduling it on a national holiday, then being unable to resist the temptation of going and enjoying some nice food and beer, sat outside enjoying the evening warmth. Anyway, between the work yesterday and today, we've some new stuff.
We've had hyper-operators for a while on lists/arrays, but now I've implemented the extra bit of the hyper operator specification that makes them work on hashes too. The dwim-ness relates to the keys of the hashes; the fat end pointing at a hash means "I want its keys". However, the operations are performed on the values. Thus if you had a hash mapping names of candidates to the number of votes they got, and wanted to total the votes of two groups (or use this to make a running total over many hashes) you could write:
my %total_votes = %group_a_votes >>+<< %group_b_votes;
Which gives you the set union of the two keys, so if one group didn't vote for a certain candidate at all we still get their votes included. You could also use it to combine the data from, say, two different monitoring stations, but ignore readings that both of them didn't supply a value for.
my %max_readings = %station_a <<max>> %station_b;
The asymmetric cases work too. I didn't find any tests for the hash hyper operator cases, so I wrote a bunch of them to exercise it.
In other matters, I did a refactor to handle roles made up of other roles and bring us more in line with the Perl 6 specification. It states that the composition of a role made up of other roles is not evaluated until the point we compose it into a class, and we were being a bit too eager about that. The upshot was that if you ended up with a diamond hierarchy of role compositions and tried to compose the bottom of the diamond into a class, Rakudo would complain about a composition conflict. I fixed this, and expected it would fix another issue that I was quite sure was due to the same underlying bug. It didn't; some hunting later I found that actually Parrot had a bug when you tried to inherit from two anonymous classes. A one-line Parrot patch later, that bug was fixed too. Both already had test cases that I could unfudge.
I also did a couple of other small tasks.
Thanks to Vienna.pm for funding this Rakudo Day.
Since I missed a couple of Rakudo days in the middle of last month, thanks to the excellent Nordic Perl Workshop and the hackathon that followed it, I did an extra one today to get caught up with them a bit and to keep things moving.
I did a first cut of enums quite a while ago. Since then, a lot of things in Rakudo have changed, parts of the spec became clear and, well, the first cut of enums just wasn't that good. Sometimes you gotta get something wrong as a precursor to getting it right, or at least righter.
The end result is pleasing in that actions.pm is now a lot shorter, since we construct enums in a quite different way. A bunch of code that only hung around because the previous enums implementation needed it has gone away. Also, the code that the compiler generates for enums is vastly more compact; the heavy lifting is done in a different place. It's also been easier to add a bunch of the missing stuff that people were asking for, and various other bugs just evaporated.
Now, for enum elements, you can introspect them some more:
enum Day <Mon Tue Wed Thu Fri Sat Sun>;
say Mon.name; # Mon
say Mon.perl; # Day::Mon
say Min.WHAT; # Day - not just the string, but Day itself
Smart-matching now works too. So if you have some variable $thingy that has had Day mixed into it an initialied, you can do things like:
given $thingy {
when Mon { say "yaaaaawwwwn" }
when Tue|Wed|Thu { say "work work work" }
when Fri { say "w00t, nearly the weekend" }
when Sat { say "OH HAI I'M AT THE BAR" }
when Sun { say "good morning, vicar" }
}
You can also do
There's still some more bits to do, but enums should now be greatly improved in Rakudo. There's a couple more tickets, and I'm sure others will help flesh out the weaknesses of this new implementation, so we can shake out the bugs.
I also dealt with a few other bits and pieces.
my $cheapest = @products.min(*.price);
Which is kinda cute.Thanks to Vienna.pm for sponsoring this Rakudo Day.
constant Pi = 3.14;
say Pi; # 3.14
Pi = 42; # error
If your Rakudo is built with ICU, you can use unicode characters for the name too.
constant $pi = 3.14;
say $pi; # 3.14
$pi = 42; # error
And even use the constant as a type constraint...
constant answer = 42;
my answer $x = 42; # ok
$x = 43; # dies
masak asked if we could possibly report the provided and expect types when reporting type-check failures, to give better feedback. I implemented this, and also managed to simplify and fix another bug inside the signature binder along the way, which had meant the following would not work:
sub foo(@a) { say @a.elems }
foo([1,2]|[3,4,5]);
This will now auto-thread and call foo twice, the first time outputting 2 and the second time 3.
I also fixed a couple of other little bugs in Rakudo.
Finally, Tene has been working for a while on getting Rakudo to live in its own HLL namespace in Parrot, which will enable us to clear up a few other things and no doubt resolve a few tickets (it'll stop us colliding with Parrot internal bits for one). There was some debugging work to do, so I spent a couple of hours on this, and got us a lot further than we were. Still much to do, but we'll get there.
Thanks to Vienna.pm for sponsoring this Rakudo Day.
I'm still feeling a little tired from NPW and the following hackathon, but finally I've found time - at long last - to write about a little of what I got up to there, for all who are curious. Before I dig in to that, I'd like to say a massive thanks to the organizers of the Nordic Perl Workshop and Hackathon. It was a wonderful event to attend and to speak at, and the venue for the workshop - central and complete with a roof terrace offering gorgeous views over Oslo - was The Awesome. The views from the Redpill-Linpro office where the hackathon was hosted were really rather nice too. So, I got my eye-candy for the trip.
I've been working on topics relating to parametric roles for a while, and a little before the hackathon had got typed arrays and hashes working. I had a few remaining small design issues to get fleshed out before I could wrap up the work and call my Hague Grant finished, and was happy to be able to spend time with Larry working out the answers to them. I then whipped up the code to implement parametric sub-typing, such that if you have a role R and two types T1 and T2 such that T1 is narrower than T2, then R[T1] will also be narrower than R[T2]. I then got S14 to be in line with the various design decisions that were made, and tidied it up again.
Another big design topic for the conference was on the subject of laziness. Fittingly, despite a bunch of us moving into a room on the Sunday morning to have the discussion, we actually managed to put it off until late afternoon. The most visible outcome of the discussion is that the prefix:<=> operator is now dead. We had some nasty hacks in Rakudo to try and make it work, but none of them really had ever flown too well and not knowing how to deal with it has been part of why we haven't yet got further with laziness in Rakudo just yet. So, now it's out the way; to read a single element from an iterator, use
One of the evenings, Patrick and I sat down to work on the Rakudo roadmap. We started by going through the existing one, from last summer, and were very happy to be able to rip many things out of it, because they had now been completed mostly or entirely. We then set about trying to get a grip on the things that were left to do, and came up with the list of tasks that will make up the roadmap from here. Patrick is in the process of prioritizing/sorting/tidying up what we came up with, and hopefully it'll be committed to the repository soon.
Elsewhere in the hackathon, cosimo, masak, and mberends worked to get socket IO in place in Rakudo, another nice step forward. This allowed for a HTTP server written entirely in Perl 6, and I saw an early cut of LWP::Simple too. Also, Gabor got syntax highlighting working in Padre based off Rakudo/PGE.
There were lots of other little bits too, but I think this is the bulk of it. Overall, I came away exhausted, but very encouraged by the progress the Perl 6 project and Rakudo are making. It was also great to have the latest Rakudo released named for my local Perl Mongers group, Bratislava.pm. We had a Perl 6-themed meeting on Thursday, with plenty of pizza and beer. Na zdravie!
The big bit of progress today was that I got lexical subs working.
{
my sub foo() { say "pes" }
foo(); # pes
}
foo(); # An error; sub is lexically scoped in the block
Actually, getting that far was the easy bit. The harder bit was making leixcally scoped multi subs work, which is a good deal more interesting. That's because rather than a wholesale replacement, you have to clone the set of outer candidates and then add the new ones. That is:
my multi bar() { say 1 }
{
my multi bar($x) { say 2 }
bar(); # 1 - from the outer candidate
bar('man'); # 2 - from the inner candidate
}
bar(); # 1 - from the outer candidate, which is still visible here
bar('girl'); # error - inner candidate out of scope
Of course, if that block was a closure or something we'd invoke many times (e.g. a sub with many lexically scoped inner multi subs), we really don't want to have to do all of the copying and augmenting of the outer candidate list per call (that'd be a bit of a performance killer). So some of the effort went into making sure we persist those. Plus while we had tests for single-dispatch lexical subs, we hadn't any for multi-dispatch lexical subs, so I wrote a bunch.
I probably mentioned recently that I've been working on import stuff, and that actually we're not doing it right just yet because we import into the package by default and not the lexpad. This bit of work on lexical subs is one step towards being able to do it right. However, there are other issues. Another big blocker that Rakudo users may be familiar with is that this:
my $x = 5; class A { method x { say $x } }; A.new.x
Doesn't work. The main reason we couldn't get it to work was due to a fairly long-standing Parrot bug. Today I got out my trusty C debugger and set about hunting the pesky thing down. Eventually I did and committed a fix to Parrot and a test, which was great apart from it only revealed another separate problem somewhere else that got in the way. After a long discussion with Patrick, we're not at a solution for that one yet. We understand it just fine, but the solution (and making sure it's a good one) will take a bit more thought. But anyway, I'm hopeful that we will nail this bug in the near future.
I also did a few other more minor things.
Much thanks to Vienna.pm for funding today's Rakudo work.