JonathanWorthington's Journal http://use.perl.org/~JonathanWorthington/journal/ JonathanWorthington's use Perl Journal en-us use Perl; is Copyright 1998-2006, Chris Nandor. Stories, comments, journals, and other submissions posted on use Perl; are Copyright their respective owners. 2012-01-25T02:18:42+00:00 pudge pudge@perl.org Technology hourly 1 1970-01-01T00:00+00:00 JonathanWorthington's Journal http://use.perl.org/images/topics/useperl.gif http://use.perl.org/~JonathanWorthington/journal/ Last Post http://use.perl.org/~JonathanWorthington/journal/40450?from=rss <p>Well, on use.perl.org, anyway. I'd like to thank those who run this place for doing so, as it's been the place I've blogged about my Perl 6 stuff for the last couple of years. However, I've found the user experience of WordPress a lot more enjoyable - it was great for the Perl 6 Advent Calendar - and figure I'll blog more if I more enjoy doing so.</p><p>So, from now you'll be able to read my Rakudo news and other Perl 6 related ramblings from my <a href="http://6guts.wordpress.com/">shiny new blog</a>. For those reading through Planet Six, no action required - I'll arrange for my new blog to be aggregated there.</p><p>See you <a href="http://6guts.wordpress.com/">over there</a>.</p> JonathanWorthington 2010-07-18T02:58:24+00:00 journal Perl Mova + YAPC::Russia http://use.perl.org/~JonathanWorthington/journal/40399?from=rss <p>On Friday I flew from rainy Sweden to scorching hot Kiev to attend a combined Perl Mova + YAPC::Russia. The passport queue wasn't overly long, and I'd happily managed to be hand luggage only, so I wasn't too long in the airport. I planned to take the "official" bus, but before it appeared I heard a lady shouting about a mashrutka going to the center, so I took that instead. Understanding basic Russian - well, it coulda been Ukrainian too - sure comes in helpful. I was dropped by a metro station, leaving the 15 euro-cent (yes, really) metro ride a few stops to where I was staying - a hotel right on the main Independence Square. I was a little bemused as I checked in to be spoken to in a mixture of Russian and Ukrainian, by the same person in the same conversation.</p><p>My plane had been slightly late, so I was also slightly late joining the pre-workshop dinner. After I'd filled up on a tasty dish of borsch, we wandered on to a pub that offered no less than 22 different beers. I was...happy.<nobr> <wbr></nobr>:-) Some beer later, it was time for a little stroll, which terminated in a Japanese restaurant, where I had my first encounter with wasabi vodka. All in all, a very fun start to my visit here.</p><p>I was worried I'd sleep badly in my room since it was a little hot, but I actually slept really quite well and was nicely refreshed for the hack meet. There were a lot of people there hacking on all kinds of different projects. Some folks were interested in contributing to Rakudo, and so we gathered around on a table and I helped each of them find and get started on a task. It went extremely well - everyone in attendance contributed Rakudo patches or code that allowed us to close an RT ticket right there on the day, or that will after I apply patches. Of note, people who were new to Rakudo hacking:</p><ul> <li>Located and unfudged tests for \e that was recently fixed, allowing the ticket about that to be closed.</li><li>Implemented<nobr> <wbr></nobr>.all,<nobr> <wbr></nobr>.any,<nobr> <wbr></nobr>.one and<nobr> <wbr></nobr>.none method versions of junction constructors and added tests for them, allowing that ticket to be closed.</li><li>Debugged and then wrote a patch that fixed a bug in range iteration, plus added tests, allowing an RT ticket to be closed.</li><li>Implemented<nobr> <wbr></nobr>.cando and did some related refactors in the area that I suggested and also located some tests to unfudge. Due to network issues that one didn't get applied on the day, though I have it and have just applied it locally, so it'll be in soon.</li><li>Tracked down what was wrong with forming colon pairs from variables involving twigils, patched it and made sure we had some test coverage of that; again, we closed a ticket.</li><li>Wrote a patch to fix a bug in the series operator that had been reported in RT, along with some test cases for it.</li></ul><p>I think it goes without saying that this is really quite impressive, especially given they had to share one Rakudo core hacker between them for guidance. In fact, I even had time to cook up a few patches myself amongst guiding others! It was great fun to hack alongside such pepole. I handed three spectest repo commit bits out that day, and I think they were all used. It's experiences like the one at this hack meet that make me really glad that we're writing most of Rakudo in Perl 6 or a restricted dialect of it - all of the above tasks required (apart from a one line change in one of them) no PIR or C knowledge at all. Some of those at the hackmeet said they may find time for a bit more Rakudo hacking in the future, which would be really great.<nobr> <wbr></nobr>:-) After the hack meet, there was more nice food - including a Chicken Kiev - and some lovely beer at prices I've become rather unacustomed to in Sweden.</p><p>Day two was talks day. I had one 40 minute talk on Perl 6 signatures which went well. I received some great questions, and I hope I answered them all to people's satisfaction. Later on, I gave a lightning talk about Rakudo * and what it would contain, and showed off a <a href="http://pivo.jnthn.net/">simple little example site</a> that used Rakudo Perl 6, including the modules JSON::Tiny by moritz++ and FakeDBI by mberends++ along with Blizkost (sorear++ for that) to get at the Perl 5 CGI module (though I'll refactor it to use Web.pm in the near future - I just wanted an excuse to show off Blizkost). The evening was, of course, more food and beer, and a lot of fun.</p><p>The final day of the workshop was a bit different - a river cruise! It was very relaxing, and gave me yet another way to enjoy this beautiful city. Certainly good for unwinding after a workshop. Most people left either after that or were flying home today; I on the other hand used "no direct flight on a Tuesday" as an excuse to get another day to potter around Kiev, and today I've enjoyed doing exactly that, gently strolling between my favorite sights and stopping in the odd open air cafe in a park to keep myself refreshed in the somewhat stuffy weather. Soon it'll be time to take my last dinner here, an evening stroll and maybe find some outdoor place to sit and nurse a final beer before it's time to get some sleep before my flight home tomorrow.</p><p>Beautiful Kiev. It gets harder to leave each time I come.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2010-06-15T15:11:09+00:00 journal Perl 6 talk in Malm&#246; http://use.perl.org/~JonathanWorthington/journal/40316?from=rss <p>Recently I moved to Sweden to join startup company <a href="http://www.edument.se/">Edument</a>. They are, happily, very open to and supportive of my involvement in the Perl 6 project, and are organizing and sponsoring me to give a <a href="http://natverk.dfs.se/node/19602">talk on Perl - both 5 and 6</a> for the <a href="http://www.dfs.se/">Dataf&#246;reningen</a> in Malm&#246; on Tuesday 27th April. I believe this is only open to those who are members of the user group; if you are, feel free to come along though.</p> JonathanWorthington 2010-04-20T09:55:36+00:00 journal The Easter Hackathon http://use.perl.org/~JonathanWorthington/journal/40303?from=rss <p>I spent my Easter up in Uppsala with masak++, who it turns out is not only awesome at breaking Rakudo, but also a great host. Amongst a lot of memes, bad puns, wonderful noms (I learned how to make k&#246;tbullar!) and even a Czech beer, we got quite a bit of Perl 6 hacking done too.<nobr> <wbr></nobr>:-)</p><p>On the first evening, we managed to fix up a bunch of issues with various escape sequences in regexes and strings, and also the fact that "\$x" was failing to escape the $ since we made ng the new master! I fixed a longstanding bug that <a href="http://use.perl.org/~masak/journal/40167">annoyed masak so much that he dedicated a whole blogpost to it</a>, did some work on enforcing zone constraints in signatures and, in an awesome bit of collaborative development along with colomon++ and moritz++ wired up their new reduction implementation to the reduction meta-operator parsing stuff. That means that you now have triangle forms of the meta-ops, chaining meta-ops work and right-associative ones do the right thing too. We never had any of these right before.</p><p>Spurred on by the interpolation work on the previous night, on the next day I started looking at what it would take to implement more of the interpolation bits, and after a bunch of hacking I accidentally the whole lot of it. You can now interpolate indexes into both arrays and hashes, calls to subroutines and method calls on all of these. The way this works in Perl 6 parsing-wise is really neat: we just call from the quote parser back into the standard expression parser, but giving it a few constraints so it knows only to parse the things that are allowed to be interpolated into strings. It's a fairly directly following of STD - we differ in a couple of minor ways for now, but the approach is the same.</p><p>Along the way, I used the regex tracer that pmichaud++ had implemented while developing the new grammar engine, and noticed that we seemed to be dropping into a lot of protoregexes we shouldn't have been. A little more exploration and I realized that we were failing to calculate a bunch of constant prefixes for regexes. Fixing that gave a 5% saving on parse times.</p><p>The day after that, I worked on something I'd really been putting off because it's hard and painful: making the setting the outer lexical scope of user programs. While some languages define a prelude, Perl 6 essentially defines a "circumlude" - it's as if your program was running as an inner lexical scope inside the built-ins. That means that you find just about everything through lexical scoping. I got the first difficult 80% done and committed, but I suspect I've now got another really difficult 80% left to go, and probably another really really difficult 80% after that. Along the way, I fixed some bugs to do with scope modifiers "leaking". Anyway, to prove it helped somewhat, I tossed a whole bunch of "our"s that had been in as hacks while we didn't have the setting as an outer lexical scope. I suspect that perfecting this is going to be a large chunk of my work on Rakudo over the next couple of weeks.</p><p>The last day, I did the (relatively easy, thanks to all the groundwork I had laid during my signatures grant) bits of work to get us able to do smart-matching against signatures. This deserves an entire post of its own, and I'll write one for you soonish, but essentially you can use it to write a given/when that dives into a hash/array/object and looks at whats in it, just using declarative signature syntax. It's just like having a sub-signature on a parameter passed to a block or routine, apart from the signature object stands alone. By the way, I do plan to submit a talk on signatures for YAPC::EU - I'm really quite excited about the declarative power they offer that goes far, far beyond a better way to do parameters than Perl 5's @_. Especially in combination with multi-dispatch.</p><p>masak++ spent a good portion of the time working on Temporal/DateTime spec and starting an implementation to back it up. It has to be one of the most bikeshedded areas in the Perl 6 spec, with plenty of "let's do something SO clever" thinking along with the much more sensible "we want something minimal but useful in the language itself, and the clever stuff belongs in modules". Anyway, masak++ and mberends++ for having the patience and diligence to take on something that we really need a finalish answer on, amongst all the various opinions people have on how it should be done. I'm hopeful that we're now approaching the point where we have something that works for the vast majority of use cases, and the answer to anyone who it doesn't work for is simply, "go write a module".</p><p>Anyway, that was The Easter Hackathon! For those not following the commits, I'm happy to report that others are also committing bits here and there, and we're steadily working towards brining you Rakudo *.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2010-04-08T23:59:06+00:00 journal A quick Rakudo update http://use.perl.org/~JonathanWorthington/journal/40258?from=rss <p>The last week has brought some <a href="http://use.perl.org/~pmichaud/journal/40248">sad news</a>. While software is of course insignificant compared to life and health, and it's absolutely right that at this time Rakudo should be the last thing Pm should be worrying about, I know a lot of people will be wondering what this means for the Rakudo * release. Myself and the other Rakudo developers are still working out the exact details, but here's an overview.</p><ul> <li>Rakudo * will be delayed in the "not in late April" sense. We all agree on this. There's no way you can take the lead developer of a project away, when the overall team isn't that big anyway, and expect to deliver the same product on the same schedule.</li><li>While I've always stated Q2 in my talks, we've also always had an internal target date of April. Of course, this is open source so nothing is internal, and April has been widely latched on to.<nobr> <wbr></nobr>:-) We're sticking with the Q2 goal, but now expect to deliver in May or June (preferably, before YAPC::NA<nobr> <wbr></nobr>:-)).</li><li>While we do of course greatly miss Pm's company, direction, and code, I'm comfortable that there's not anything on the Rakudo * ROADMAP that absolutely blocks on Pm. There's things that are harder for the rest of us to do, but we've actually tackled some of them head-on in the last several days, and now have first cuts of many of them - or in some cases are virtually done with them.</li></ul><p>There's been a lot of exciting progress in Rakudo recently - the ChangeLog for the last release gives a bunch of it, but we've done a whole load more since then too. I'll try and blog about some of it soonish. In the meantime, I'd like to thank all of the Rakudo and Perl 6 team for being amazing to work with on this, and my thoughts and prayers are with Pm and Paula.</p> JonathanWorthington 2010-03-22T01:09:03+00:00 journal My life refactored, Zavolaj, workshops, Rakudo and more! http://use.perl.org/~JonathanWorthington/journal/40247?from=rss <p>It's been a while since I scribbled anything here, so a few quick updates.</p><p>First, an update on me. Since last time I wrote, I've moved country. Yup, I actually left behind my beloved Bratislava and I'm now hacking from Lund in Sweden. The beer - like everything else - is more expensive, and I don't have views of a beautiful city castle from my apartment here. But the city is overall very pleasant, I'm located a short walk to the center and the train station, and the supermarkets have a great range of international nom (and HP sauce!) I've yet to try the nearby curry house, or any of the other many appealing restaurants here, but I'm optimistic there will be plenty of WIN. Anyway, here's hoping I can get nicely settled down here.</p><p>I was greatly assisted with the move to Sweden by Perl 6 hacker Martin Berends, to whom I am extremely grateful. On the way, we went via the Netherlands Perl Workshop. I gave a couple of talks; I'll get the slides online shortly. It was really lovely to be at the Netherlands Perl Workshop again - the atmosphere was every bit as warm and friendly as I remembered from the last time. To everyone who organized and attended: thanks for a wonderful couple of days.<nobr> <wbr></nobr>:-)</p><p>In the couple of days before the move and during the NPW, mberends++ and I hacked on a shiny new project: <a href="http://github.com/jnthn/zavolaj/">Zavolaj!</a> It is a way of doing native calls from Perl 6, just by adding a trait to a stub subroutine, and is a first draft for native calling in Perl 6. The upshot is that we now have a working MySQL client written using Zavolaj, in pure Perl 6! mberends++ is continuing to build this out, and happily this all means that Rakudo * will ship with database access, amongst other things. (I've also done an example of calling a Win32 API, and we're both pondering an ImageMagick binding with it too).</p><p>My first days in Scandinavia were actually mostly spent in Copenhagen, at a Perl 6 hackathon organized by Copenhagen.pm - a group I hope to drop in on now and then, since I'm only an hour or so's train ride away. There's a wonderful <a href="http://perlgeek.de/blog-en/perl-6/copenhagen-hackathon.html">write-up by moritz++</a> that tells a lot of what went on, and I can only echo his sentiments about how much fun it was to hack and hang out in meat space with other Perl 6 folks. I'm already looking forward to the next time we get to do that!</p><p>Rakudo wise, the work goes on at a good pace. I've been working on lots of bits, and colomon++ has been having quite the patch fest too. The release later this week is a good step forward from last month's release, which was the first after the ng branch was merged. We've won back a lot of functionality again, and this is the first release ever that has some basic support for versioned modules - a direct consequence of the hackathon - and also regexes taking parameters, thanks to a patch from bkeeler++.</p><p>Anyway, that's all for now. Only to note that I expect to be making it to the Nordic Perl Workshop in Iceland in May, and with a bit of luck to the Perl workshops in France and Ukraine too (will take some fun logistics to do the two, but I did miss the French Perl Workshop last year, and I always really love to visit Ukraine, so I'll try and work out a way). Have fun, and I'll try and blog again soonish!</p> JonathanWorthington 2010-03-16T00:48:36+00:00 journal Unpacking data structures with signatures http://use.perl.org/~JonathanWorthington/journal/40196?from=rss <p>My signature improvements Hague Grant is pretty much wrapped up. I wrote a couple of posts already about the new signature binder and also about signature introspection. In this post I want to talk about some of the other cool stuff I've been working on as part of it.</p><p>First, a little background. When you make a call in Perl 6, the arguments are packaged up into a data structure called a capture. A capture contains an arrayish part (for positional parameters) and a hashish part (for smok^Wnamed parameters). The thing you're calling has a signature, which essentially describes where we want the data from a capture to end up. The signature binder is the chunk of code that takes a capture and a signature as inputs, and maps things in the capture to - most of the time, anyway - variables in the lexpad, according to the names given in the signature.</p><p>Where things get interesting is that if you take a parameter and coerce it to a Capture, then you can bind that too against a signature. And it so turns out that Perl 6 allows you to write a signature within another signature just for this very purpose. Let's take a look.</p><p> <code>multi quicksort([$pivot, *@values]) {<br> &nbsp;&nbsp;&nbsp;&nbsp;my @before = @values.grep({ $^n &lt; $pivot });<br> &nbsp;&nbsp;&nbsp;&nbsp;my @after = @values.grep({ $^n &gt;= $pivot });<br> &nbsp;&nbsp;&nbsp;&nbsp;(quicksort(@before), $pivot, quicksort(@after))<br> }<br> multi quicksort( [] ) { () }<br> </code> </p><p>Here, instead of writing an array in the signature, we use [...] to specify we want a sub-signature. The binder takes the incoming array and coerces it into a Capture, which essentially flattens it out. We then bind the sub-signature against it, which puts the first item in the incoming array into $pivot and the rest into @values. We then just partition the values and recurse.</p><p>The second multi candidate has a nested empty signature, which binds only if the capture is empty. Thus when we have an empty list, we end up there, since the first candidate requires at least one item to bind to $pivot. Multi-dispatch is smart enough to know about sub-signatures and treat them like constraints, which means that you can now use multi-dispatch to distinguish between the deeper structure of your incoming parameters. So, to try it out...</p><p> <code>my @unsorted = 1, 9, 28, 3, -9, 10;<br> my @sorted = quicksort(@unsorted);<br> say @sorted.perl; # [-9, 1, 3, 9, 10, 28]<br> </code> </p><p>It's not just for lists either. An incoming hash can be unpacked as if it had named parameters; for that write the nested signature in (...) rather than [...] (we could have use (...) above too, but [...] implies we expect to be passed a Positional). For any other object, we coerce to a capture by looking at all of the public attributes (things declared has $.foo) up the class hierarchy and making those available as named parameters. Here's an example.</p><p> <code>class TreeNode { has $.left; has $.right; }<br> sub unpack(TreeNode $node (:$left,<nobr> <wbr></nobr>:$right)) {<br> &nbsp;&nbsp;&nbsp;&nbsp;say "Node has L: $left, R: $right";<br> }<br> unpack(TreeNode.new(left =&gt; 42, right =&gt; 99));<br> </code> </p><p>This outputs:</p><p> <code>Node has L: 42, R: 99<br> </code> </p><p>You can probably imagine that a multi and some constraints on the branches gives you some interesting possibilities in writing tree transversals. Also fun is that you can also unpack return values. When you write things like:</p><p> <code>my ($a, $b) = foo();<br> </code> </p><p>Then you get list assignment. No surprises there. What maybe will surprise you a bit is that Perl 6 actually parses a signature after the my, not just a list of variables. There's a few reasons for that, not least that you can put different type constraints on the variables too. I've referred to signature binding a lot, and it turns out that if instead of writing the assignment operator you write the binding operator, you get signature binding semantics. Which means...you can do unpacks on return values too. So assuming the same TreeNode class:</p><p> <code>sub foo() {<br> &nbsp;&nbsp;&nbsp;&nbsp;return TreeNode.new(left =&gt; 'lol', right =&gt; 'rofl');<br> }<br> my ($node (:$left,<nobr> <wbr></nobr>:$right))<nobr> <wbr></nobr>:= foo();<br> say "Node has L: $left, R: $right";<br> </code> </p><p>This, as you might have guessed, outputs:</p><p> <code>Node has L: lol, R: rofl<br> </code> </p><p>Note that if you didn't need the $node, you could just omit it (put keep the things that follow nested in another level of parentheses). This works with some built-in classes too, by the way.</p><p>It works for some built-in types with accessors too:</p><p> <code>sub frac() { return 2/3; }<br> my ((:$numerator,<nobr> <wbr></nobr>:$denominator))<nobr> <wbr></nobr>:= frac();<br> say "$numerator, $denominator";<br> </code> </p><p>Have fun, be creative, submit bugs.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2010-02-20T00:21:50+00:00 journal The first release from ng is coming! http://use.perl.org/~JonathanWorthington/journal/40190?from=rss <p>Tomorrow's regularly scheduled Rakudo release is the first one since the long-running "ng" branch became master. It represents both a huge step forward and at the same time a fairly major regression. Internally, the changes are enormous; some of the biggest include:</p><ul> <li>We're parsing using a new implementation of Perl 6 regexes by pmichaud++. It is a huge improvement, supporting amongst other things protoregexes, a basic form of LTM, variable declarations - including contextuals - inside regexes and more. The AST it generates is part of the PAST tree rather than having a distinct AST, which is a neater, more hackable approach. The issues with lexical scopes and regexes are resolved. Closures in regexes work.</li><li>NQP is also re-built atop of this. It incorporates regex and grammar support, so now we run both grammar and actions through the one compiler. It's bootstrapped.</li><li>In light of those major changes, we started putting the grammar back together from scratch. A large part of this was copy and paste - from STD.pm. The grammar we have now is far, far closer to STD than what we had before. Operator precedence parsing is handled in the same kind of way. We've started to incorporate some of the nice STD error detection bits, and catch and nicely report some Perl 5-isms.</li><li>Since the grammar got re-done, we've been taking the same approach with the actions (the methods that take parse tree nodes and make AST nodes). Thanks to contextual variable support and other improvements, a lot of stuff got WAY cleaner.</li><li>The list/array implementation has been done over, and this time it's lazy. There's certainly rough edges, but it's getting better every day. The work to implement laziness has led to many areas of the spec getting fleshed out, too - a consequence of being the first implementation on the scene I guess.</li><li>All class and role construction is done through a meta-model rather than "magic". The Parrot role composition algorithm is no longer relied upon, instead we have our own implementation mostly written in NQP.</li><li>The assignment model was improved to do much less copying, so we should potentially perform a bit better there.</li><li>Lexical handling was refactored somewhat, and the changes should eliminate a common source of those pesky Null PMC Access errors.</li></ul><p>Every one of these - and some others I didn't mention - are important for getting us towards the Rakudo * release. The downside is that since we've essentially taken Rakudo apart and put it back together again - albeit on far, far better foundations - we're still some way from getting all of the language constructs, built-in types and functions back in place that we had before. It's often not just a case of copy-paste; many of the list related things now have to be written with laziness in mind, for example.</p><p>So anyway, if you download tomorrow's release and your code doesn't compile or run, this post should explain - at least at a higher level - why. After a slower December and January, Rakudo development has now once again picked up an incredible pace, and the last couple of week's efforts by many Rakudo hackers have made this release far better than I had feared it was going to be. If we can keep this up, the March release should be a very exciting one.</p> JonathanWorthington 2010-02-18T01:18:33+00:00 journal Catching up: two Rakudo Days from December http://use.perl.org/~JonathanWorthington/journal/40161?from=rss <p>Today plenty happened in Rakudo land - in fact, it was my most active day's Rakudo hacking in quite a while. colomon++ also made some great commits, and between us a lot of things moved forward today. For my part, hashes and pairs are in much better shape.</p><p>I wrote before that I'd got some Rakudo days left to write up; there are two of them, but I'll cover them both in this post, since some of the work crossed the two of them anyway. Here's what I got up to between them.</p><ul> <li>Filled out attribute composition logic for role application. A good chunk of this was written in NQP - in fact, all of the role appliers are.<nobr> <wbr></nobr>:-) Along the way I brought roles up to speed with the attribute part of the meta-object protocol - I'd forgotten that when doing it for classes, though since we couldn't compose attributes at that time it wasn't so interesting anyway. The end result was that we could pass S14-role/attributes.t again.</li><li>The specification states that if in a role you do inheritance, then this is just passed on to the class that the role is eventually composed in to, and added to that class's parents. We never had any support for this in master; with a neat meta-model approach it became rather easier to get it in place in ng.</li><li>Got BUILD/CREATE fixed up a bit and added back support for "has $.answer = 42" style declarations, again through the new attribute sub-protocol.</li><li>Got us handling non-block where blocks again, and added Block.ACCEPTS back - in Perl 6.</li><li>We had various "helpers" to let us do some of the low-levelish stuff in PIR. This is mostly for the places where we need those things in place in order to be able to compile the rest of the built-ins that are written in Perl 6. However, a couple of these helpers knew too much about Parrot and too little about the meta-model, which abstracts it away. So, I re-wrote some of those in terms of the meta-model. Much cleaner.</li><li>Before we relied entirely on Parrot for our "do we do this role" checks. However, given the unfortunate semantic mis-match between Parrot's built-in role support and what we need for Perl 6 (I did try and influence things in a different direction back when we were doing Parrot's role support, but failed), I've been gradually working us towards not relying on those for Perl 6's role support. (In master, it felt to me like we have almost as much code working around the semantics of Parrot roles as we'll need to have to not use them.) Anyways, the divorce isn't quite complete yet, and it's not even a goal for the ng branch. However, I did make a notable step towards it by getting our<nobr> <wbr></nobr>.does checks implemented entirely in terms of the meta-model. In the long run, I'm hoping we may be able to write the entire role implementation in NQP, which helps with the even-longer-run dreams I have of Rakudo having additional backends. But that's for The Future.<nobr> <wbr></nobr>:-)</li><li>Cleaned up and re-enabled sigil based type checking in signatures.</li></ul><p>Thanks to Vienna.pm for sponsoring me to hack on Rakudo, not only for these two days, but also throughout 2009!</p> JonathanWorthington 2010-02-06T22:45:42+00:00 journal The importance of a break http://use.perl.org/~JonathanWorthington/journal/40160?from=rss <p>Several days before Christmas, encouraged by my mum asking, "when you're going to start your Christmas break", I stopped working and hacking on stuff and started relaxing. Until then, I hadn't realized just how tired I was. I slept quite a few ten hour nights in the following week, and had an enjoyable Christmas break. I'd figured I'd maybe take a week or so's break, and then get straight back to things, but a week on I had no motivation or energy to dig in again whatsoever. So, I decided my break would go on through New Years. New Year's celebrations this year involved curry - something I certainly wouldn't mind it involving again.</p><p>Early January brought several days in Sweden, part of planning for an upcoming refactoring of my work/location - there's <a href="http://www.jnthn.net/cgi-bin/blog_read.pl?id=713">details on my personal blog</a>, but the short version is that I've accepted a job at a Swedish startup and will be moving there in March. It's not full time, so I'll continue to have time for Perl 6 development. They know about and, happily, are supportive of my involvement in Perl 6 and my continued attendance of Perl conferences.</p><p>I spent a weekend in Prague on the way home. I did it by train rather than flying, which was enjoyable. It snowed almost my entire time in Prague, and I caught a cold in the following week, but it was kinda worth it to wander around this beautiful city. Didn't bother studying Czech at all, and sorta got by with speaking Slovak, though some folks heard me speak and immediately concluded English would be easier.<nobr> <wbr></nobr>:-) Somehow it kinda felt like I was back somewhere I belonged, even though I'd never been there before. I love central Europe, and excited as I am about Sweden, I know I'll miss this part of the world a lot.</p><p>Anyway, I eased back into some work in January, but mostly took it quite easy. The happy result is that, come February, I'm finding myself recharged and ready to dig back into things again. I got some nice commits done to Rakudo yesterday, and today I meant to, but instead participated on an interesting thread on p6l and did some other useful meta stuff (like this post). Tomorrow should have plenty of hacking time though, and I'm looking forward to it. I also have a couple of blog posts to do about Vienna.pm-funded Rakudo Days I did in December, but never got around to writing up; thankfully I did make notes on what I did on them.<nobr> <wbr></nobr>:-) My main focuses from here on will be on:</p><ul> <li>Continuing to get Rakudo's ng branch into shape - we'll make it master soon. A lot is missing, but things are going back fast and often very neatly. It's easy to focus on what it doesn't yet do that master does, but it has many things right that master does not - now including laziness!</li><li>Finishing up my signatures grant. I really, really want to do that within the next couple of weeks.</li></ul><p>Anyway, that's what's been up with me. If you take away anything, it's that you may not realize how much you need a break from something until you take it, and if it's not the only thing putting food on the table, then it's probably better to take the needed amount of break and come back revitalized. I guess the other option is to dig back in regardless, but I suspect that's the path to burnout, something I'm quite keen to avoid.</p><p>More technical blabbering here soon.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2010-02-06T00:31:49+00:00 journal Attribute sub-protocol and other role bits http://use.perl.org/~JonathanWorthington/journal/39997?from=rss <p>Work in the ng branch continues; this post is an update on some of the work I've done on a slightly broken-up Rakudo Day (things have been a tad disrupted while I've come over to England for the month).</p><p>Perhaps the most exciting bit of news is that I've done an implementation of the attribute meta-protocol. The main upshot of this is that we now don't hard-code the creation of accessors, but rather call a "compose" routine on the Attribute object - the meta-object that represents an attribute - and it does the accessor creation. Amongst other things, this opens the door for custom accessor generation, which I believe is something that many Moose extensions do, and so will be useful for Perl 6 meta-programmers. As a bonus, the Attribute class is implemented in NQP, not PIR.</p><p>I've also continued working on roles. I'm happy to say that, unlike in master, we implement the does and but operators - which do runtime mix-ins of roles - in Perl 6 now. They don't actually do much - the heavy lifting is done by the RoleToObjectApplicator class. That one is implemented in NQP, with only one small Parrot-specific bit. It feels great to be writing these bits in something higher level, and - importantly - means they're all building on a fairly small set of primitives rather than having deep magic everywhere.</p><p>In other smaller bits of news, I got the dotty form of<nobr> <wbr></nobr>.= working, and also implemented the<nobr> <wbr></nobr>.^ dotty operator too (which makes metaclass calls prettier).</p><p>All in all, I'm fairly happy with how things are shaping up. I'm hoping another Rakudo-day sized push and we'll be able to run many of the roles related tests again, but this time atop a much cleaner and more correct implementation, a large part of which is in Perl 6 (or NQP, which is a subset of Perl 6). Thanks to Vienna.pm for sponsoring this work.</p> JonathanWorthington 2009-12-10T10:15:23+00:00 journal Laying the role meta-foundations http://use.perl.org/~JonathanWorthington/journal/39943?from=rss <p>I got back from Latvia yesterday. The first Baltic Perl Workshop was very small, but also very enjoyable. I also very much enjoyed meeting up with masak++, who shares my love of good food - curry included! Riga turned out to be a great place for having a good nom, both in terms of local stuff and nice hot curry. And of course, amongst the workshop and some sightseeing, some hacking got done too.</p><p>Today was Rakudo Day, and it was time to get back to meta-model matters. I actually spent a while during my time in Riga reading a good bit of Moose and Class::MOP - I'd seen bits of them before, but this time gave the parts relating to roles a much closer look and got myself a refresher on some other bits. I also spent the plane ride home (pretty much the entire flight!) reading an interesting paper on <a href="http://scg.unibe.ch/archive/papers/Duca05ySafeMetaclassTrait.pdf">composing meta-objects from traits</a>, which turned out to be a much easier read than I had expected, but left me wanting to read many of its references too.</p><p>The first thing I worked on today was getting more details into <a href="http://github.com/rakudo/rakudo/blob/ng/docs/metamodel.pod">metamodel.pod</a>, which is the currently under heavy construction specification for Rakudo's meta-model. It will in the future become a proposal for inclusion in the Perl 6 specification too, at which point it'll be time to co-ordinate with other implementations to get something that we can all agree on. Anyway, if you have an interest in meta-programming, you may be interested to give it a read and leave comments, or catch me on #perl6. For those just wanting to see some code rather than read a spec, skip straight down to the example of how writing a module to support something AOP-ish could look. It'll no doubt evolve somewhat, and we won't be able to run it for a little bit yet, but I think it's a nice example of how easy it should be to implement such things in Perl 6.</p><p>After that, I dug into starting to get some implementation work done, with a focus on working towards having role composition working as spec'd in the metamodel document. So far I've got a lot of the foundations laid, and I'm now starting to build up on them. So far, it's mostly just plumbing, but composing a role into a class and the class then gaining the role's methods does now work, and a bunch of what we need for required methods and conflict resolution is stubbed in, though untested. I also was able to write a chunk of this in NQP rather than PIR, something I'd like to see us increasingly do in the stage 1 compiler, where it makes sense to do so (the stage 1 compiler containing the things we need in order to compile the setting; the smaller we can keep this part, the better it'll be for us overall...for now the goal is mostly getting the ng branch landed rather than small-stage-1-nirvana though<nobr> <wbr></nobr>:-)). Anyway, I think I can probably progress pretty fast from here and get our role stuff up and running in the next couple of days.</p><p>Thanks to Vienna.pm for sponsoring Rakudo Day.</p> JonathanWorthington 2009-11-25T01:03:13+00:00 journal Progress and minor frustration http://use.perl.org/~JonathanWorthington/journal/39917?from=rss <p>This is a summary of what I did on my Rakudo day on Monday as well as other little bits of random hacking since then on the Rakudo ng branch.</p><ul> <li>Corrected dispatches on proto-objects, and along the way fixed a few things that were quite likely bugs in master.</li><li>Got signature introspection back in place.</li><li>Switched back on @ requiring the Positional role and &amp; requiring the Callable role when binding parameters.</li><li>Got parallel dispatch back in place, and we parse it a like STD now.</li><li>Added back parsing of fatarrow, and its ability to be used to supply named parameters</li><li>Added back submethods.</li><li>Made<nobr> <wbr></nobr>.= parse and work again; we almost certainly need to re-visit the way I did it, though, since while I didn't find a better way yet, myself and Pm both agree the current way is not ideal.</li><li>Put back Inf, -Inf, +Inf and NaN, which parse as literals.</li><li>Brought<nobr> <wbr></nobr>.arity and<nobr> <wbr></nobr>.count back over from Rakudo master and simplified them a bit along the way, thanks to other refactors.</li><li>Get placeholder variables parsing and mostly working again.</li></ul><p>So, a whole grab back of little bits here and there. Things continue to come together nicely, but a source of some frustration is that our passing test count remains low. The issue here isn't that we aren't adding back lots and lots of things, but that the test files often require features aside from the things they're actually testing. This is part and parcel of having a Perl 6 test harness - indeed, one issue we have is that we didn't put back eval yet, so eval_dies_ok tests can't yet be run. But another issue is the temptation for test writers - and I'm just as guilty as anyone - to write tests using Perl 6 language features unrelated to the thing being tested themselves. For example, many of the tests for signature introspection rely on the parallel dispatch operator. The test file itself is beautiful and elegant, but unfortunately I'm still working towards putting back things we need - even though I have signature introspection back in place - because of other features it just assumes. So I expect we'll hit a point where we very suddenly get back loads of tests - the problem is that in the meantime, we don't have as much testing to tell us of regressions as would be ideal.</p><p>OK, time for me to fly to Latvia, for the first Baltic Perl Workshop, now!<nobr> <wbr></nobr>:-) Thanks to Vienna.pm for funding this work.</p> JonathanWorthington 2009-11-19T10:41:38+00:00 journal Bringing back the setting, and bug hunting http://use.perl.org/~JonathanWorthington/journal/39904?from=rss <p>This is a report on what I did on my Rakudo Day on Friday, but didn't manage to finish writing up until now.</p><p>It was time to switch focus from building up metamodel primitives in the Rakudo refactor for a bit and dig into the work of bringing back a bunch of the Perl 6 setting - the built-in classes and functions that are implemented in Perl 6. In doing so, I knew I'd discover lots of other little bits that would need putting back, which would guide my efforts. I chose to work towards bringing back Any-num.pm first - various bits relating to numbers. Preparing to do that led me to fix up a whole raft of other bits.</p><ul> <li>Needed to bring the way we parsed method declarations more in line with STD.pm.</li><li>Needed to update the identifier rule to match STD.pm so we could have identifiers with hyphens in.</li><li>Got basic support for when/default blocks in place; mathw++ had already done given blocks.</li></ul><p>After that, I could also bring Int.pm back into the setting. That went pretty smoothly, though needed a few tweaks since subs have "my" scope by default in Perl 6 now, something that we never updated master to handle, but have got right from the start in the ng branch. Adding that in allowed me to get rid of a few of the "cheats" that we'd had in place beforehand.</p><p>Continuing on the theme of number handling, my next act was to try and bring back Rat and Complex. Rat went in with a bit more hassle. I discovered that I'd managed to make a mess of declaration order mapping between source and the PAST tree we built. Happily, once I understood the problem, the fix was only a few lines of patching. However, then I hit on something else weird: we seemed to be calling wrong multi-candidates.</p><p>I was sure I'd made a mistake in the code-gen...but no, it all looked right. Bug in the signature generation? Nope, those looked good too. It wasn't going to be a multi-dispatcher bug really, given that this had been brought in practically unchanged from master - or at least, without any notable changes. After a lot of searching I discovered that we really weren't entering the multi-dispatcher at all, but instead always jumping to the same candidate - but only in other code in the setting, not in stuff done in the REPL. Realizing it was only an issue with code in the same compilation unit was a big clue. Due to changes in the way we're doing code generation now - in order to not create so many bogus objects of the wrong type at startup that we only then have to re-bless - a nasty Parrot issue had shown up to crash the party. Worse, while I'd call it an optimization bug, pmichaud++ correctly pointed out that a lot of existing code was almost certainly going to be relying on things working this way - the optimization was providing a feature, pretty much. Thus my proposal to just disable it wasn't quite going to fly. This issue still somewhat remains "to solve", but I'm hopeful we'll get some solution of some sort today, as I suspect it's going to become a bit of a serious blocker otherwise. Blocker as in, we probably can't pass any of the multi dispatch tests before it's fixed, even though the dispatcher itself is likely fine. What a PITA.</p><p>After hunting that down, I only really had time to get Complex.pm back in. There was much commenting out still, but that will guide me on what I needed to hack on next. And happily, having put these bits back led colomon++ to hack away on them a bit over the weekend. Thanks to Vienna.pm for sponsoring my work on Rakudo.</p> JonathanWorthington 2009-11-16T11:47:13+00:00 journal Starting to get over the hump http://use.perl.org/~JonathanWorthington/journal/39873?from=rss <p>So, a new week is here, and hopefully this will be a week of lots and lots of progress on the <a href="http://github.com/rakudo/rakudo/commits/ng">ng branch of Rakudo</a>. We've made a pretty good start, and things are just about starting to hit the exciting stage now where we're starting - piece by piece - to bring back the Perl 6 setting. In there now we already have cases of operator and trait modifier definitions and well as some initial class declarations. Basically, that means that we've got a lot of the fundamentals in that we need for the rest of the setting. A lot of the bits we are still missing aren't actually that hard either. In fact, it's starting to feel like we might be getting to the point where this refactor is nearly "over the hump" and we can start winning back passing spectests quickly.</p><p>I continue to be happy with how things seem to be overall much cleaner now in the grammar and actions, as well as the general organization of the code base. My focus has been mostly on continuing to get things related to the object model and the type system back in place. Some of the highlights have been:</p><ul> <li>Adding the augment scope modifier, which is for adding extra bits to a class definition. This is especially useful when we want to define most of a class in Perl 6, but need to have some of the low-level bits available in the stage 1 compiler, which we use to compile the setting. This hadn't used to be a scope declarator, but rather a compiler-handled trait; we've regressed on various tests for quite a while since the spec changed, but we'll be able to have those back when the branch lands.</li><li>Getting multi-methods back in place again. This wasn't too much work at all, and I'm quite happy with the end result, which no doubt does some better error detection. We use these in the setting heavily, so they - along with augment - were another big roadblock removed.</li><li>Getting sub traits back in place, which unblocked pmichaud++ to work on export things. I'll leave him to blog the details, but we've made some nice wins there, in terms of handling things better than we have before.</li><li>Being able to pretty much copy-paste the error handling for bad type names in scoped declarators from STD.pm into Rakudo's grammar. STD is really pretty good at giving decent diagnostics when things go wrong, and with the new grammar engine we're being able to bring a bunch of these into Rakudo. "In "my" declaration, typename Monkey must be predeclared" is such a nicer error to get than the rather less informative "Malformed declaration", which doesn't even tell you what sort of declaration it was, let alone what was wrong with it.</li><li>Adding back a couple of things that we were writing in PIR before but with Perl 6 implementations this time. Strangely enough, they're much nicer to look at.<nobr> <wbr></nobr>:-)</li></ul><p>Thanks to Vienna.pm for sponsoring this work. I'm now done with catching up the Rakudo days I didn't do in September while wandering around Asia, so now it's into the ones I didn't do in October. Given how much there is to do, I'm glad I've had a nice store of them to draw on at this point!<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2009-11-11T00:23:00+00:00 journal More Rakudo ng hacking http://use.perl.org/~JonathanWorthington/journal/39856?from=rss <p>I've spent most of my waking hours today (and a bit of yesterday) starting to build things out on top of the metamodel foundations in Rakudo's ng branch. It's feeling much cleaner that what we had before, and I've not found anywhere yet where it's feeling like I'm having to stick magic into the actions that is specific to a certain type of package. In fact, compared to before, the package_def action method is really quite compact and clean. The whole process of constructing a class (or role or grammar) is just a series of method calls on the metaclass.</p><p>Here's a list of things that I've got done since the last blog post.</p><ul> <li>Got classes inheriting from Any again by default, which took me on a detour into getting the add_parent method of the metaclass working. We can't quite do inheritance again yet though - that needs traits to be completed too.</li><li>Added handling of the grammar package declarator back in. Now the "inherit from Grammar by default and not Any" stuff is just done by us having a metaclass for grammars - GrammarHOW - that subclasses the metaclass for classes - ClassHOW. It simply overrides the compose method (what we call when we're finished defining a package) and checks if there are any explicit parents, and if not adds Grammar as a parent, then delegates up to the superclass to do the rest of the work.</li><li>Got us parsing trait modifiers and building AST for them. We have protoregexes now, so I could make it just pretty much just like STD. There were also some tweaks to synchronize with. Much of it was just brining code in from the existing actions and re-organizing it a bit.</li><li>One I had those in place, I got attribute accessor generation done. I had wanted traits so I could straight away add the "is rw" variant too. Again, the accessor method is added thorugh the metaclass, and this time it got a proper signature (I suspect there was probably a bug in the previous implementation to be found - maybe even a ticket - over this).</li><li>In doing the above, I realized that I needed to sort out handling of the invocant. Before, we were having Parrot handle it for us, which meant it was handled out of band with respect to the signature binder. In ng we just handle it like any other parameter, apart from we also make it available through "self" (or only make it available through self if the invocant is not specifically requested). This fixes the "can't put where clauses on the invocant" issue that came up a while back - now (in theory at least - didn't test it) you'll be able to write a bunch of multis that dispatch based on constraints on the invocant.</li><li>I started preparing to put roles - parametric ones too - back in place, again built around the metamodel approach. I'm hoping things get cleaner again as a result of that.</li><li>Helped Patrick with working out why compilation is so painfully slow. I timed all of the stages, and it turns out - at least on my box - that we're spending about 10/11ths of the time in the very final stage, turning POST (a tree representing the Parrot opcodes) to PIR. Since that just walks a tree and builds up a string, that's all rather strange. pmichaud++ is investigating.</li></ul><p>Basically, my aim is to get us able to start bringing the Perl 6 setting back as soon as possible. Today Patrick did the infrastructural work for that, which was a great start. I was close to us having inheritance working by writing a trait modifier in there, but then realized that while lexical subs now work great (like, way cleaner than in master), package ones don't. Fail. Ah well, it's 3am and I need sleep. Thanks to Vienna.pm for sponsoring this chunk of work.</p> JonathanWorthington 2009-11-07T01:57:27+00:00 journal Relaying the metamodel foundations http://use.perl.org/~JonathanWorthington/journal/39844?from=rss <p>It's taken me pretty much a whole Rakudo day to figure out how to go about the package declarator support in the ng branch, but I think it's been worth it. The result is not only a design, but some initial code that starts to re-build our classes and modules support in terms of it.</p><p>There are a couple of big things to take away from this. The first is that we're going to have the vast majority of things in place for custom meta-classes when the ng branch merges. This is something we were rather more distant from in master, and something that people were hoping we'd get in place by Rakudo *, so I'm glad that we're going to be getting most of the way there on this. The really big deal here is that we're getting rid of a lot of stuff where things "just worked somehow" and re-building it all on top of calls to the meta-class. We're re-building how stuff happens with the new grammar and actions anyway, so it is a fitting time to get it in place.</p><p>So what are your options if you want to write a custom metaclass? First, you go and write the class - possibly subclassing one of the existing built-in metaclasses if you just want to do some small customization (for example, I expect GrammarHOW, the metaclass for grammar packages, will just be a very simple subclass of ClassHOW that at class composition time checks if we have a parent and if not makes Grammar the default parent rather than Any, then delegates to the super class). Second, you declare a sub-language and introduce a new package declarator to the grammar; if we want to support "controller" we'd add something a rule to our sub-language like:</p><p> <code>token package_declarator:sym&lt;controller&gt; {<br> &nbsp;&nbsp;&nbsp;&nbsp;:my $*PKGDECL<nobr> <wbr></nobr>:= 'controller';<br> &nbsp;&nbsp;&nbsp;&nbsp;&lt;sym&gt; &lt;package_def&gt;<br> }<br> </code> </p><p>Note that you're actually adding a new type of package to the language here that will be associated with your metaclass. So how do we make this association? There's a context variable %*HOW which contains the mapping from the value set in $*PKGDECL to the name of the metaclass to use. You'll notice this means that we could actually just want our controller keyword for documentation, and just map it straight through to ClassHOW, and not write a metaclass, or we can associate multiple package declarators with the same metaclass.</p><p>While this covers the common cases, there's a few times where we need to do some slightly different code generation in terms of different packages. There's some hooks for this too, and while those will not be spec, but rather Rakudo specific, it does mean that the overall implementation will get a lot cleaner, with the Actions.pm file that maps parse tree to abstract syntax tree now not having to know the details of different types of package.</p><p>Already it feels a lot cleaner with the few bits that are in place so far, and I think as we start to build out on top of this, it's going to lead to something that overall is a lot less magical, and a lot more maintainable and extensible, than what we had before. All of which make a pause for thought, rather than a charge into the code, very much worthwhile. Thanks to Vienna.pm for making it possible for me to take the time to do these things.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2009-11-05T00:59:36+00:00 journal A productive weekend in Rakudo land http://use.perl.org/~JonathanWorthington/journal/39829?from=rss <p>I wanted to get a little rest and seeing friends in this weekend alongside helping move <a href="http://github.com/rakudo/rakudo/commits/ng">ng</a> along - that is, the branch where we're re-building Rakudo atop of the new regex engine. Thus I took a Rakudo day and split it up over the two days, yesterday and today, so I could get a nice combination of work and play. Here's some of what I've been up to; I expect Patrick will blog later on today some of the outstanding work he has been doing too.</p><p>I guess the work we've done over this weekend could be divided into two overlapping tracks. Patrick has taken on a lot of the language fundamentals and had a key focus on us being able to run Test.pm again, which will open up the rest of the sepctest suite to us. It's currently looking fairly likely that we'll reach that milestone within the next 24 hours, which is amazing progress. I've also been doing some things that will help us on the way to that, but my focus has on getting some key infrastructure in again so we can soon start bringing parts of the Perl 6 setting back into play.</p><p>One of the things I had to get back in place was type name handling, so we can differentiate our types from our other names. That was a lot neater this time, since it could just go in much more STD-ish as an assertion - written in NQP rather than PIR (there's a lot of this now). After that, it was easy to get typed signatures parsing again. I fixed up our name parsing along the way a bit too - the NQP grammar did something simpler, whereas we needed something a bit more complex for Perl 6.</p><p>One of the big changes we've had in from the start - in order to bring ourselves in line with the spec - is that subs are lexically scoped rather than package scoped by default. After some pondering, I realized that we could handle lexical multis in a far, far nicer way than we had been doing in the past, so I got that implemented. In fact, I'm now expecting I can follow the same pattern for methods too. It's a lot, lot cleaner than what we were doing before, and additionally avoids a lot of start-up reblessing, which was one of our big startup costs before. We really are taking the opportunity to apply our past experience and the new grammar engine features to build the grammars actions up in a much neater way, and I'm really happy with the results so far.</p><p>One of the things Patrick has been working to get in place is the Parcel data structure. Along with that, we've been reshaping our approach to issues like flattening. I've mostly been involved with that at the level of discussing the approach with Patrick, and he's led the implementation charge on it. Today he hit on the need for a small tweak to the method dispatcher, which was familiar ground for me, so I got it patched up. A silly fail on my part later, it was all working rather nicely. Finally, I got some pieces of roles patched back in to the new build.</p><p>One thing I've been preparing for is getting our class and role declaration support back in place. I'm planning to approach it in a rather different way than we have been up until now, the motivation partly being neater and more maintainable code, but also getting us on track to handle custom meta-classes. I'm not saying we'll actually have those when the ng branch lands, but equally I'm aiming for us to be a whole bunch closer. The other big difference in approach here is that I'm aiming for us to be able to keep a lot of the package information trivially introspectable during the building up of the class or role, which will make some things much easier and better encapsulated. Further, we can also keep that information around for assisting a future optimizer/type checker. (As an aside, I've been arranging signature things in a similar way during the course of my signatures grant. I guess the general principle is that while PAST is pretty much sufficient for expressing the semantics of the program, sometimes it's desirable to keep some higher level information around for entities that don't have a specific type of PAST node dedicated to them - mostly because languages tend to care about them in very different ways, so they don't belong in PAST anyway.)</p><p>Thanks to Vienna.pm for funding my work at this important stage in Rakudo's development.</p> JonathanWorthington 2009-11-02T01:05:58+00:00 journal Rakudo Day: Starting to put Rakudo together again http://use.perl.org/~JonathanWorthington/journal/39821?from=rss <p>Recently Patrick wrote about some <a href="http://use.perl.org/~pmichaud/journal/39779">major upcoming changes to Rakudo</a>, which will be a huge step along the way towards Rakudo *. If you didn't read Patrick's post, I suggest you do so first and then come back to mine - it gives a lot of detail on why we're doing what we're doing at the moment. Anyway, after much hard work - on Patrick's part getting nqp-rx ready and on my part getting us building and running on top of the updated Parrot Calling Conventions - today we were finally ready to start on step 7 of the plan. Essentially, that involves a complete re-do of Rakudo's grammar and actions.</p><p>nqp-rx was already in a pretty awesome state at the start of the day. It was missing a make install target, which is the first thing I worked on. With that in place, I was curious to see if it could build the current Rakudo NQP files. Given that it was a ground-up re-implementation of NQP on top of a new implementation of the regex engine, which had changed in many ways, I was really quite impressed how little effort it took to build Rakudo's NQP parts on top of it and have us make it through the test suite mostly passing. This wasn't actually the goal - it was well beyond what Patrick and I had guessed we would manage - but it did shake out a couple of missing features in nqp-rx, which we were able to add quickly between us, as well as a bug, which I fixed. It was nice to have them out of the way early on, before we dug into really using nqp-rx to re-build Rakudo on top of.</p><p>With those fixes in place, Patrick then ripped a bunch of stuff out of the Rakudo makefile and added the NQP grammar and actions as a base for us to start building up from again. This means we've gone back to having something that won't even run the sanity tests completely yet. However, what's there so far is also far, far closer to STD than we were before, using protoregexes in various places as well as contextual variable declarations inside regexes and an operator precedence parser alike the one from STD. Certainly, the bits that I have put back in so far have gone in far, far cleaner than they appear in current Rakudo.</p><p>Unlike NQP, which just uses Parrot's default signature binder, Rakudo running on the new grammar just from today's efforts is already switched over to building Perl 6 signatures and is using the signature binder I've been developing as part of my current Hague Grant. Multi-dispatch and operator parsing should hopefully be landing tomorrow - since this is just a re-write of the way we parse and build up the AST, rather than all of the other guts and the setting - we're being able to put back major features in a short time-frame. All of that said, we're not shying away from using this as an opportunity to deal with other a couple of other refactors we've been wanting to do for a while along the way. Generally, they are fairly fundemental things that would have been quite a pain to deal with individually, and are easy to get in as assumptions from early on as we build things up again.</p><p>This is the first Rakudo day I've done in a while - I was wandering around Asia for a month, and then have been focused on my grant. That means I've got a nice little stash of Rakudo days to spend during the next couple of weeks, which will be great in getting Rakudo back on its feet, more featureful and stable than before, and with a grammar much closer to STD. Thanks to Vienna.pm for funding Rakudo Days, and I'll keep you posted on how things are progressing in the <a href="http://github.com/rakudo/rakudo/commits/ng">ng branch</a>.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2009-10-31T01:29:06+00:00 journal Signature introspection http://use.perl.org/~JonathanWorthington/journal/39810?from=rss <p>One of my Hague Grant tasks was to get a signature introspection interface into the specification and get Rakudo conforming to it. While there may still be some minor tweaks to come, this part is essentially done now. You can read the spec additions in <a href="http://svn.pugscode.org/pugs/docs/Perl6/Spec/S06-routines.pod">S06</a>.</p><p>First off, you can call<nobr> <wbr></nobr>.signature on any block to get hold of its signature object - an instance of the Signature classs. Calling<nobr> <wbr></nobr>.params on a Signature object will in turn give you a List of Parameter objects, which describe each of the parameters specified in the signature. We are able to introspect the types, the names, whether the parameter is optional, get a closure that returns any default value and more. Here's an example.</p><p> <code> sub example(Int $x, Str $y?) { }<br> my $sig = &amp;example.signature;<br> for $sig.params -&gt; $param {<br> &nbsp;&nbsp;&nbsp;&nbsp;say "Name: {$param.name}, Type: {$param.type}, Optional: {$param.optional}";<br> }<br> </code> </p><p>This gives the output:</p><p> <code>Name: $x, Type: Int(), Optional: 0<br> Name: $y, Type: Str(), Optional: 1</code> </p><p>To make sure that all of the information was really there and accessible, I then went and re-implemented the<nobr> <wbr></nobr>.perl method on a Signature in Perl 6 in the setting, using the Parameter objects. The new version lacked several of the bugs that the previous version had, which cleared up a few RT tickets. And finally, moritz++ wrote a bunch of spectests for signature introspection, following the new spec.</p> JonathanWorthington 2009-10-27T17:45:55+00:00 journal The new Rakudo signature binder has landed http://use.perl.org/~JonathanWorthington/journal/39772?from=rss <p>Since I got back from my travels in Asia - actually, as soon as I'd got a good night's sleep after the long flight - I've been busily hacking away on my latest Hague Grant. In fact, I've done rather little else. After a couple of weeks of work - with many late nights debugging - I've finally reached the first major milestone of the grant: the new signature binder has landed.</p><p>So what is a signature binder anyway? It's the thing that takes the signature on a block (or some more interesting type of code object, such as a sub or a method) along with the arguments that it was invoked with and - if possible - binds them against the signature, such that the arguments passed end up in the correct variables. It sounds relatively simple, and in some senses it is. However, Perl 6 signatures offer a lot of powerful features, making things rather less trivial.</p><p>If you've been using Rakudo before now, you'll probably have been writing things with signatures and passing arguments along to them quite happily, and be wondering, "why a new binder?" It's a fair enough question, and the simple answer is because we'd reached the point where the approach Rakudo had taken so far had gone as far as we could sensibly take it. We've often used the defaults provided by Parrot in the past to allow us to make progress on developing a useful compiler; we did so with multiple dispatch, method dispatch, objects, roles and so forth. The approach has paid off: people are writing cool stuff with Rakudo today (I played a board game online with masak++ earlier today; the game itself was implemented in Perl 6, used the Web.pm framework and was being served by a web server written in Perl 6 too). In the meantime, we've been replacing those Parrot defaults with things that get the more subtle and/or powerful parts of the Perl 6 specification correct, and usually that do so in a more efficient manner.</p><p>The main things that prompted development of a custom binder were:</p><ul> <li>While Parrot could bind the arguments to registers, we then had to follow that up with code to put those into lexical variables. We then had to make a second pass over the arguments to do type checks and enforce context. This was inefficient - we had to do a lot of extra lookups as well as two passes over all of the arguments. It was really also quite bad if we were just binding the signature because we had a multi-dispatch candidate that needed a bindability check: we'd have already got a lot of the work done before we could figure out that actually, the first argument had a constraint on it that immediately ruled it out.</li><li>Perl 6 allows binding of named parameters to positional variables - that is, even positional variables can be passed as named parameters too. Parrot didn't support this, and was going to be really quite hard to layer onto the existing model we had. Doing it efficiently was fairly out of the question. Getting Parrot to support it was also an option, but bulking up the VM for a feature only one language really wants is not really optimal.</li><li>If multi-dispatch had already decided that a candidate's types matched those of the arguments being passed, there wasn't a good way to avoid the argument type checks. At the same time, we had to be sure to enforce them if a multi candidate was somehow invoked without going through the multi-dispatcher.</li><li>If multi-dispatch depended on a bindability check, we'd like to just keep the candidate around, and not re-bind everything. There wasn't an easy way to do that either.</li><li>Nor was there a particularly easy way to implement nested signatures. These allow powerful functional-style matching as well as deep data structure unwrapping.</li><li>Signature binding in Perl 6 isn't just done when calling a routine. For example, you can use a signature to bind and unpack return arguments too. These return unpack signatures can be just as complex as those used when calling a routine. Heck, you can even have a signature literal and just smart-match against it to attempt binding. We needed something that was going to support these use cases too, which argued against having Parrot doing half the work and us patching it up later.</li></ul><p>The new signature binder I have merged into master this evening either supports or is designed with later supporting all of these needs in mind. It handles binding named arguments to positionals right out of the box, for example. So now if you declare a sub:</p><p> <code>sub plot($x, $y) { }<br> </code> </p><p>Then you can call it as any of:</p><p> <code>plot(4,2);<br> plot(x =&gt; 4, y =&gt; 2);<br> plot(4, y =&gt; 2);<br> plot(2, x =&gt; 4);<br> plot(y =&gt; 2, x =&gt; 4);<br> </code> </p><p>Under the new signature binder, it was also trivial to add the code to avoid re-doing the type checks for a multi-dispatch candidate. While I didn't yet add the "don't re-bind when we did already passed a bindability check" improvement, that's mostly because it requires a slightly less trivial refactor to the multi-dispatcher - there's some stuff stubbed in to support it when I get that done though. And also there's a stub for nested signatures, which will be coming along later. Those are going to be way cool.</p><p>A couple of other things came up during the refactor that also led to some on-the-side bonus improvements. The most immediately useful one is that lexical variables declared outside of packages are now visible inside them - a source of much past frustration.</p><p> <code>my $dips = 0;<br> <br> class Chip {<br> &nbsp;&nbsp;&nbsp;&nbsp;method dip() {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;$dips++;<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> }<br> <br> my @chips = Chip.new, Chip.new, Chip.new;<br> @chips&gt;&gt;.dip for 1..10;<br> <br> say $dips; # 30<br> </code> </p><p>I also fixed the bug with junction auto-threading and for loops, so if the signature of your for loop declares a non-junctional type then you'll get the body run multiple times, once for each thingy in the junction. Also, signatures can now be written as literals, without getting weird errors.</p><p>Finally, I mentioned that one goal was getting better performance. I've not yet started to optimize the new binder - yes, it has some built-in by design performance improvements, but I didn't really dig in to the real effort of making it faster yet. Even so, the gains are notable. The figures below are the percentage of time it now takes to run some micro-benchmarks on specific language constructs compared to before I started working on the signature and binding improvements. So for example, 10,000 method dispatches now take about 26% of the time they used to.</p><p> <code>Startup: 76%<br> 10,000 sub dispatches: 45%<br> 10,000 multi dispatches: 35%<br> 10,000 method dispatches: 26%<br> 10,000 multi-method dispatches: 20%<br> 10,000 operator dispatches: 33%<br> postfix:&lt;++&gt; 10,000 times: 57%<br> </code> </p><p>These figures sure don't mean that Rakudo is fast yet - we've a long way to go until I'd say that - but I think for the "we didn't start optimizing the new binder yet" phase, these are an encouraging start.</p><p>More as the grant progresses. In the meantime, enjoy the improvements.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2009-10-19T23:34:20+00:00 journal Backpacking done, now back to Rakudo http://use.perl.org/~JonathanWorthington/journal/39726?from=rss <p>I'm just back from my yearly autumn backpacking trip, and once again I took in a few Perl things amongst my vacation. I attended YAPC::Asia for the first time, and gave a couple of talks; you may be interested to see <a href="http://www.jnthn.net/articles.shtml">the slides</a>. I think they were both quite well received, though one of them had a couple of bugs. I muchly enjoyed staying with the Pauleys - thanks for the hospitality...and introducing me to tonkatsu!</p><p>Other than Japan, I also visited South Korea. I gave a talk at Seoul.pm, and many of the Perl mongers there were kind enough to take time out to show me around and have dinner and beers with me, which was really great. As well as great Korean hospitality, I can't not mention that Korean food is SO good - I'm missing it lots already.</p><p>So, it was a great time (thanks to everyone who made it so - too many to mention here), and now I'm back, re-energized and ready to dig back into hacking on Rakudo. In fact, this is my last real vacation before Rakudo *, so I made the most of it. The next few months are going to be a serious amount of work - but never fear, I'm hacking on it already.<nobr> <wbr></nobr>:-)</p><p>My signatures Hague Grant got approved. I've started on that already, and I'll try and blog some progress in the next few days (for now I've done nothing more exciting than some refactors in preparation for the changes that will really make a difference...but with my current uncommitted bits they seem to shave roughly 10% off Rakudo startup time anyway, and that's before I begin the changes I expect to make more of a difference.) Also, during a Rakudo day before my trip and some hacking at YAPC::Asia, Blizkost made some more progress. I'll blog about that soon too.<nobr> <wbr></nobr>:-)</p><p>OK, back to hunting down bugs in my latest changes...</p> JonathanWorthington 2009-10-07T09:33:42+00:00 journal Starting on Perl 5 interop for Rakudo http://use.perl.org/~JonathanWorthington/journal/39530?from=rss <p>I used the first chunk of my Rakudo day today to kick off a new project. At YAPC::EU, we had the Rakudo BOF where we collected together suggestions of features that mattered to people for Rakudo * - the major, usable Rakudo release we're planning for Q2 2010. One that came up was interoperability with Perl 5. This is something that I think had generally been assumed as something far off for Rakudo, or at least not really discussed in any great deal. The legacy is hardly great: the Ponie project, which tried to re-target Perl 5 to the Parrot Virtual Machine, was put out to pasture after plenty of effort for a variety of reasons, including Parrot being too much of a moving target at that point.</p><p>As far as I can see, re-targeting Perl 5 would be really quite a lot of work, and require a lot of guts knowledge. A lot of work in a specialized area presents a resourcing problem, and while I would happily cheer on a renewed effort to try and port Perl 5 to run on Parrot, I'm not at all expectant that anyone is going to jump in to such a task soon. Even if that happens, the chances of having something ready for Rakudo * seem to me fairly slim.</p><p>So, enter <a href="http://github.com/jnthn/blizkost/tree/master">Blizkost</a>. The Blizkost project is aiming to embed the Perl 5 interpreter and then build various bridges between it and Parrot. We'll start with the easy things - today I've done the easiest thing - and then build out from there. Rather than this being something built into Rakudo, instead Blizkost operates at the level of any other Parrot language implementation, meaning that it supports the HLLCompiler interface. This means that any other Parrot language that knows how to use this interface to eval code in another language, for example, will be able to make use of this.</p><p>The simplest thing I figured we could do is make it possible to eval a string of Perl 5 code. You can't get the value it returns, nor will it find any outer lexicals, but this means that you can now do (copy paste from the REPL):</p><p> <code>&gt; eval('print "Hello from Perl 5\n"',<nobr> <wbr></nobr>:lang&lt;perl5&gt;)<br> Hello from Perl 5<br> </code> </p><p>Rakudo itself needed no changes to deal with this (other than a minor fix to deal with a language inter-op regression that affected eval in any language). All I had to do was to configure, make and make install Blizkost, using the same installed Parrot as Rakudo is built against, and it Just Worked.</p><p>So, we have a starting point. It's a small one, but then I only spent several hours on this so far.<nobr> <wbr></nobr>:-) I really have little knowledge of Perl 5 guts, so my dream situation is that now I've seeded this, people will come, look in horror at what I've done and find themselves helplessly patching it. It's <a href="http://github.com/jnthn/blizkost/tree/master">on GitHub</a> so you can go fork it if you want to hack, and I'm also going to be giving out commit bits quite freely, so get in touch if you feel like joining in.</p><p>I spent remaining bit of my Rakudo day dealing with some bugs.</p><ul> <li>Fixed code generation bug when you tried to bind an attribute. Added a spectest.</li><li>Stopped a sprintf that is given too few arguments from exploding with a Null PMC Exception. Now it just returns a Failure. Another couple of spectests started passing.</li><li>Finally figured out how to successfully catch attempts to get hold of attributes on a type object. You can't, and you got a Null PMC Access exception if you tried (people hit this one a decent bit). However, my previous attempt to give a nicer error failed. Today I realized why it had failed, made a cup of tea, then implemented something that did work - it now tells you what you did and also the name of the attribute you tried to get hold of in order to aid debugging. I found and corrected a bogus spectest along the way, enabled the one that checks this doesn't die with a Null PMC Access, and managed to close two tickets from this. However, I have a suspicion there's at least another one or two that I didn't spot that are in the queue and boiled down to this same problem. Anyway, that's another of our major sources of Null PMC Access errors plugged.</li></ul><p>I also spotted two other RT tickets that represented issues that I knew had been solved recently and closed them. I noted that a third was fixed and could be closed if somebody wrote tests to cover it and make sure it doesn't come back. Happily, we have some awesome test writers who take care of such things.<nobr> <wbr></nobr>:-)</p><p>Thanks to Vienna.pm for funding this Rakudo day.</p> JonathanWorthington 2009-08-25T23:46:02+00:00 journal Playing with traits http://use.perl.org/~JonathanWorthington/journal/39504?from=rss <blockquote><div><p>Fly on the wings of despair<br> No one is holding you back<br> The call of the wild is internal<br> Conquer the silence you fear<br> Tomorrow will not fade to black<br> Remember<br> No one can save you today<br> <em>Wings Of Despair - Kamelot</em></p></div> </blockquote><p>Welcome to the final post in my Hague Grant updates. Yup, I've made it through, and will soon be submitting the final report. In the meantime, let me talk a bit about the remaining big topic in my grant that I have not yet discussed: traits.</p><p>Traits are the Perl 6 way of hooking in to various declarations. The mechanism is based upon multiple dispatch. That is, it's possible to write a trait handler by writing a multi-dispatch sub that declares what sort of declaration it applies to and how it is identified. The types of declarations that can be hooked into in this way are:</p><ul> <li>Classes and roles</li><li>Routines</li><li>Variables</li><li>Attributes</li></ul><p>Generally, a trait handler will be called with some object representing the thing being declared, something representing the trait and, optionally, any arguments that were supplied.</p><p>A few days back, somebody (sorry, forget who) on #perl6 was asking about being able to introduce additional accessor methods for an attribute. If I can't remember their handle, I sure ain't going to remember their use case, so let me make one up. In Perl 6, we can write:</p><p> <code>class ValueWithError {<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.min_value;<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.value;<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.max_value;<br> }<br> </code> </p><p>And we get a class that lets us represent a value as best we have it with minimum and maximum values that it could be, which we presumably compute by knowing the errors in some calculations. Thing is, everyone using our class keeps trying to call ".min" and ".minimum" instead of ".min_value", so we decide that we may as well support those too. Well, we could write some accessor methods ourselves, but instead we'll write a small bit of code that introduces a new trait called "aliased". After we've implemented this, we'll then be able to write:</p><p> <code>class ValueWithError {<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.min_value is aliased&lt;min minimum&gt;;<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.value;<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.max_value is aliased&lt;max maximum&gt;;<br> }<br> </code> </p><p>So, let's get started. First, we need to declare a trait handler.</p><p> <code>multi trait_mod:&lt;is&gt;(AttributeDeclarand $a, $names,<nobr> <wbr></nobr>:$aliased!) {<br> <br> }<br> </code> </p><p>The first argument to a trait handler is always the declarand - that is, something representing the thing being declared. For attributes that is an instance of AttributeDeclarand, which gives us a reference to the container but also some extra bits of information, such as the attribute's name and the metaclass of the class it is defined in. Beyond this, we either need a required named argument matching the name of the trait, which we put as a named parameter at the end of the signature, or we need the second positional parameter to be the name of a type. When dispatching a trait, the compiler will see if it's a known type name, and dispatch with the type object for that type if so and a named parameter if not. Finally, we can optionally add another positional parameter if we want our trait to be able to accept arguments (here, that will contain the list of names we will alias to).</p><p>Now we'll start to fill out this stub a little bit.</p><p> <code>multi trait_mod:&lt;is&gt;(AttributeDeclarand $a, $names,<nobr> <wbr></nobr>:$aliased!) {<br> &nbsp;&nbsp;&nbsp;&nbsp;my $accessor_name = $a.name.substr(2);<br> &nbsp;&nbsp;&nbsp;&nbsp;for $names.list -&gt; $name {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;say "aliasing $name to $accessor_name";<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> }<br> </code> </p><p>AttributeDeclarand has a<nobr> <wbr></nobr>.name property, which will hold the name of the attribute. Remember that all attributes have a real name of $!foo, even if declared $.foo to get an accessor generated. We take this and strip off the $! at the start, to get the name of the generated accessor method that we will forward to. Then we will take $names (which were the set of arguments that were passed to the trait), put it into list context and iterate over it. For now, I just added a say in there, so if we actually try creating the class I showed earlier, we'll get the following output:</p><p> <code>aliasing min to min_value<br> aliasing minimum to min_value<br> aliasing max to max_value<br> aliasing maximum to max_value<br> </code> </p><p>So, what is the magic that we need to do in order to finish this? Well, we need to use the metaclass - accessible through the AttributeDeclarand object - and with it add a method. And what do we add? Well, we'll use the anonymous method syntax, the indirect method call syntax and closure semantics to build us a forwarder method, and then in the loop call add_method on the metaclass to add that method under each of the aliased names. Our final code looks like this - I also include the class and some example code.</p><p> <code>multi trait_mod:&lt;is&gt;(AttributeDeclarand $a, $names,<nobr> <wbr></nobr>:$aliased!) {<br> &nbsp;&nbsp;&nbsp;&nbsp;my $accessor_name = $a.name.substr(2);<br> &nbsp;&nbsp;&nbsp;&nbsp;my $meth = method { self."$accessor_name" };<br> &nbsp;&nbsp;&nbsp;&nbsp;for $names.list -&gt; $name {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;$a.how.add_method($a.how, $name, $meth);<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> }<br> <br> class ValueWithError {<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.min_value is aliased&lt;min minimum&gt;;<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.value;<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.max_value is aliased&lt;max maximum&gt;;<br> }<br> <br> my $v = ValueWithError.new(value =&gt; 42, min_value =&gt; 41.5, max_value =&gt; 42.8);<br> say $v.min_value;<br> say $v.min;<br> say $v.minimum;<br> say $v.max_value;<br> say $v.max;<br> say $v.maximum;<br> </code> </p><p>And here is the output:</p><p> <code>41.5<br> 41.5<br> 41.5<br> 42.8<br> 42.8<br> 42.8<br> </code> </p><p>And that's it, we wrote our first trait handler - and it was actually useful! And with that, I bring this series of blog posts on my current Hague Grant to an end. I hope they've been interesting. Worried you're going to miss them? Well, fear not - in a couple of days I'll be applying for my next Hague Grant. In the meantime, have fun playing with and finding creative ways of breaking the new features.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2009-08-21T18:31:05+00:00 journal Ramblings on Rakudo startup http://use.perl.org/~JonathanWorthington/journal/39500?from=rss <p>I try and blog about each Rakudo day on the day it happens or shortly thereafter. Occasionally I fail. This post is a result of such a failure. Worse, I was silly enough to leave myself the following notes:</p><ul> <li>Fixed a couple of minor RT tickets.</li><li>Worked on branch to investigate eliminating reblessing into Perl6MultiSub</li></ul><p>Which are, uh, really great for writing a blog post from a couple of months later. Anyway, since exactly what RT tickets I fixed on that day are probably not of so much interest now, plus I'd have to go and figure out what the date was and what I committed and hope the commit messages had the RT ticket IDs in, I'll instead spend a while talking about what the branch was all about. It actually didn't end up getting merged, but some of the things discovered in it have been applied already, and one of the things I explored in it is going to happen in the future anyway.</p><p>Folks playing with Rakudo will probably have noticed that our startup time is, well, not exactly stellar. Part of the blame lay with Parrot for a while, but the Parrot team have done some really great work on improving things in that aspect, and a lot of it now is that Rakudo just does a lot of work at startup. Basically, at startup, for every block (including routines) we do some setup work. One of those bits of setup work is transforming the Parrot MultiSub data structure into a Perl6MultiSub, which implements the Perl 6 multiple dispatch semantics. What would be far better would be to create that in the first place. That needed a Parrot tweak first, which I did.</p><p>The thing is, Perl6MultiSub depends on the subs having a Perl 6 Signature object attached to them. A Signature object in Perl 6 stores details of the parameters that a block or routine takes, along with any type constraints, whether the parameter is optional or not and so forth. (Right now, some of that is duplicated in Parrot's own signature handling, but Rakudo will be moving away from that and binding entirely based upon its own Signature objects in the next couple of months.) And thus arose an issue: we were using multiple dispatch in various places in Rakudo, but specifying things using some Parrot syntax unrelated to the Perl 6 Signature objects. Some of them were guts, but others were things that really needed to end up with Perl 6 signatures as they'd be user facing.</p><p>Basically, the thing that really killed the branch was that there were just too many places that we'd have to tweak, and it wasn't worthwhile doing that yet. These days, we are writing more and more bits of Rakudo in Perl 6. We build a first stage compiler, then run it to compile the built-ins. All of those get a Perl 6 Signature object generated for them. Writing a signature object by hand in PIR, on the other hand, is a lot of effort. I did work out a way to generate them for a lot of cases, and that led to ideas about caching and possible sharing of signatures, which I expect we will implement in the future. But overall, it was too much pain, considering that if we waited to do the change for some months, it'd be a bunch less painful. It has been worth keeping in mind during development since then, though, and I am trying keep us on a path that will lead to this conversion eventually.</p><p>The other big cost at startup is the creation of the signature objects. This is something that I expect will become much cheaper in the next couple of months, as I review and refactor the way we build them. I'm also considering if - or how far - we can try and do more at compile time and less at runtime in this area.</p><p>Anyway, that was a quick little look through some Rakudo guts. Hope it wasn't excessively uninteresting. And thanks to Vienna.pm for funding this Rakudo day a couple of months ago.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2009-08-21T01:38:16+00:00 journal Deferral http://use.perl.org/~JonathanWorthington/journal/39499?from=rss <blockquote><div><p>The ones who seek justice,<br> Will pray for it all their lives.<br> They can and they will skin us all one day,<br> Oh can you hear them cries.<br> As far as the man can run from us,<br> We're following the trail of blood,<br> So hunt my young ones,<br> The pack they have always feared is back.<br> <em>Ain't Your Fairytale - Sonata Arctica</em></p></div> </blockquote><p>In this Ian Hague Grant update post, I'm going to discuss deferral. Deferral allows a method to pass control to the next best candidate that method dispatch would have called had the current method not existed. Control can either be passed on entirely - that is, we stop executing the current method and skip to the next one - or you can capture the results of the next method in the chain and do some post-processing on them. Additionally, you can massage the arguments before deferring, or alternatively just pass on the arguments as they were originally provided.</p><p>So what do I mean by next best candidate? Let's look at a simple example. Suppose I have a class that knows how to format and output some legal report.</p><p> <code>class ReportRenderer {<br> &nbsp;&nbsp;&nbsp;&nbsp;method render(Report $r) {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;...<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> }<br> </code> </p><p>This works fine for internal things, however when such reports are published on the web we need to redact certain things. Thus we can write a subclass with a report method that does this redaction by modifying the data structure, and then defers to the parent's render method to actually do the display work.</p><p> <code>class PublicReportRenderer is ReportRenderer {<br> &nbsp;&nbsp;&nbsp;&nbsp;method render(Report $r) {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;# Code to redact the report...<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nextwith($r);<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> }<br> </code> </p><p>At this point you may be thinking, "hey, is this not just like a call to the superclass?" And in a way, yes, it is. However, a couple of things make deferral interesting. First of all, "who is my parent" gets more interesting in a multiple inheritance world; deferral instead walks the full method resolution order as seen from the object you initially dispatched on. Second, there's more than one way to defer - in fact, there are four:</p><ul> <li>nextwith($arg1, $arg2,<nobr> <wbr></nobr>...) ends execution of the current method and passes control to the next method, using the supplied arguments.</li><li>nextsame() ends execution of the current method and passes control to the next method, using the arguments the current method was originally invoked with.</li><li>callwith($arg1, $arg2,<nobr> <wbr></nobr>...) calls the next method, using the supplied arguments. It returns with the return value(s) of the method that was deferred to.</li><li>callsame() calls the next method, using the arguments the current method was originally invoked with. It returns with the return value(s) of the method that was deferred to.</li></ul><p>Deferral isn't just up the inheritance hierarchy, however. It's also possible to defer to multiple dispatch variants that are less specific. For example, suppose that we have a web crawler, and one class in the system is responsible for persisting the interesting pages that we found while crawling. We get a report that long pages take too much processing later, and we are asked to store a summary. Imagining our class looks along the lines of...</p><p> <code>class CrawlerPersistence {<br> &nbsp;&nbsp;&nbsp;&nbsp;method save(Str $page) {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;# various code to break up the page and store it<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> }<br> </code> </p><p>We could do a few things. One is that we could put the check and the logic inside of the save method to make the summary, but this may make the method longer than is really desirable. We could write a worker sub, do a length check in the save method and call that, which is better. However, in Perl 6 we have a third option: make the methods multis, and write an additional multi that handles the summarization process and defers.</p><p> <code>class CrawlerPersistence {<br> &nbsp;&nbsp;&nbsp;&nbsp;multi method save(Str $page) {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;# various code to break up the page and store it<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> <br> &nbsp;&nbsp;&nbsp;&nbsp;multi method save(Str $page where { $page.chars &gt; 10000 }) {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;# summarize<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nextwith($summary);<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> }<br> </code> </p><p>Note that a multi with a constraint is considered to be narrower than one with the same type and no constraint, so we always will call the second of these if the constraint matches.</p><p>In general, deferral gives you a nice way of saying "call or pass on control to the next best thing". You can use it when you need to customize behavior, or in cases where you get data that you don't know what to do with but know a more general method accepting more general types or further up the inheritance hierarchy may be able to handle better.</p><p>Under the hood, every method dispatch saves enough information to be able to resume the dispatch and find the next candidate. It does not compute all of the possibilities in advance, since a lot of the time when we do a method invocation we won't defer. Thus we compute the candidates lazily.</p><p>So, that's deferral. Check in soon for my final Hague Grant update post for this grant, in which I'll talk about traits.</p> JonathanWorthington 2009-08-20T22:14:47+00:00 journal Rakudo pre-release bug hunting http://use.perl.org/~JonathanWorthington/journal/39498?from=rss <p>Between vacations and conferences, I've not done a Vienna.pm Rakudo Day for a little while. Anyway, today I took the opportunity to spend the day doing Rakudo improvements before tomorrow's monthly release.</p><p>As more and more people explore Rakudo, we get more and more issues and corner cases reported. At the start of today we had 470 tickets either new or open in the Rakudo RT queue. I hadn't reviewed the ticket queue in a while; back when it was more like 150 tickets I did it every week, but it's a bit too long a job for doing it weekly now. Doing so today was very worthwhile, however: I was able to close six tickets that related to missing features that had already been added or bugs that had already been fixed, and masak++ was able to confirm that another two that he had filed were now dealt with and close them as well. 8 tickets down before I'd even got through my first cup of coffee of the day. Not bad. moritz++ also made inquiries to the perl6-language list over a ticket I brought up for discussion on the channel.</p><p>"Null PMC Access" is in many ways the Parrot equivalent of a segfault (though Parrot does actually segfault now and then too, so we get the best of both worlds...<nobr> <wbr></nobr>;-)). Whenever one happens in Rakudo, it's Just Plain Wrong, and means we overlooked a check somewhere. Today I identified a group of tickets that all gave this error for very related reasons, patched them and unfudged various tests. That was another four tickets sorted out.</p><p>Another easy win was doing the last push on a ticket related to Failure object handling, which others had got mostly there. I did that and closed the ticket. Then I fixed a bug that has been causing some irritation for a while:</p><p> <code>sub foo returns Int { fail("oh noes!") }<br> </code> </p><p>This is meant to work fine, since failure is always acceptable for any non-native type. However, in reality it did not, since the type check wasn't allowing the Failure to get out. It's fixed now. And as a bonus, I could now use fail on one of the typed routines in the setting to fix another bug: Rakudo was dying if you called ord('') with an empty string, when it fact it should only have been failing.</p><p>I spent a chunk of the day dealing with some multiple dispatch tweaks. The first was a trivial bug fix: we were missing something out in the PAST-generation for "multi foo() {<nobr> <wbr></nobr>... }" but putting it in for "multi sub foo() {<nobr> <wbr></nobr>... }". The second two tickets were trickier, and related to how we dealt with slurpy parameters and optional parameters when doing the narrowness analysis to sort candidates. First, the slurpy one.</p><p> <code>multi foo (*@a){ 1 }<br> multi foo () { 2 }<br> say foo();<br> </code> </p><p>This was being treated as ambiguous before. To deal with this, we've implemented the rule that if two candidates are tied, they may be disambiguated if one has a slurpy parameter and the other does not. The one without the slurpy is considered narrower. The thinking behind this is that specifying exactly how many arguments you can take is more specific than saying you can take just any number. The upshot of this is that the above program now prints 2 instead of dying. The optional parameter one was a different issue.</p><p> <code>multi foo (@a) { 1 }<br> multi foo ($a, %h?) { 2 }<br> say foo(&lt;1 2 3&gt;);<br> </code> </p><p>The narrowness analysis was looking at these two and saying "well, they have different numbers of items in the parameter list, so I can't compare them, so they're tied". I relaxed this rule a bit to allow comparison of these based on the non-optional parameters they have, and now since @a is narrower than $a (just out of the normal dispatch rules we already had in place), this example now outputs 1 instead of dying.</p><p>Finally, recently I've been helping __ash__, a new contributor to Rakudo, work on a patch to get us able to do method dispatches from the point of view of a different class. He showed up and said he was hacking on the patch, which was exciting because it's rather non-trivial as a way to get into Rakudo development and he was looking in the right places, but also challenging as I knew fairly well how to do it after some discussion with others on #perl6 about what the spec called for, but didn't want to end up just writing the patch for him, otherwise it'd be a crappy learning experience. Anyway, today __ash__++ made it, with a little more input, to a working patch, which I was very happy to review and apply. So now you can do this (the bottom line is the new thing):</p><p> <code>class A { method foo { say 42 } }<br> class B is A { method foo { say 69 } }<br> my $x = B.new;<br> $x.foo; # 69<br> $x.A::foo; # 42<br> </code> </p><p>And that, along with the usual assorted discussions on #perl6 plus some updates to the ChangeLog ahead of the release, was what I got up to today. Thanks to Vienna.pm for funding it.</p> JonathanWorthington 2009-08-19T23:31:13+00:00 journal A Little Reflection http://use.perl.org/~JonathanWorthington/journal/39455?from=rss <blockquote><div><p>I still remember the sunlight on your face that warm November day,<br> And I still remember my heartbeat quickened by desire, unaware of prices I would pay<br> I still remember the closing door the night we said goodbye,<br> And I still remember losing you for good and knowing that a part of me had died<br> <em>Memory - Redemption</em></p></div> </blockquote><p>Sorry it's been a while since I last posted - YAPC::Europe and preparing for it were something of a distraction, as has been visiting some family and friends, though the real killer has been that in the last month I've managed to have myself a couple of colds/flu-ish things that left me without a great deal of energy during a lot of the time I've been here at home and working. Anyway, I think I've almost shaken off the current infection, and amongst it all I've been able to keep my Hague Grant moving along - in so far as the code, if not the blogging.</p><p>One of the big pieces of my grant that I've been working on is improving Rakudo's support for introspecting classes and roles. Before I started working on this, Rakudo didn't really have any such support. Of course, Parrot provided all of the primitives. But there was some work to be done in building up an implementation of the Perl 6 introspection interface around it.</p><p>To introspect an object, first you need to get hold of its metaclass. This class actually provides the various introspection-related methods. The metaclass can be obtained by using the<nobr> <wbr></nobr>.HOW macro (actually in Rakudo, implemented as a method, but we'll fix that in the future). All metaclass methods take as a first parameter the thing that you want to introspect. So for example, if I want to find the parents of the built-in List class, I would do (showing this using the REPL):</p><p> <code>&gt; say List.HOW.parents(List).perl<br> [Any, Object]<br> </code> </p><p>Where<nobr> <wbr></nobr>.perl gets a Perlish representation of a data structure (think Data::Dumper built into the language). At this point you may be thinking, gee, this sucks, I have to say List twice. Which is why there's a<nobr> <wbr></nobr>.^ operator, that evaluates what is on its left hand side once, calls<nobr> <wbr></nobr>.HOW on it, then calls the method you specify on the metaclass, fudging what was evaluated on the LHS in as the first argument to the call. Thus instead of the above you can just do:</p><p> <code>&gt; say List.^parents.perl<br> [Any, Object]<br> </code> </p><p>Which is much better.<nobr> <wbr></nobr>.^parents takes a few options too.</p><p> <code>&gt; class A { }<br> &gt; class B { }<br> &gt; class C is A is B { }<br> &gt; say C.^parents.perl; # all, flat list<br> [A, B, Any, Object]<br> &gt; say C.^parents(:local).perl; # immediate only<br> [A, B]<br> &gt; say C.^parents(:tree).perl; # all, tree<br> [[A, [Any, [Object]]], [B, [Any, [Object]]]]<br> </code> </p><p>Of course, it's not just parents that can be introspected, but also methods and attributes - both on your own classes and built-in ones. For example, here we look at the methods defined in the Num class. Note the use of the parallel dispatch operator - implemented earlier on in this grant - to get the name property of each of the methods returned in the list from the methods method; we then join the list.</p><p> <code>&gt; say Num.^methods(:local)&gt;&gt;.name.join(", ");<br> tan, cos, sin, pred, acosec, sinh, asinh, cosech, acosech,<br> acotan, Str, asec, cotanh, acotanh, sech, ACCEPTS, asech, atan,<br> acos, tanh, asin, atanh, cosh, cosec, acosh, succ, WHICH, perl,<br> cotan, atan2, Scalar, sec<br> </code> </p><p>Here is a very similar example for introspecting attributes.</p><p> <code>&gt; say Pair.^attributes(:local)&gt;&gt;.name.join(", ");<br> $!key, $!value<br> </code> </p><p>While I have been using these with<nobr> <wbr></nobr>:local, they also support being called parameterless to get all attributes up the hierarchy, as well as<nobr> <wbr></nobr>:tree to build something similar to what you saw earlier for<nobr> <wbr></nobr>.^parents.</p><p>At this point you may have realized that all of this means that you can use a module at the REPL and then explore its classes using the introspection interface.</p><p>Anyway, for my final trick, here's some code that walks a class hierarchy and prints out an ASCII tree-ish thing depicting the class hierarchy. To stop it going and printing all the stuff in Any and Object - classes inherit from Any by default - I've put in a little check for that. It makes the output shorter and probably more useful.</p><p> <code>multi describe($c, $prefix) {<br> &nbsp;&nbsp;&nbsp;&nbsp;take "$prefix " ~ $c.perl ~ "\n";<br> &nbsp;&nbsp;&nbsp;&nbsp;for $c.^attributes(:local) {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;take "$prefix + Attribute: " ~<nobr> <wbr></nobr>.name ~ "\n";<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> &nbsp;&nbsp;&nbsp;&nbsp;for $c.^methods(:local) {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;take "$prefix + Method: " ~<nobr> <wbr></nobr>.name ~ "\n";<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> }<br> <br> multi describe(@list, $prefix) {<br> &nbsp;&nbsp;&nbsp;&nbsp;for @list -&gt; $item {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;unless $item eq Any|Object {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;describe($item, $prefix ~ "&nbsp;&nbsp;&nbsp;&nbsp;");<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> }<br> <br> multi describe($start) {<br> &nbsp;&nbsp;&nbsp;&nbsp;return [~] gather {<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;describe($start, "");<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;describe($start.^parents(:tree)[0], "");<br> &nbsp;&nbsp;&nbsp;&nbsp;}<br> }<br> </code> </p><p>This code demonstrates not only introspection, but also multiple dispatch by sigil, gather/take, the reduce meta-operator (which combined with gather/take lets us build up a string from a bunch of recursive routines without having to worry about building up and passing along a return value all the way up), junctions and a bunch of more basic features. Here's an example program using this.</p><p> <code>class A {<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.a;<br> }<br> class B is A {<br> &nbsp;&nbsp;&nbsp;&nbsp;has $!b;<br> &nbsp;&nbsp;&nbsp;&nbsp;method foo() { }<br> }<br> class C is B {<br> &nbsp;&nbsp;&nbsp;&nbsp;has $.c;<br> &nbsp;&nbsp;&nbsp;&nbsp;method bar() { }<br> }<br> say describe(C);<br> </code> </p><p>And the output:</p><p> <code> C<br> + Attribute: $!c<br> + Method: bar<br> + Method: c<br> &nbsp;&nbsp;&nbsp;&nbsp; B<br> &nbsp;&nbsp;&nbsp;&nbsp; + Attribute: $!b<br> &nbsp;&nbsp;&nbsp;&nbsp; + Method: foo<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; A<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; + Attribute: $!a<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; + Method: a<br> </code> </p><p>Notice how it shows the auto-generated accessor methods for the non-private attributes too.</p><p>So, that's introspection. Have fun, send bug reports.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2009-08-13T12:28:22+00:00 journal The Great Method Dispatch Refactor http://use.perl.org/~JonathanWorthington/journal/39239?from=rss <blockquote><div><p>What don't kill you will make you more strong<br> <em>Broken, Beat &amp; Scarred - Metallica</em></p></div> </blockquote><p>The largest, trickiest part of my current Hague Grant has been an extensive refactor of method dispatch. It's the second major refactor of method dispatch that I have done in the course of Rakudo's development, and while I know there's going to be some tweaks from here to address some more subtle aspects of the semantics, I'm very hopeful that this will have been the last major refactor needed on the path to a Rakudo 1.0.</p><p>The last time I refactored method dispatch, it was to allow us to get a bunch of new features implemented, and to allow fast prototyping of many of the things we'd need. In that sense, it was a very successful refactor, and the ease with which I've been able to get all kinds of things in place - role punning, the handles stuff and junction dispatch from the last grant, correct multi-method dispatch semantics and more - has been very helpful for Rakudo's development. However, as we've come further along, a few things made it clear that this approach wasn't going to sustain us through to 1.0.</p><p>Firstly, it just didn't perform well enough. Method dispatch was running notably slower than sub dispatch. However, the real killer was multi-method dispatch. The previous refactor had opened the door to getting the semantics correct, but while multi sub dispatch was running nice and fast, there just wasn't a nice way to get the level of interplay between the method and multi dispatchers that I felt we really needed to have a shot at acceptable performance.</p><p>Secondly, my original thoughts on how we might implement deference - a more exception based model - turned out not to be the way we really needed to go. I will write in more detail about this in a future post with some examples, but in a nutshell deference is the idea that a method can defer to the next one in a superclass. However, if it's a multi then we defer to the next best multi candidate if there is one instead. Deference can keep the original arguments or use a new set, and can either fully defer (like a tailcall) or do something more akin to a call and get the results back to do further processing. Larry also felt much more strongly than I had expected that<nobr> <wbr></nobr>.wrap functionality on routines should operate through the same mechanism (I agreed that we really should unify the mechanism after pondering it for a while). Again, this issue involves the interplay between multi dispatch and method dispatch - there wasn't just a performance issue to worry about, but also a big semantic one.</p><p>Thirdly, the previous dispatch refactor had hurt our language interoperability a bit. Up until recently, the promised land of high level language interoperability in Parrot had been more promise than reality, but that picture has changed of late, with Tene++ posting concrete examples of calling between Cardinal (an early Ruby on Parrot compiler) and Rakudo. There's really not much point implementing Perl 6 on Parrot if we're not going to be able to interoperate decently with other high level languages on Parrot.</p><p>The performance and interop problems were partly the same issue: the previous dispatcher wasn't really built on top of the Parrot model for doing such things (which is, subclass Parrot's Object PMC override find_method). So one of the principals for the refactor was to build Perl 6 method dispatch semantics on the Parrot find_method/invoke model.</p><p>Figuring out how to build a method dispatcher as complex as the one needed for Perl 6, which would fit neatly into the Parrot model, have a good chance of working well when invoked from other high level languages, solve the deference problem, get the interplay with multiple dispatch right and on top of that lot actually have a chance of performing well, was non-trivial. I got us a bit of the way in small steps, but eventually I hit the point where it was time to do the big switch. It goes without saying that I was especially glad of the test suite at this point. The fact that I was able to do a complete switch in the way how something so fundamental worked and hear of almost no regressions to people's real world Perl 6 code when I committed it is a testament to the hard work that many have put into Perl 6 testing. So, a big thank you goes out to all involved!</p><p>So how does method dispatch look now in Rakudo? First, let me note the steps in invoking a method in Parrot. First, the find_method vtable method is called on the PMC representing the object. This hands back another PMC, which we call the invoke vtable method on to invoke it. Normally in Parrot, when you do a find_method, Parrot goes off and finds the PMC representing the chunk of code we need to execute (a Sub object or some subclass of that) and returns it. In my overridden find_method, we instead hand back a P6invocation PMC. This contains the method we will dispatch to. However, it also contains something else: the information needed to continue looking for more things to dispatch to in the event we later are asked to defer. That is, it doesn't find all of the things we might have to dispatch to. It just makes sure we have the information to do so at some future point. This lazy approach means that ordinary dispatch-and-we're-done calls aren't paying the cost of deference. However, in the event we need to find more methods, it can act like an iterator and provide them. This, conveniently, is exactly what we need to implement<nobr> <wbr></nobr>.can, which in Perl 6 returns a lazy iterator of all of the possible methods rather than just a boolean value (though I didn't implement this just yet - I'll get to it). One final thing: I think I can keep the initial P6invocation immutable, so we'd be able to cache it in a method cache at some future point and have even less work to do.</p><p>So anyway, that's what our find_method does. P6invocation's invoke, then, just knows the first result and goes and invokes it. But it has to do one little thing more than that: it needs to store itself in a place accessible to things like callsame/nextsame and friends which defer. It turns out that fudging it into the lexpad during the invoke is a very neat way to do this. For one because it means if people start doing evil things like returning a closure from within a method that defers, we'll probably do something sensible. It also means its lifetime is neatly managed for us, since it's tied to the "activation record".</p><p>So how far along is all of this? Well, the refactor is done, pushed and if you're using the last release of Rakudo or the latest git head then you're already using it now. I'm not yet done with building all of the bits of deference on top of it yet, so don't expect all of that to work at the moment. It's coming soon (I'm happy that the dispatcher fundamentals are correct, the issues are actually now in callsame and other such routines). Furthermore, I already refactored method wrapping to work in terms of candidate lists and P6invocation too. So that bit of my grant can be ticked off. (This refactor also meant that we started passing some more wrap tests too. In fact, until somebody - I forget who - recently discovered an odd bug in interaction between wrappers and lexical scopes, I really thought wrap was, uh, all wrapped up. Well, such is software development...</p><p>So, that's another installment in my Hauge Grant progress blog posts, and another significant step forward for Rakudo. Next time, I'll probably look at traits or deference. And yes, there'll be more code and less discussion of bird guts.<nobr> <wbr></nobr>:-)</p> JonathanWorthington 2009-07-06T22:55:13+00:00 journal Lots of little improvements http://use.perl.org/~JonathanWorthington/journal/39196?from=rss <p>I'm back from a nice break in Italy and have been digging back in to Perl 6 stuff again. Today I've been doing a Vienna.pm-funded Rakudo day, and here's what I got up to.</p><p>First off, I went for a look through our RT queue. We now have over 400 tickets that are either new or open. While on the one hand that means we've a lot of work to do, it's also a sign that people, more and more, are playing with and exercising Rakudo. In just browsing through it, I found a bunch of things I could work on and hopefully resolve fairly easily during the day, and also another bunch of things that were already resolved. Just spotting the latter allowed me to mark 3 tickets resolved.</p><p>A couple of the things I worked on related to subtyping. Of note, the standard grammar accepted:</p><p> <code>subset Foo of Int;<br> </code> </p><p>Without requiring a where clause. Rakudo now also accepts this, and our parsing is a little closer to STD.pm too (we parse traits on subtypes, but we don't do anything with them just yet). Next, I got Rakudo to support a neater syntax for declaring anonymous subtypes in signatures. If you just want to match a specific value, you can write the value in the signature, and that's it. For example, here is yet another way to do factorial (a recursive version).</p><p> <code>multi factorial(0) { 1 }<br> multi factorial(Int $n) { $n * factorial($n - 1) }<br> say factorial(5); # 120<br> </code> </p><p>A signature<nobr> <wbr></nobr>:(0) is equivalent to<nobr> <wbr></nobr>:(Int $ where 0). This means that it will sort in the candidate list with Int. More generally, any literal value in where will get a nominal type based on the<nobr> <wbr></nobr>.WHAT of the value and have the value made into an anonymous subtype, so the signature<nobr> <wbr></nobr>:("tava") is just like<nobr> <wbr></nobr>:(Str $ where "tava"). I added some tests for all of this too.</p><p>Lyle++ had sent in a patch a while back for $*CWD and chdir. I took a look at these today. The $*CWD one looked pretty good, so I applied that with just a minor tweak. The chdir one needed some more attention and fixing up first, but I got that applied and extended the tests to better exercise it. Then I got both test files added to spectest.data. So now chdir and $*CWD are both functional. Here's some play with them in the REPL.</p><p> <code>&gt; my $fh = open("spectest.data",<nobr> <wbr></nobr>:r);<br> Unable to open filehandle from path 'spectest.data'<br> in Main (:1)<br> &gt; say $*CWD;<br> C:\Consulting\parrot\trunk\languages\rakudo<br> &gt; chdir "t";<br> &gt; say $*CWD;<br> C:\Consulting\parrot\trunk\languages\rakudo\t<br> &gt; my $fh = open("spectest.data",<nobr> <wbr></nobr>:r);<br> &gt;<br> </code> </p><p>We had a couple of tickets relating to the interaction of<nobr> <wbr></nobr>//= and state variables. A little investigation, some discussion on #parrot and a fix later, I was able to unfudge tests and mark those resolved. A small inheritance bug was a similar story.</p><p>Finally, in preparation to improve type check failure error reporting and resolve at least one ticket in that area, I factored all type check error generation out to one routine, which we now call consistently. That means errors that previously missed out mentioning the expected and received types now do so, and the other issues I can fix - on some future Rakudo day - in one place, and everywhere that reports such errors will benefit.</p><p>In the course of the day, I also discovered a couple of other tickets that I had opened up to investigate at the start of the day were also already-fixed issues, so I made sure we had proper test coverage and got them closed up.</p><p>So, a pretty productive day. Thanks to Vienna.pm for funding!</p> JonathanWorthington 2009-06-30T21:09:56+00:00 journal