Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

JonathanWorthington (6049)

JonathanWorthington
  (email not shown publicly)
http://www.jnthn.net/
Saturday July 17, 2010
09:58 PM

Last Post

Well, on use.perl.org, anyway. I'd like to thank those who run this place for doing so, as it's been the place I've blogged about my Perl 6 stuff for the last couple of years. However, I've found the user experience of WordPress a lot more enjoyable - it was great for the Perl 6 Advent Calendar - and figure I'll blog more if I more enjoy doing so.

So, from now you'll be able to read my Rakudo news and other Perl 6 related ramblings from my shiny new blog. For those reading through Planet Six, no action required - I'll arrange for my new blog to be aggregated there.

See you over there.

Tuesday June 15, 2010
10:11 AM

Perl Mova + YAPC::Russia

On Friday I flew from rainy Sweden to scorching hot Kiev to attend a combined Perl Mova + YAPC::Russia. The passport queue wasn't overly long, and I'd happily managed to be hand luggage only, so I wasn't too long in the airport. I planned to take the "official" bus, but before it appeared I heard a lady shouting about a mashrutka going to the center, so I took that instead. Understanding basic Russian - well, it coulda been Ukrainian too - sure comes in helpful. I was dropped by a metro station, leaving the 15 euro-cent (yes, really) metro ride a few stops to where I was staying - a hotel right on the main Independence Square. I was a little bemused as I checked in to be spoken to in a mixture of Russian and Ukrainian, by the same person in the same conversation.

My plane had been slightly late, so I was also slightly late joining the pre-workshop dinner. After I'd filled up on a tasty dish of borsch, we wandered on to a pub that offered no less than 22 different beers. I was...happy. :-) Some beer later, it was time for a little stroll, which terminated in a Japanese restaurant, where I had my first encounter with wasabi vodka. All in all, a very fun start to my visit here.

I was worried I'd sleep badly in my room since it was a little hot, but I actually slept really quite well and was nicely refreshed for the hack meet. There were a lot of people there hacking on all kinds of different projects. Some folks were interested in contributing to Rakudo, and so we gathered around on a table and I helped each of them find and get started on a task. It went extremely well - everyone in attendance contributed Rakudo patches or code that allowed us to close an RT ticket right there on the day, or that will after I apply patches. Of note, people who were new to Rakudo hacking:

  • Located and unfudged tests for \e that was recently fixed, allowing the ticket about that to be closed.
  • Implemented .all, .any, .one and .none method versions of junction constructors and added tests for them, allowing that ticket to be closed.
  • Debugged and then wrote a patch that fixed a bug in range iteration, plus added tests, allowing an RT ticket to be closed.
  • Implemented .cando and did some related refactors in the area that I suggested and also located some tests to unfudge. Due to network issues that one didn't get applied on the day, though I have it and have just applied it locally, so it'll be in soon.
  • Tracked down what was wrong with forming colon pairs from variables involving twigils, patched it and made sure we had some test coverage of that; again, we closed a ticket.
  • Wrote a patch to fix a bug in the series operator that had been reported in RT, along with some test cases for it.

I think it goes without saying that this is really quite impressive, especially given they had to share one Rakudo core hacker between them for guidance. In fact, I even had time to cook up a few patches myself amongst guiding others! It was great fun to hack alongside such pepole. I handed three spectest repo commit bits out that day, and I think they were all used. It's experiences like the one at this hack meet that make me really glad that we're writing most of Rakudo in Perl 6 or a restricted dialect of it - all of the above tasks required (apart from a one line change in one of them) no PIR or C knowledge at all. Some of those at the hackmeet said they may find time for a bit more Rakudo hacking in the future, which would be really great. :-) After the hack meet, there was more nice food - including a Chicken Kiev - and some lovely beer at prices I've become rather unacustomed to in Sweden.

Day two was talks day. I had one 40 minute talk on Perl 6 signatures which went well. I received some great questions, and I hope I answered them all to people's satisfaction. Later on, I gave a lightning talk about Rakudo * and what it would contain, and showed off a simple little example site that used Rakudo Perl 6, including the modules JSON::Tiny by moritz++ and FakeDBI by mberends++ along with Blizkost (sorear++ for that) to get at the Perl 5 CGI module (though I'll refactor it to use Web.pm in the near future - I just wanted an excuse to show off Blizkost). The evening was, of course, more food and beer, and a lot of fun.

The final day of the workshop was a bit different - a river cruise! It was very relaxing, and gave me yet another way to enjoy this beautiful city. Certainly good for unwinding after a workshop. Most people left either after that or were flying home today; I on the other hand used "no direct flight on a Tuesday" as an excuse to get another day to potter around Kiev, and today I've enjoyed doing exactly that, gently strolling between my favorite sights and stopping in the odd open air cafe in a park to keep myself refreshed in the somewhat stuffy weather. Soon it'll be time to take my last dinner here, an evening stroll and maybe find some outdoor place to sit and nurse a final beer before it's time to get some sleep before my flight home tomorrow.

Beautiful Kiev. It gets harder to leave each time I come. :-)

Tuesday April 20, 2010
04:55 AM

Perl 6 talk in Malmö

Recently I moved to Sweden to join startup company Edument. They are, happily, very open to and supportive of my involvement in the Perl 6 project, and are organizing and sponsoring me to give a talk on Perl - both 5 and 6 for the Dataföreningen in Malmö on Tuesday 27th April. I believe this is only open to those who are members of the user group; if you are, feel free to come along though.

Thursday April 08, 2010
06:59 PM

The Easter Hackathon

I spent my Easter up in Uppsala with masak++, who it turns out is not only awesome at breaking Rakudo, but also a great host. Amongst a lot of memes, bad puns, wonderful noms (I learned how to make kötbullar!) and even a Czech beer, we got quite a bit of Perl 6 hacking done too. :-)

On the first evening, we managed to fix up a bunch of issues with various escape sequences in regexes and strings, and also the fact that "\$x" was failing to escape the $ since we made ng the new master! I fixed a longstanding bug that annoyed masak so much that he dedicated a whole blogpost to it, did some work on enforcing zone constraints in signatures and, in an awesome bit of collaborative development along with colomon++ and moritz++ wired up their new reduction implementation to the reduction meta-operator parsing stuff. That means that you now have triangle forms of the meta-ops, chaining meta-ops work and right-associative ones do the right thing too. We never had any of these right before.

Spurred on by the interpolation work on the previous night, on the next day I started looking at what it would take to implement more of the interpolation bits, and after a bunch of hacking I accidentally the whole lot of it. You can now interpolate indexes into both arrays and hashes, calls to subroutines and method calls on all of these. The way this works in Perl 6 parsing-wise is really neat: we just call from the quote parser back into the standard expression parser, but giving it a few constraints so it knows only to parse the things that are allowed to be interpolated into strings. It's a fairly directly following of STD - we differ in a couple of minor ways for now, but the approach is the same.

Along the way, I used the regex tracer that pmichaud++ had implemented while developing the new grammar engine, and noticed that we seemed to be dropping into a lot of protoregexes we shouldn't have been. A little more exploration and I realized that we were failing to calculate a bunch of constant prefixes for regexes. Fixing that gave a 5% saving on parse times.

The day after that, I worked on something I'd really been putting off because it's hard and painful: making the setting the outer lexical scope of user programs. While some languages define a prelude, Perl 6 essentially defines a "circumlude" - it's as if your program was running as an inner lexical scope inside the built-ins. That means that you find just about everything through lexical scoping. I got the first difficult 80% done and committed, but I suspect I've now got another really difficult 80% left to go, and probably another really really difficult 80% after that. Along the way, I fixed some bugs to do with scope modifiers "leaking". Anyway, to prove it helped somewhat, I tossed a whole bunch of "our"s that had been in as hacks while we didn't have the setting as an outer lexical scope. I suspect that perfecting this is going to be a large chunk of my work on Rakudo over the next couple of weeks.

The last day, I did the (relatively easy, thanks to all the groundwork I had laid during my signatures grant) bits of work to get us able to do smart-matching against signatures. This deserves an entire post of its own, and I'll write one for you soonish, but essentially you can use it to write a given/when that dives into a hash/array/object and looks at whats in it, just using declarative signature syntax. It's just like having a sub-signature on a parameter passed to a block or routine, apart from the signature object stands alone. By the way, I do plan to submit a talk on signatures for YAPC::EU - I'm really quite excited about the declarative power they offer that goes far, far beyond a better way to do parameters than Perl 5's @_. Especially in combination with multi-dispatch.

masak++ spent a good portion of the time working on Temporal/DateTime spec and starting an implementation to back it up. It has to be one of the most bikeshedded areas in the Perl 6 spec, with plenty of "let's do something SO clever" thinking along with the much more sensible "we want something minimal but useful in the language itself, and the clever stuff belongs in modules". Anyway, masak++ and mberends++ for having the patience and diligence to take on something that we really need a finalish answer on, amongst all the various opinions people have on how it should be done. I'm hopeful that we're now approaching the point where we have something that works for the vast majority of use cases, and the answer to anyone who it doesn't work for is simply, "go write a module".

Anyway, that was The Easter Hackathon! For those not following the commits, I'm happy to report that others are also committing bits here and there, and we're steadily working towards brining you Rakudo *. :-)

Sunday March 21, 2010
08:09 PM

A quick Rakudo update

The last week has brought some sad news. While software is of course insignificant compared to life and health, and it's absolutely right that at this time Rakudo should be the last thing Pm should be worrying about, I know a lot of people will be wondering what this means for the Rakudo * release. Myself and the other Rakudo developers are still working out the exact details, but here's an overview.

  • Rakudo * will be delayed in the "not in late April" sense. We all agree on this. There's no way you can take the lead developer of a project away, when the overall team isn't that big anyway, and expect to deliver the same product on the same schedule.
  • While I've always stated Q2 in my talks, we've also always had an internal target date of April. Of course, this is open source so nothing is internal, and April has been widely latched on to. :-) We're sticking with the Q2 goal, but now expect to deliver in May or June (preferably, before YAPC::NA :-)).
  • While we do of course greatly miss Pm's company, direction, and code, I'm comfortable that there's not anything on the Rakudo * ROADMAP that absolutely blocks on Pm. There's things that are harder for the rest of us to do, but we've actually tackled some of them head-on in the last several days, and now have first cuts of many of them - or in some cases are virtually done with them.

There's been a lot of exciting progress in Rakudo recently - the ChangeLog for the last release gives a bunch of it, but we've done a whole load more since then too. I'll try and blog about some of it soonish. In the meantime, I'd like to thank all of the Rakudo and Perl 6 team for being amazing to work with on this, and my thoughts and prayers are with Pm and Paula.

Monday March 15, 2010
07:48 PM

My life refactored, Zavolaj, workshops, Rakudo and more!

It's been a while since I scribbled anything here, so a few quick updates.

First, an update on me. Since last time I wrote, I've moved country. Yup, I actually left behind my beloved Bratislava and I'm now hacking from Lund in Sweden. The beer - like everything else - is more expensive, and I don't have views of a beautiful city castle from my apartment here. But the city is overall very pleasant, I'm located a short walk to the center and the train station, and the supermarkets have a great range of international nom (and HP sauce!) I've yet to try the nearby curry house, or any of the other many appealing restaurants here, but I'm optimistic there will be plenty of WIN. Anyway, here's hoping I can get nicely settled down here.

I was greatly assisted with the move to Sweden by Perl 6 hacker Martin Berends, to whom I am extremely grateful. On the way, we went via the Netherlands Perl Workshop. I gave a couple of talks; I'll get the slides online shortly. It was really lovely to be at the Netherlands Perl Workshop again - the atmosphere was every bit as warm and friendly as I remembered from the last time. To everyone who organized and attended: thanks for a wonderful couple of days. :-)

In the couple of days before the move and during the NPW, mberends++ and I hacked on a shiny new project: Zavolaj! It is a way of doing native calls from Perl 6, just by adding a trait to a stub subroutine, and is a first draft for native calling in Perl 6. The upshot is that we now have a working MySQL client written using Zavolaj, in pure Perl 6! mberends++ is continuing to build this out, and happily this all means that Rakudo * will ship with database access, amongst other things. (I've also done an example of calling a Win32 API, and we're both pondering an ImageMagick binding with it too).

My first days in Scandinavia were actually mostly spent in Copenhagen, at a Perl 6 hackathon organized by Copenhagen.pm - a group I hope to drop in on now and then, since I'm only an hour or so's train ride away. There's a wonderful write-up by moritz++ that tells a lot of what went on, and I can only echo his sentiments about how much fun it was to hack and hang out in meat space with other Perl 6 folks. I'm already looking forward to the next time we get to do that!

Rakudo wise, the work goes on at a good pace. I've been working on lots of bits, and colomon++ has been having quite the patch fest too. The release later this week is a good step forward from last month's release, which was the first after the ng branch was merged. We've won back a lot of functionality again, and this is the first release ever that has some basic support for versioned modules - a direct consequence of the hackathon - and also regexes taking parameters, thanks to a patch from bkeeler++.

Anyway, that's all for now. Only to note that I expect to be making it to the Nordic Perl Workshop in Iceland in May, and with a bit of luck to the Perl workshops in France and Ukraine too (will take some fun logistics to do the two, but I did miss the French Perl Workshop last year, and I always really love to visit Ukraine, so I'll try and work out a way). Have fun, and I'll try and blog again soonish!

Friday February 19, 2010
07:21 PM

Unpacking data structures with signatures

My signature improvements Hague Grant is pretty much wrapped up. I wrote a couple of posts already about the new signature binder and also about signature introspection. In this post I want to talk about some of the other cool stuff I've been working on as part of it.

First, a little background. When you make a call in Perl 6, the arguments are packaged up into a data structure called a capture. A capture contains an arrayish part (for positional parameters) and a hashish part (for smok^Wnamed parameters). The thing you're calling has a signature, which essentially describes where we want the data from a capture to end up. The signature binder is the chunk of code that takes a capture and a signature as inputs, and maps things in the capture to - most of the time, anyway - variables in the lexpad, according to the names given in the signature.

Where things get interesting is that if you take a parameter and coerce it to a Capture, then you can bind that too against a signature. And it so turns out that Perl 6 allows you to write a signature within another signature just for this very purpose. Let's take a look.

multi quicksort([$pivot, *@values]) {
    my @before = @values.grep({ $^n < $pivot });
    my @after = @values.grep({ $^n >= $pivot });
    (quicksort(@before), $pivot, quicksort(@after))
}
multi quicksort( [] ) { () }

Here, instead of writing an array in the signature, we use [...] to specify we want a sub-signature. The binder takes the incoming array and coerces it into a Capture, which essentially flattens it out. We then bind the sub-signature against it, which puts the first item in the incoming array into $pivot and the rest into @values. We then just partition the values and recurse.

The second multi candidate has a nested empty signature, which binds only if the capture is empty. Thus when we have an empty list, we end up there, since the first candidate requires at least one item to bind to $pivot. Multi-dispatch is smart enough to know about sub-signatures and treat them like constraints, which means that you can now use multi-dispatch to distinguish between the deeper structure of your incoming parameters. So, to try it out...

my @unsorted = 1, 9, 28, 3, -9, 10;
my @sorted = quicksort(@unsorted);
say @sorted.perl; # [-9, 1, 3, 9, 10, 28]

It's not just for lists either. An incoming hash can be unpacked as if it had named parameters; for that write the nested signature in (...) rather than [...] (we could have use (...) above too, but [...] implies we expect to be passed a Positional). For any other object, we coerce to a capture by looking at all of the public attributes (things declared has $.foo) up the class hierarchy and making those available as named parameters. Here's an example.

class TreeNode { has $.left; has $.right; }
sub unpack(TreeNode $node (:$left, :$right)) {
    say "Node has L: $left, R: $right";
}
unpack(TreeNode.new(left => 42, right => 99));

This outputs:

Node has L: 42, R: 99

You can probably imagine that a multi and some constraints on the branches gives you some interesting possibilities in writing tree transversals. Also fun is that you can also unpack return values. When you write things like:

my ($a, $b) = foo();

Then you get list assignment. No surprises there. What maybe will surprise you a bit is that Perl 6 actually parses a signature after the my, not just a list of variables. There's a few reasons for that, not least that you can put different type constraints on the variables too. I've referred to signature binding a lot, and it turns out that if instead of writing the assignment operator you write the binding operator, you get signature binding semantics. Which means...you can do unpacks on return values too. So assuming the same TreeNode class:

sub foo() {
    return TreeNode.new(left => 'lol', right => 'rofl');
}
my ($node (:$left, :$right)) := foo();
say "Node has L: $left, R: $right";

This, as you might have guessed, outputs:

Node has L: lol, R: rofl

Note that if you didn't need the $node, you could just omit it (put keep the things that follow nested in another level of parentheses). This works with some built-in classes too, by the way.

It works for some built-in types with accessors too:

sub frac() { return 2/3; }
my ((:$numerator, :$denominator)) := frac();
say "$numerator, $denominator";

Have fun, be creative, submit bugs. :-)

Wednesday February 17, 2010
08:18 PM

The first release from ng is coming!

Tomorrow's regularly scheduled Rakudo release is the first one since the long-running "ng" branch became master. It represents both a huge step forward and at the same time a fairly major regression. Internally, the changes are enormous; some of the biggest include:

  • We're parsing using a new implementation of Perl 6 regexes by pmichaud++. It is a huge improvement, supporting amongst other things protoregexes, a basic form of LTM, variable declarations - including contextuals - inside regexes and more. The AST it generates is part of the PAST tree rather than having a distinct AST, which is a neater, more hackable approach. The issues with lexical scopes and regexes are resolved. Closures in regexes work.
  • NQP is also re-built atop of this. It incorporates regex and grammar support, so now we run both grammar and actions through the one compiler. It's bootstrapped.
  • In light of those major changes, we started putting the grammar back together from scratch. A large part of this was copy and paste - from STD.pm. The grammar we have now is far, far closer to STD than what we had before. Operator precedence parsing is handled in the same kind of way. We've started to incorporate some of the nice STD error detection bits, and catch and nicely report some Perl 5-isms.
  • Since the grammar got re-done, we've been taking the same approach with the actions (the methods that take parse tree nodes and make AST nodes). Thanks to contextual variable support and other improvements, a lot of stuff got WAY cleaner.
  • The list/array implementation has been done over, and this time it's lazy. There's certainly rough edges, but it's getting better every day. The work to implement laziness has led to many areas of the spec getting fleshed out, too - a consequence of being the first implementation on the scene I guess.
  • All class and role construction is done through a meta-model rather than "magic". The Parrot role composition algorithm is no longer relied upon, instead we have our own implementation mostly written in NQP.
  • The assignment model was improved to do much less copying, so we should potentially perform a bit better there.
  • Lexical handling was refactored somewhat, and the changes should eliminate a common source of those pesky Null PMC Access errors.

Every one of these - and some others I didn't mention - are important for getting us towards the Rakudo * release. The downside is that since we've essentially taken Rakudo apart and put it back together again - albeit on far, far better foundations - we're still some way from getting all of the language constructs, built-in types and functions back in place that we had before. It's often not just a case of copy-paste; many of the list related things now have to be written with laziness in mind, for example.

So anyway, if you download tomorrow's release and your code doesn't compile or run, this post should explain - at least at a higher level - why. After a slower December and January, Rakudo development has now once again picked up an incredible pace, and the last couple of week's efforts by many Rakudo hackers have made this release far better than I had feared it was going to be. If we can keep this up, the March release should be a very exciting one.

Saturday February 06, 2010
05:45 PM

Catching up: two Rakudo Days from December

Today plenty happened in Rakudo land - in fact, it was my most active day's Rakudo hacking in quite a while. colomon++ also made some great commits, and between us a lot of things moved forward today. For my part, hashes and pairs are in much better shape.

I wrote before that I'd got some Rakudo days left to write up; there are two of them, but I'll cover them both in this post, since some of the work crossed the two of them anyway. Here's what I got up to between them.

  • Filled out attribute composition logic for role application. A good chunk of this was written in NQP - in fact, all of the role appliers are. :-) Along the way I brought roles up to speed with the attribute part of the meta-object protocol - I'd forgotten that when doing it for classes, though since we couldn't compose attributes at that time it wasn't so interesting anyway. The end result was that we could pass S14-role/attributes.t again.
  • The specification states that if in a role you do inheritance, then this is just passed on to the class that the role is eventually composed in to, and added to that class's parents. We never had any support for this in master; with a neat meta-model approach it became rather easier to get it in place in ng.
  • Got BUILD/CREATE fixed up a bit and added back support for "has $.answer = 42" style declarations, again through the new attribute sub-protocol.
  • Got us handling non-block where blocks again, and added Block.ACCEPTS back - in Perl 6.
  • We had various "helpers" to let us do some of the low-levelish stuff in PIR. This is mostly for the places where we need those things in place in order to be able to compile the rest of the built-ins that are written in Perl 6. However, a couple of these helpers knew too much about Parrot and too little about the meta-model, which abstracts it away. So, I re-wrote some of those in terms of the meta-model. Much cleaner.
  • Before we relied entirely on Parrot for our "do we do this role" checks. However, given the unfortunate semantic mis-match between Parrot's built-in role support and what we need for Perl 6 (I did try and influence things in a different direction back when we were doing Parrot's role support, but failed), I've been gradually working us towards not relying on those for Perl 6's role support. (In master, it felt to me like we have almost as much code working around the semantics of Parrot roles as we'll need to have to not use them.) Anyways, the divorce isn't quite complete yet, and it's not even a goal for the ng branch. However, I did make a notable step towards it by getting our .does checks implemented entirely in terms of the meta-model. In the long run, I'm hoping we may be able to write the entire role implementation in NQP, which helps with the even-longer-run dreams I have of Rakudo having additional backends. But that's for The Future. :-)
  • Cleaned up and re-enabled sigil based type checking in signatures.

Thanks to Vienna.pm for sponsoring me to hack on Rakudo, not only for these two days, but also throughout 2009!

Friday February 05, 2010
07:31 PM

The importance of a break

Several days before Christmas, encouraged by my mum asking, "when you're going to start your Christmas break", I stopped working and hacking on stuff and started relaxing. Until then, I hadn't realized just how tired I was. I slept quite a few ten hour nights in the following week, and had an enjoyable Christmas break. I'd figured I'd maybe take a week or so's break, and then get straight back to things, but a week on I had no motivation or energy to dig in again whatsoever. So, I decided my break would go on through New Years. New Year's celebrations this year involved curry - something I certainly wouldn't mind it involving again.

Early January brought several days in Sweden, part of planning for an upcoming refactoring of my work/location - there's details on my personal blog, but the short version is that I've accepted a job at a Swedish startup and will be moving there in March. It's not full time, so I'll continue to have time for Perl 6 development. They know about and, happily, are supportive of my involvement in Perl 6 and my continued attendance of Perl conferences.

I spent a weekend in Prague on the way home. I did it by train rather than flying, which was enjoyable. It snowed almost my entire time in Prague, and I caught a cold in the following week, but it was kinda worth it to wander around this beautiful city. Didn't bother studying Czech at all, and sorta got by with speaking Slovak, though some folks heard me speak and immediately concluded English would be easier. :-) Somehow it kinda felt like I was back somewhere I belonged, even though I'd never been there before. I love central Europe, and excited as I am about Sweden, I know I'll miss this part of the world a lot.

Anyway, I eased back into some work in January, but mostly took it quite easy. The happy result is that, come February, I'm finding myself recharged and ready to dig back into things again. I got some nice commits done to Rakudo yesterday, and today I meant to, but instead participated on an interesting thread on p6l and did some other useful meta stuff (like this post). Tomorrow should have plenty of hacking time though, and I'm looking forward to it. I also have a couple of blog posts to do about Vienna.pm-funded Rakudo Days I did in December, but never got around to writing up; thankfully I did make notes on what I did on them. :-) My main focuses from here on will be on:

  • Continuing to get Rakudo's ng branch into shape - we'll make it master soon. A lot is missing, but things are going back fast and often very neatly. It's easy to focus on what it doesn't yet do that master does, but it has many things right that master does not - now including laziness!
  • Finishing up my signatures grant. I really, really want to do that within the next couple of weeks.

Anyway, that's what's been up with me. If you take away anything, it's that you may not realize how much you need a break from something until you take it, and if it's not the only thing putting food on the table, then it's probably better to take the needed amount of break and come back revitalized. I guess the other option is to dig back in regardless, but I suspect that's the path to burnout, something I'm quite keen to avoid.

More technical blabbering here soon. :-)