use Perl

The Perl community needs to discuss the DarkPAN and the present and future of the language and its ecosystem in much more productive ways than accusations, hyperbole, and strawmen. This includes the people who want to destroy the DarkPAN as well as the people who believe that its stability must be preserved.

I write provocatively and deliberately. I'm writing a manifesto, after all. Yet I believe I never devolve to name-calling, strawmen arguments, nor misquoting people. (If I do, please tell me and I'll correct it.)

My motives are simple: I want regular releases of Perl 5. I want the default behavior of Perl 5.12 to encourage modern Perl programming practices. I want missing features added to the language, even in small steps at first if necessary. I want Perl 5 to be a better language for creating interesting new programs than for maintaining clunky old programs.

Here are some selected quotes from Rafaël Garcia-Suarez's The DarkPAN matters:

DarkPAN is just a slang word to express the fact that Perl 5 is now currently used all over the world in production and sometimes critical systems... asking this question, even rhetorically, makes you sound as if you didn't knew that Perl is actually used in production.

When I use the word "DarkPAN", I mean precisely this: code which p5p cannot see, cannot search, does not know the features of, and which may or may not exist. See also "We can't change this misfeature in the core, because it may break the DarkPAN!" -- funny how that sentence always ends there, rather than including "But of course, we may not, and we'll never know, because it's the DarkPAN."

I tend to think that the regular contributors of perl5-porters are a lot more likely to have informed opinions about the Perl 5 core development than people who don't even read it.

Pick on me instead then. If you don't like my opinions, feel free to revert all of my patches over the past eight years.

[Who's] trying to be realistic by attempting to release software with some quality expectations, notably by making the upgrade process seamless and introducing as little bugs as possible?

Yet those quality expectations did not work in the case of Perl 5.10, with well-known defects continuing to spread as more and more people upgrade to Perl 5.10. The existing process does not work. Call it "realistic" if you like, but it did not work for 5.10. (Note also that one of the blockers to 5.10.1 is RT #22977 -- see also RT #50528 -- a bug almost six years old that's been present in eleven stable releases of Perl 5.)

First, we should not consider breaking it. Breakage happens, but our goal is to avoid it. Secondly, breakage is detected after upgrades, usually (or it would have been avoided).

The original point referred to deliberate, backwards-incompatible changes. Note, however, that a regular release policy could revert accidental breakages on a known, fixed schedule. The current Perl 5 release policy does not address this at all, which is part of my objection.

We just don't like to introduce incompatible changes just for the sake of being incompatible.

The only people suggesting making incompatible changes for the sake of incompatibility are people setting up a strawman argument against people like me who write tests, write testing modules, write documentation, write manuals, explore bugs, give talks, teach new users how to write Perl effectively, and occasionally write patches for new features who say "You know, some of the existing features are wrong, and they'd be better off this way."

In other words, this is an attempt to diffuse the real issue. The question is not and has never been "Should existing code randomly stop working?" Instead the question is "Can the defaults change in such a way that existing code will still work with perhaps a one-time, one-line change?" (It may not even require that much change.)

[Code] changes don't happen because someone yells about it on a blog. They happen because someone actually writes code.

I wrote code. Rejected. ("What's the point of a class keywords? You can always use package and @ISA anyway! If you need something more, just use any of several dozen Class::* modules on the CPAN!") Jonathan Leto put together a potential Perl 5.10.1 release that needed only a pull from the appropriate perldelta and a pumpking to release it. Rejected. ("Perl 5.10.1 is just around the corner, really, this time we mean it! Small releases are silly and expensive because of stat calls! A regular release cycle is like believing in astrology!")

But we're not in the land of toy projects here. We can't bless any change (or revert it two releases afterwards) just because it looks shiny at the time. This is not the parrot you're looking for. People do use Perl 5 for serious things, and we must ensure that a certain level of quality is preserved.

I can predict the day and date of Parrot releases into the next century. The number of contributors to Parrot grows over time, as does the frequency of commits. Parrot right now supports features that the Perl 5 VM will never support, and that feature list will only increase over time.

Dismiss all of that because Parrot is neither as widely-used nor as old as Perl 5 if you like, or even because you think I'm a jerk and you hate everything I care about, but there's one overriding metric for software project management: can people use the software? (The corollary is that unreleased software effectively does not exist.)

Quibble about the length of our deprecation policy. Complain that we haven't implemented your pet feature in a way that you like. Beg us to move a milestone forward. Berate us for not fixing a bug you don't report. That's fine.

Yet I believe almost every Parrot committer will agree that our defined support and deprecation policies and our regular release cycle have rescued the project from the very real threats of irrelevance, quality problems, and uselessness.

"Shut up and write some code!" is just a polite (relatively) way of saying "Do it our way or not at all, and above all, do not question the process or its results -- unless you're a core hacker."

Parrot releases are harder to count because of their prolific release cycle.

Brian Wisti, In Which Brian Whinges About The Perl 5 Release Schedule

This is a trivial nitpick (read the rest of the article! It's very good!), but Parrot releases are very, very easy to count. The same goes for Rakudo releases.

The other day someone asked what Parrot might look like in a hundred years. I laughed and thought, "What would its version number be?" Then I realized that I can predict its version number in 100 years. Parrot 101.6 will be out, with Parrot 101.7 on the way.

For all of the lofty talk about "stability" and "maturity" and "predictability" which results in Perl 5 not getting released, the fact that I can predict the release date and version number of a piece of software one hundred years in the future says something about the stability, maturity, predictability, and reliability of a very different kind of development process.

(Oh, and Alias -- I can crash several so-called stable releases of Perl 5 with a one-liner the same way you crashed Parrot in December 2008 with a short program. If you want to make the argument that the 30 stable monthly releases of Parrot in a row don't actually exist because they had bugs, the two so-called stable releases of Perl 5 from the same time period don't exist either. Ontological debates are easy to lose.)

The Perl 6 design team met by phone on 27 May 2009. Larry, Allison, Patrick, Jerry, and chromatic attended.

Larry:

• changed the time function to return a Rat
• thinking about the traits that have been bothering Jonathan and others
• have some changes to check into the spec when I'm happy with them
• thinking about the primitives we use to define use
• breaks down into load and import
• thinking of establishing compile-time keywords for both concepts
• intended so that I can import from anything acting like a module -- an inlined role, for example
• otherwise trying to keep up with the flow of IRC

Allison:

• working on the Parrot book
• changed its focus to a small, 100-page PIR book from a monolithic Parrot book
• the intent is to get something out for YAPC and OSCON
• will send out a draft for review
• will merge my changes into the repo later this week

Patrick:

• released Rakudo #17 last week
• was easy again
• 875 more tests since #16, so we pass 68% of the spectest suite
• finished implementing the root_new opcode in Parrot
• cleans up a lot of the PMCProxy issues from moving Rakudo to its own HLL
• gained half of the speed we lost from the migration
• we'll get more back as we update more places that need it
• NQP never expected anything like that
• I have to rework some it and PCT
• haven't quite figured out how to do that
• refactoring use and import in Rakudo
• the current implementation doesn't work
• will hopefully match with what Larry's putting in the spec
• it seems like the logical way to do things
• updated Rakudo's ROADMAP in docs/ROADMAP
• gives us an idea of dependencies and next tasks
• may also help people understand what blocks features they want

Jerry:

• the bonding period has ended for GSoC
• time for students to start coding
• everyone on the Perl 6 and Parrot projects is ready

c:

• fixed some memory leaks in Parrot and Rakudo
• there are still some in Rakudo, but the web examples should be able to live longer
• did more profiling
• think NFG is important for Parrot in the near term
• have some documentation to write
• have been editing the Parrot book

Patrick:

• how's the command line for Rakudo coming?

Jerry:

• I expect to get back to that

Patrick:

• the "parens build captures" decision surprised me
• what's the rationale?
• I really liked "parens mean grouping"
• maybe I haven't reconfigured my worldview yet, but it feels messy

Larry:

• when used in an argument list, it has the same effect as a capture

Patrick:

• it even works when they're used as a term

Larry:

• they still mean that you have to look at what you're binding to and decide
• am I binding this to a scalar or to an array?
• (1, 2, 3) bound to an array...

Patrick:

• I'm going to have to think about that
• the zip operator in slice context....
• is this three or one positional arguments? zip($a,$b,$c)
• how many positional arguments are in this case? zip($a,$b,$c;$d)

Larry:

• one slice
• you wouldn't want to write that

Patrick:

• what in the arg list distinguishes the use of the semicolon versus the comma
• inside of an argument list we have to recognize a variety of syntactic things
• comma, semicolon, colon, array or hash sigil, named parameters
• seems like captures need more information than just positional
• they need to store metadata about positional arguments
• I like the syntactic stuff showing up in the argument list
• but I don't want to handle them in three different ways

Larry:

• I'll have to think about that

Patrick:

• haven't figured out how to deal with slice context either

Larry:

• might say that the presence of a semicolon implies the presence of other parens
• the comma implies...
• that might be more consistent binding for a top-level list

Patrick:

• I half expected that answer
• I can see the semicolon as just a lower precedence grouping operator

Larry:

• otherwise you have a semicolon that's just not there in every other argument list

Patrick:

• assuming that, the other commas form an argument list through the infix semicolon
• an array in there means Capture of Capture of Capture
• we were about to refactor List and Array in Rakudo anyway
• the question is "Do we really have a List type now?"
• Rakudo assumes that

Larry:

• if we can unify args list with List, that's probably healthy

Patrick:

• I'd really like that
• that makes things a lot cleaner
• infix comma and infix semis now just create arglists

Larry:

• or Lists
• if you define List as "something that has out of band metadata"

Patrick:

• any more decisions that you can make about that will help our implementation
• I probably won't get around to that this week

Larry:

• we make syntactic distinctions
• we know that this is an arg list
• we treat pairs as named arguments
• we don't do that if we know it's not an argument list
• it stays positional
• that's the only distinction between an arg list and a List
• purely syntactic

Patrick:

• to summarize
• zip($a, $b, $c) has three positional arguments
• zip($a, $b, $c; $d) has two, the first of which is itself a list/capture

The Perl 6 design team met by phone on 20 May 2009. Larry, Allison, Patrick, Jerry, Nicholas, and chromatic attended.

Larry:

• trying to delegate some of my spec writing
• if someone has something that needs to change and that person understands the design principles, someone can go ahead
• Moritz respecced each to make it a conjectural junction
• other people are working on the NFG specification
• I haven't seen any changes committed on that yet
• thinking about how importation works
• particularly with inlined modules
• expect that there will be an import declarator implied by use but usable explicitly
• if you say import with an inline module or role declaration, it'll perform the export
• which won't happen by default
• thinking about how that declaration works at the moment
• instead of trait_auxiliary and trait_verb, which fill the same syntactic category, we have trait_mod
• they all occur in the same spot
• you're always looking for them both simultaneously
• the hander routines are now all upperclass autocalled, like other handlers
• IS or TRAIT_IS or APPLY_IS, or some such
• still thinking about namespaces for typeless traits not based on classes
• perhaps rw or readonly traits aren't types
• they need to be in a namespace somewhere
• maybe just a syntactic category
• added the use of ampersand variables on method calls to STD, .&foo
• you can already say $.foo, call a code reference as if it were a method
• & is also a code reference; ought to be allowed there too
• Jonathan was playing with ampersand attributes, .&!foo
• the rudimentary POD parser now complains if you don't have a matching =end for your =begin, except in the case of =begin END
• if you put a default onto a named parameter, parser assumed it to be a positional parameter
• gave a bogus error message
• I fixed that
• another interesting grammar tweak
• defining an infix:<< >> and then a signature
• if you leave a space out between the name and the signature parentheses, it would misparse
• French angles ambiguously hyper parentheses there
• I didn't want to require the space there
• it's nice that you can write the declaration as if it a were call with the parens
• if you're going to hyperoperate on parentheses, you have to use the dot form
• >>( is now specifically not a hyperoperator within interpolation
• >>.( is
• that seems to be pretty DWIMmy
• everything else is cage cleaning and hanging out

Patrick:

• did not write any posts
• will fix that first
• mostly worked with others to make these happen
• Tene and I ported Rakudo to run in its own HLL namespace instead of Parrot's
• caused a 40-50% slowdown in execution time
• we know the cause and have a fix before tomorrow's Rakudo release
• I'll compare Rakudo's speed to before the switch
• want to credit chromatic for any performance optimizations

c:

• I have a 6.5% improvement for all function calls

Patrick:

• we have better error reporting now
• can point to the point in your code with the error, rather than some point in generated PIR code
• added the ability to define custom operators in Rakudo
• infix, prefix, postfix
• in select cases also circumfix
• we can move even more parts of the builtins to Perl 6 as a Setting
• exposed a few other places we need cleanup
• Jonathan is working on those
• added a variable to PCT which gives the compiler the names of the files it's compiling
• improved error reporting there
• added an inline PIR construct to NQP to match Rakudo's
• added the qx// quoting term
• can now run shell commands and capture the output into a variable
• answering questions online and taking care of other little things
• preparing for tomorrow's release
• though it may occur very late tonight

Allison:

• mainly release prep this week
• lots of ticket review and patch application
• submitted an article to Linux Magazine, "Why Parrot is Important"
• more high-level than a tutorial
• continuing to work on the book
• a good week for high-level design discussions
• seems like that happens when I have more time to be on IRC

Nicholas:

• an update on smart-match
• Paul Fenwick gave the position of a Perl trainer
• the design decision of not being able to overload the left side of the smart match bothered him
• there's more discussion there

c:

• optimizing Parrot and Rakudo
• finding bottlenecks
• fixing many of them
• editing the Parrot book
• doing a little more design work on nanoparrot

Nicholas:

• how far off are you from letting other people release Rakudo?

Patrick:

• the instructions are in the repository
• they're straightforward
• I could turn it over to Jonathan or Jerry or any number of people with ease
• they could just do it
• it's basically a Makefile target
• you do some prep work, run the target, then upload the tarball to the right places
• I stole liberally from how Parrot does things
• I can hand it off any time I get tired of making them

Nicholas:

• so the bus number is low

Jerry:

• any plans to turn it over soon?

Patrick:

• if other folks think it's important, I can do that
• I could pick someone at YAPC::NA to do it
• the release day is the Hackathon day there
• we could just walk a few people through it that day
• even if they don't upload the tarball and push the tag
• we'll target that for the June release, even if no one else releases an official release

Jerry:

• I've been trying to get the Parrot release bus number above 10
• we're close, which is a great place to be

Patrick:

• that was evident, as yesterday's release manager couldn't make it, and Mark Glines picked it up

Jerry:

• and it was his first release

The Perl 6 design team met by phone on 13 May 2009. Larry, Allison, Patrick, Jerry, Will, Nicholas, and chromatic attended.

Larry:

• continuing to think about how braided languages work
• implementing the notion in the STD parser
• the notion of how you switch between these languages
• now called slangs
• thinking about how macros work in embedded lexical scopes
• if you define a role which contains a trait auxiliary definition
• label it is export
• how does it export?
• to which lexical scope?
• when you do the role or what?
• no defined way of exporting to outer
• that's a problem in some tests
• with augment slang, we have a way of embedding grammar modifications inline
• when do ordinary macros embedded in subscopes desugar to that sort of thing?
• %*LANG now holds the whole language braid
• the $*PARSER variable is just the MAIN element of that
• I backed off some of the errors in the STD parser to just warnings
• don't really handle module and role long names yet
• you get bogus "can't augment" warnings now
• it thinks something isn't there that should be there and vice versa
• fixed a test where STD needs to parse =for; now it does
• added the slang declarator so that we could augment them
• a twigil indicates a current slang subset of the current braid
• fixed a bug to recognize the invocant marker after an unspace
• that was a longest-token match problem
• the message I added about a missing block gobbled by a function is now two separate messages
• one talks about the function which needs parens
• the other talks about the missing block
• they point to two different areas as a matter of clarity
• I'm really enjoying adding clever error messages

Patrick:

• Rakudo now passes 11,200+ tests
• up by 70 tests since yesterday
• a couple of hundred in the past week
• mostly doing little bug fixes and refactors
• typically, I'll start to add a feature and find some code which doesn't look right
• then I add the feature
• slurpy arrays and hashes weren't added properly to subs
• always added automatically regardless of the type of Block
• I cleaned that up
• also cleaned up lots of parameter handling in the code
• that made actions.pm 100 lines shorter, and I removed three duplicate subroutines

c:

• that also speeds up startup

Patrick:

• every time we do that it helps
• found a bug dealing with Unicode characters in grammars
• prevented us from properly parsing French quotes as an angle delimiter
• working on adding operator parsing to Rakudo today
• define your own custom operators
• refer to them using canonical names
• expect to have that done today
• they won't be lexically scoped
• I'll work on fixing that, when we get more to lexical scopes and language braids
• continuing to consider protoregexes and LTM in PGE
• probably will implement contextual variables in NQP and Rakudo

Allison:

• we'll have packages in Ubuntu soon
• just a straight sync of the Debian packages
• no reason to make Ubuntu-specific packages, if that works
• working on editing the book
• working on the calling conventions branch
• writing a Parrot article

Jerry:

• GSoC mentors and students seem to be bonding in their bonding period
• set up parrot.org to accept donations from individuals
• there's a sidebar link under the Foundation heading
• and we received our first donation!
• thank you, Bradley Kuhn
• we'll work on placement and prominence
• think I'll give a keynote at YAPC::NA, following Larry

Larry:

• if you pay me enough, I'll run over

Patrick:

• if you don't pay him, I suspect he'll run over
• you are what stands between everybody and lunch

Will:

• added some regression tests
• made some stuff work in Perl 5.8.8, so we don't have to upgrade our dependency
• wrote a script to summarize our old branches
• convinced people to delete a bunch of old branches
• hopefully we can weed them out
• TPF's grant committee votes this week on the current crop of proposals
• one of them is for Perl 6
• if you have any comments, please post on that
• Partcl is still dead
• blocking on the ability to build a language from an installed Parrot
• we've made some progress on that
• there are still things that do not work
• most of them have tickets

Patrick:

• Rakudo's blocking on the same thing
• Tene is working on getting Rakudo to run in its own HLL
• he's blocking on the HLL load_library problem
• I'll have him post to the list and give an update on the ticket

Nicholas:

• Rafael has merged his smart-match branch back into blead
• he tried to make smart-matches match ranges
• it won't work for various reasons
• precedence is the wrong way around for one
• $foo ~~ [ 1, 2, [ 3 .. 5 ] ] won't work either
• Perl 5 ranges aren't lazy
• smart-match can't interrogate Range objects lazily
• that's why we can't emulate everything from Perl 6
• we don't have the low-level infrastructure to do it all
• even if we had that infrastructure, there are parsing issues
• I don't see that coming in a hurry
• if someone implements lazy ranges, maybe someone can add them to smart-match
• giving the enjoyment people derive from implementing such things, I'll be surprised if anyone does that

c:

• added some optimizations
• heading toward the "not really worth doing" kind
• fixed some bugs
• including a Unicode parsing bug with named parameters and arguments in IMCC

Nicholas:

• did you write about the abolition of unary equals, Patrick?

Patrick:

• not yet
• I feel badly about that
• I need to catch up
• there are so many interesting problems to work on
• writing is not as interesting as working on code

The Perl 6 design team met by phone on 06 May 2009. Larry, Allison, Patrick, Nicholas, and chromatic attended.

Larry:

• refactored my phone
• refactored various functions
• comb now defaults to matching single characters
• a words method now does what comb used to do
• did some work on pick
• looked at most of the functions in containers to see whether they out to return a List or a Capture which returns an Item
• depends on whether the function would ever be used to pull a single thing out
• whether people expected it to do the right thing as an Item
• pick is one of those functions
• worked on the semantics of has
• the initializer semantics; the syntax hasn't changed
• worked on the relationship between multiple has if you have multiple attributes
• straightened out a mismatch between prefix parsing specification and implementation
• turned the term "protoobject" into "type object"
• thinking about the notion of braided languages
• Perl isn't a single language
• also a regexp language, a quoting language, a quasi-quoting language
• they hand off to each other
• they have to keep track of each other; I call that a "braid"
• how you override one language in another
• how do you handle regexp language changes from regular Perl, when you're not in the regexp language at the time?
• thinking about how to get the derivations to work correctly
• when you parse a regexp or a quote
• now instead of starting at a language like q, it starts at the current q-ish language in your braid
• in support of that, I implemented temporization of context variables in gimme5
• used to temporize current braids to a lexical scope
• added a block after try in parsing
• it uses that rather than parsing the block as the first argument of a list operator
• likewise the other statement prefix operators
• added the auxiliary hides trait
• the \c notation for characters allows any radix of integer inside the list
• the \c, \o, and \x notations all parse consistently now
• enums are parsed more correctly
• they don't give bogus duplicate warnings
• if you declare a lexical symbol with :: in it, assumes you're referring to a subpackage of the current lexical scope
• means something different from a symbol intended to scan outward
• along with the notion of braided languages and derivability, derived languages can add more parser variables ($?foo) with a virtual function for lookup
• avoids clobbering the core definitions
• worked on role parameters
• they're now in the role's pad, like normal signature parameters
• started playing with the YOU_ARE_HERE stub used to define the effective insertion location of your code into a Setting
• determining which lexical pad to dump as the Setting's context for use by other compilation units
• did a lot of work on error messsages
• some of my ||s suppressed panic messages
• more useful error message on the null pattern (//)
• gives a better message if you slurp your control block in the preceding expression

Patrick:

• had a lot of discussions online
• not as much coding as I like
• seem to be answering questions for others
• quite a bit of work on Rakudo internals refactoring
• that'll make it easier to switch it over to be its own HLL in Parrot
• moved things around
• made a single constant definition we can change when we move it over to its own HLL
• also moved all of the Parrot HLL mangling into its own source code files
• easy for us to manage there
• fixed some bugs
• talked with Jonathan about refactoring the P6object implementation
• used in Rakudo and PCT for Perl 6 object-like behaviors
• he's working on that refactoring now
• found and fixed a couple of bugs handling implicit slurpy positionals
• made Rakudo's standard IO handles default to UTF-8 encoding
• gives people behavior more like they expect
• working on changes to the Array and List implementation
• currently List inherits from ResizablePMCArray
• we'll change that to contain it
• that should clean up a lot of behavior
• Rakudo passed the 11,000 spectest mark as of two days ago

Allison:

• spent some time on ticket, milestone, and roadmap maintenance
• helping other people start or continue their tasks
• launched the install PDD out of draft
• it doesn't address every question, but I cleaned out all of the inaccuracies
• fits our current plan
• probably needs more expansion when we figure out the source of some of our problems building Rakudo and Partcl
• the general problem is that we've built all of our languages from the build tree, never and installed Parrot

c:

• I've been explaning Perl 6 roles on my new weblog
• lots of feedback
• still lots of explaining to do
• also been profiling Parrot and Rakudo startup
• Parrot's a lot faster now (cotto and bacek's work on vtable init helped immensely)
• Rakudo's getting faster, but it has a ways to go
• will keep working on that, but the progress has been decent with little investment
• discussed exception handling with Allison
• came up with a preliminary design to handle the interior runloop exit there
• just syntax to hide tricky semantics

Patrick:

• when dealing with exceptions, handlers, and returns for subroutines
• it'd be nice to force a return from a subroutine higher up in the caller chain
• akin to Perl 6's leave semantics
• trying to do that without throwing an exception....
• if we do it with exceptions requires every block to have a handler

Allison:

• invoke the return continuation several levels back?

Patrick:

• is that easily doable at this point

Allison:

• we have all of the ability for that, but no syntax for it

Patrick:

• I don't necessarily need syntax for it
• the natural approach is to find the appropriate caller
• I need to do that anyway
• then ask for its ReturnContinuation
• then invoke that with the values I want to return to its caller

c:

• makes sense to me

Patrick:

• that may exist in PIR syntax like that anyway
• if the easy invoke doesn't work, use some combination of set_returns

Allison:

• not sure if we have in PIR right now the way to ask the caller for its return continuation
• you can do it from C
• we can add something which does that

Patrick:

• if I ask for a Sub's current context
• I could write a method on a sub to do that
• it'll be easier and nicer when our contexts are PMCs
• as an intermediate case, that would be it
• related to that, this might be there too
• is there a way for a Sub to register behavior to execute just before it exits

Allison:

• are those like the LEAVE hooks in Perl 6?
• we have a place to store them now
• we don't right now have a way of flagging a particular handler to execute at block scope exit
• would it make sense to call that an event?

Patrick:

• it might
• before saying what we do and don't have
• one piece of this is that there's still a push_action opcode

Allison:

• it never worked
• we're removing it
• the global stack never synchronized with continuations
• it dummied up to work, but it doesn't have the automatic stack cleanup behavior

Patrick:

• the other possibility is the presence of ONEXIT or LEAVE hooks on Subs in Parrot

Allison:

• do you want all Subs to have that
• or at runtime would you want to attach a handler to that sub?

Patrick:

• it might be nice to have that at compile time
• I suspect it'll end up being an attached block

Allison:

• can we tie that cleanly with other execution pieces?
• NEXT and LAST etc

Patrick:

• the biggest difference with LEAVE is that it's not exception-based
• Larry thinks of it as natural to a block
• PCT models the rest as exceptional

Allison:

• it sounds like a handler to store in the local handler list
• mark it so that it's clearly not an exception
• we have handler flags -- exception, event, etc
• we add to the execution code for Subs
• as it returns or ends, it checks to see if there was a handler
• invokes it if so

Patrick:

• sounds good to me
• it's come up if you're thinking about these issues anyway

Nicholas:

• how useful is MAD that's in Perl 5 blead still?

Larry:

• I don't know
• it may turn out to be very important
• depends on how possible it is to write a decent Perl 5 parser in Perl 6
• where decent is a very peculiar definition meaning "indecent"
• this is unknown
• for interleaving Perl 5 and Perl 6 on various VMs, some VMs will use a Perl 5 compiler written in Perl 6
• that may not be a solution as nice as using the real Perl 5 parser which knows its own idiosyncracies

c:

• sometimes it does

Larry:

• that's the point
• I'd hate to have to have to hack it up again
• that was a lot of work

Nicholas:

• it seems to suffer slow bitrot
• no one seems to try to keep it up to date
• when people change the parser, they don't change it
• the question is if people still use it

Larry:

• the vision is still to have a Perl 5 to Perl 6 translator
• that's been on hold until we have a viable target to translate to
• the demand for that will re-arise at some point
• if it bitrots, it bitrots
• that doesn't bother me as much as if people felt like ripping it out
• the way to unbitrot it was roundtripping the test suite and fixing any problems
• we'd just go back to doing that again until everything was reannotated the way it needs to be
• the standard for when you know you're storing enough information is being able to reproduce it

c:

• is there a test target to handle that?

Larry:

• I used special scripts to do that
• didn't expect other people to maintain that
• it's MAD for a reason
• nobody's tried to write a Perl 5 parser in Perl 6 yet
• it's theoretically possible

Patrick:

• depending how much fidelity you need for the actual Perl 5, you can get a pretty good way

Larry:

• considering how much effort has gone into PPI, you can get a PPI-like result
• that only takes you so far though

Patrick:

• maybe a GSoC project next year would be for someone to start hacking something together with PPI
• it'd be a worthwhile project

Larry:

• we already have a Perl 5 to Perl 6 translator based on the state of MAD

The Perl 6 design team met by phone on 29 April 2009. Larry, Allison, Jerry, Will, Patrick, and Nicholas.

Larry:

• let's start with the specs, in reverse chronological order
• we reserved hash notation in rule syntax, so it does not now have to be supported. It was kind of bogus anyway.
• subsets and enums were defined with a syntax that didn't allow traits, so we couldn't export them. Now traits are allowed right before the subset's where clause or the enum's list.
• while messing with the enum syntax I decided that using [] around the list didn't make sense, so that changed to (), to be consistent with the <<>> and «» forms, which are defined in terms of parenthesized lists. In any case, some form of bracketing is required to prevent ambiguity with any preceding trait, so the parens may not be omitted, if that form is used.
• in S3, temp and let are parsed as named unary ops
• there was an ambiguity in how Unicode bracketing characters were parsed. We can't allow one to many, but no there's no problem with many to one. That is, there are many ways to start a particular quote, but only one quote [character] that ends it.
• that's fine
• had to think of this, because we added Pi and Pf as valid characters, which they already were, because of the way we use the various angle brackets
• the lines function now has a limiter on it, so you can tell it how many lines to input if you don't want all of them
• big news of last week was killing off prefix = and renaming readline to get
• put the the split function back to taking the delimiter in the first argument, as it was in Perl 5
• documented the when statement modifier, which we'd all assumed was there, but wasn't actually in the specs
• STD hacking:
• tweaked the table of openers and closers to match S2.
• killed off the = prefix operator, both there and in the tests, including the poor fish operator
• today I hacked in duplicate checking for type names
• been doing various simplifications
• got rid of the fulltypename rule, Combined it with typename.
• last week got rid of the extrapost rule, at Patrick's suggestion

Patrick:

• I don't remember asking for that, just asking what it is

Larry:

• it's a simplification that you triggered.
• better error messages about variant forms of elsif
• modified the grammar for exportable subsets and enums
• noticed a few differences from the spec:
• the spec allows double angle brackets for enums, and I didn't successfully parse that
• the non-declarative enum that acts like a listop is now parsed as a listop, rather than being special cased
• refactoring to support the new caps method
• had to refactor PRE and POST
• the way chained and list precedence is reduced
• found a few parse tree malformations:
• the symbol is not copied up for the X operator brackets, so it was not having the correct precedence
• my nouns were dangling from the terminator nodes after them, and that was not good.
• driven by ongoing goal of making viv replace gimme5
• making gimme5 more regular in its output: it no longer uses auto-quoting inside curlies and it spells out the quotes exactly
• got most of the method expr correctly translated by viv now
• probably another couple of weeks or so before I can get viv doing everything gimme5 does
• the viv AST (VAST) used to bless objects into generic classes for (basically) precedence level. That's too generic for a lot of things -- you want to know which rule matched, so it now blesses to a class for each node. Those classes can be made to derive from a more generic class.
• people have been asking for per-rule for some time -- I probably should have done that sooner

Patrick:

• Rakudo release number 16, codename 'Bratislava", went incredibly smoothly. Really easy. What. I want.
• wrote a release guide: the steps you do to make a release
• can pass that off, when we want to groom other people to make releases, but really easy right now
• did lots of small fixes. Getting rid of deprecations and things.
• stringification of proto-objects in Parrot and Rakudo to match the spec
• regexps do the right thing in boolean context now
• regexps compile to a Regexp type, rather than a Parrot subroutine, so we can do multi-dispatch on a Regexp now

Larry:

• yay

Patrick:

• fixed the alpha rule so it includes _, which it didn't before
• inside character shortcut \c, fixed so it allows whitespace, but not sure if it's part of the spec

Larry:

• actually the spec does have whitespace

Patrick:

• is that true for all the other \ escapes, such as \x and \o?
• can I put spaces around the numbers in each of those?

Larry:

• probably
• the \x already parses whitespace

Patrick:

• worked with Moritz to figure out how to traverse parse trees in Perl 6
• worked with Tene to get Rakudo to work in its own namespace, rather than the Parrot namespace
• found and fixed SEGVs [related to that]
• lots of little other things
• need to do blog reports and roadmap items.
• keep getting distracted by other more fun bug fixing things
• need to turn off all my IRC windows and write text
• planning to work more on getting Rakudo working as a HLL with Tene and getting Rakudo to build with an installed Parrot

Jerry:

• this week, getting set up for a Parrot VM workshop, working with the organisers [of YAPC::NA] and TPF, and put together an actual schedule, to get some structure.
• we have 25 or 26 people signed up already
• it's the weekend before YAPC::NA this year
• free at the Urban Mountain gathering place, in Pittsburgh
• it's a workshop for Parrot and languages on top of it
• learn about Parrot, PCT, Rakudo, etc

Allison:

• went to SF and up to Bellingham, Washington, giving Parrot talks last week
• went well, good feedback
• travel takes time, so not as much development work
• worked with Partcl, to get it up and running, to find out issues about building languages off an installed Parrot
• I have a list of three core things to fix off of that, plus helping Coke with Partcl
• made changes to language shell generator, because as I was writing the talk I found things that were hard to explain
• there has been some discussion on the TPF board about Richard's proposal for the Hague grant. This is ongoing.
• historically TPF directors have stepped down before taking funding
• Larry stepped down
• I only took travel funding
• not stepping down is legal, but controversial inside and out
• the work is definitely worthwhile
• collecting info for the 501(c)3 application for the Parrot Foundation
• it's the mother of all forms, about 27 pages long

Will:

• post the Parrot 1.1 release I updated docs.parrot.org to reflect that
• hopefully we can automate it for future releases, but it's already much easier than the old Parrot site.
• had stabs at getting Partcl running again
• it's at the point where it builds the C components
• some issues to iron out before the PIR will build

Nicholas:

• why did interpolated hashes get eliminated from rules?

Larry:

• all superseded by the longest token matcher
• it was a preliminary attempt to do a longest token matcher, but fit it into notation of hash, but it didn't really work
• it's a pain to implement it the way that it was spec'd
• I don't think that anyone really wanted to implement it

Patrick:

• it's a real pain to try to implement, but we have no strong need.
• we took it out, but reserved it so that it could go back in
• I didn't want to implement it
• I resisted it for three years, and I'm glad that I did.

Patrick:

• the main reason I wrote create_language as an alternative to mk_language_shell was that others had checked in changes that I didn't want to rip out by fiat, as we're doing things differently

Allison:

• things that Rakudo is doing differently could be command line options
• they are additions, not fundamentally different

Patrick:

• one of the things I didn't like was it did use options
• you need to know in advance if you will end up with local PMCs or ops

Allison:

• if you don't know, you don't need them

Patrick:

• you have to figure out the things you need, retrospectively, and how they go into the Makefile
• it would be nice if they were present but commented out

Allison:

• have the same one to generate the Makefiles, and a new script to add things to the Makefiles?
• either you get everything all at once, when you don't need it or very little at the start, but you need to add it later

Patrick:

• adding it later is problematic
• you want to do releases, and to do so you modify the build scripts
• coming up with something able to patch the build scripts implies more modularity, which would increase the learning curve

Allison:

• so does a large section of the Makefile for build processes that you don't need

Patrick:

• also, I don't want separate Makefiles

Allison:

• you need that if you want to use installed Parrots; you can't have dynpmcs

Patrick:

• I was going to do it without those
• I've seen the way it's being done to avoid the scripts that are deprecated
• was going to avoid doing it from multiple Makefiles

Allison:

• (having everything in from the start), you have to understand all that's being dumped in

Patrick:

• if it's in the Makefile, but commented out, then it's already there, so you can just adjust the build

Jerry:

• as a conditional line?

Patrick:

• if you're not doing anything that hits the target, then it doesn't show up

Allison:

• I can accept it with lots of comments, like some Linux packages do
• 5 line of commented out configuration
• 20 lines of explanatory comments
• "You can cut all this out unless you need X,Y,Z"

Patrick:

• Agree
• it would nice if the Makefile said "here's the stuff you need for dynpmcs"

Allison:

• I want command line options to turn it off if I'm making a new language and I know I don't need that

Patrick:

• Also I didn't like make_language because it was making separate Makefiles. That's more that I have to maintain, co-ordinating changes between them.

Allison:

• it's cleaner to have Makefiles in subdirectories
• it's more the recommended procedure
• it has that going for it, but I'll take the complexity argument

Patrick:

• that's the gens
• I didn't want to rip out other peoples' code, but we can unify them
• I needed something for my talk in Norway and mk_languages_shell broke

Allison:

• in the dynpmc there's a vtable function that is wrong -- documentation says that it's setprop, but it's set

Jerry:

• the spec seems to be pretty stable now
• how can we increase the number of implementors and testers this summer?
• are there things that we can do from the design perspective, to make it easier for people to jump on the bandwagon and contribute?

Patrick:

• I was very encouraged by what happened in Norway.
• we ended up with two or three new contributors who have contributed since the workshop

• - part of that, I hope, was how I structured my talks, so that they lead into the hackathon

• I broke out tasks that were hackable
• Rakudo has reached the point where it's possible to assign tasks
• Rakudo has really reached this point now, more so than a few months ago

Jerry:

• once something is added, all the tickets are assigned to moritz++ to write tests. He can only do so much.

Patrick:

• Moritz is a placeholder. It's not that we expect him to do them all.
• It's a way to tweak RT. You can look for all tickets assigned to Moritz.

Jerry:

• or create an anonymous spec test user

Allison:

• or a custom query

Patrick:

• it's easy right now to use the drop down and assign an owner
• we could create a spectest account, but effectively Moritz's account is serving as a placeholder for "needs a test"
• if getting another field is easy, go for it, but a spectest user might be easier still

Jerry:

• lighting talks are a good way to introduce something in a short time

Allison:

• hackathons

Jerry:

• the Parrot VM workshop will be useful
• "Yes, you can do real things with Parrot. It's easy. We need your help."
• you can add Index X<>s to the spec
• you can write tests
• you can make sure that the smartlinks inside the specs work

Patrick:

• my talk "Hacking Rakudo Perl" was 50% about Rakudo and roughly 50% about everything else in the Perl 6 universe that needs hacking
• what surprised me at NPW was that the people who came to the workshop, a lot of what was done was writing modules. We got IO::Socket, XML parsers, and a number of other modules, such as IO::Prompt.
• people building modules that would be for something CPAN-like.
• even discussions on how the modules could be organised into a CPAN-like repository
• I expect more of this this summer
• it wasn't like this last summer.
• it's because you can write builtins in Perl 6
• you can write classes. You can do things.
• you get something, even if what you get may not be reasonable :-)
• NPW was exciting because it was easier to recruit people into the project
• so far the best driver of Rakudo development continues to be people writing applications and modules. They really expose where Rakudo and Perl 6 need help. much more than tests.
• Carl Mäsak's talk about November was great in that respect. As much about why it is important to write modules and applications as about November itself.
• Rakudo was depicted as a Swiss cheese -- it covers specs, but there are holes
• applications sometimes intersect with holes, holes which the tests don't intersect. It's a really good analogy, and an important point.
• Carl's talk was by far my favourite of the conference

The Perl 6 design team met by phone on 22 April 2009. Allison, Jerry, Will, Patrick, Nicholas, and chromatic attended.

Allison:

• worked mainly on the book this week
• reviewing patches, preparing for the release
• did more work on the calling conventions branch
• you have to update all of the last scattered bits throughout the cold
• gave a talk to the SF LUG last night
• good questions there

Jerry:

• worked on GSoC
• TPF has nine slots this year
• two projects are for Parrot and one is for Perl 6
• we originally had 10, but one student had duplicate proposals into another organization
• all of the mentors and students are excited and ready to go
• we're looking forward to the start of coding in May
• getting everyone set up with the proper credentials and prerequisites for commits
• will have a good amount of time to the Perl 6 command line in the near future

Patrick:

• spent the last week at the Nordic Perl Workshop and a subsequent hackathon
• the workshop was excellent and well-organized
• a two-day track for Perl 6 and Parrot
• quite a bit of enthusiasm from attendees about Perl 6 and Parrot
• the initial guesses about attendance were wrong; we had to switch to bigger rooms for the Perl 6 stuff
• LinPro sponsored the three-day hackathon
• it went extremely well
• Larry and I attended Gabor's Perl 6 introduction
• I wanted to see where people would have trouble with Rakudo and the like
• other people worked on various modules in Perl 6
• we now have socket support in Rakudo
• people can write web servers and clients in pure Perl 6, no PIR
• quite a few people asked for something to do
• we gave them challenges that they might not be able to complete
• most of them all worked out though
• we found lots of little bigs
• we found some conceptual problems
• we implemented a lot of new features
• I worked with Gabor on integration of syntax highlighting of Rakudo in Padre
• PCT now provides the information Padre needs
• any PCT-based language could quickly get syntax highlighting through Padre
• that works out very well
• it took some time to get the design right, but Gabor took it from there and started doing amazing things
• Jonathan and I cornered Larry on some issues
• we kept asking him with hard questions and frustrated him with our ideas
• at one point, he started banging his shoe on the table to get our attention
• the prefix:= operator is gone -- iterate an iterator operator

Jerry:

• I'm going to miss the prefix fish operator!

Patrick:

• I'm going to write about it
• when we were trying to nail down laziness in Perl 6 and its meaning, we realized that prefix:<=> was trying to solve two different problems
• they're contextually related
• but multidispatch makes getting context information more difficult
• people in the tutorial found the syntax ugly, especially in a scalar assignment
• reading a single line from a POD filehandle was even uglier: my $x = =$=podhandle;
• it's now .get for a single item and .lines to read in list context
• that makes the code much easier to read in general
• the fish operator is now lines()
• it's more readable for people who don't know Perl
• as Masak has previously noted, whenever someone mentions the need for something (the desire for golf or obfuscation), he says "There will be modules."
• we managed to put Larry's ideas about binding in place
• that's been difficult in Parrot with PCT and PMCs as currently designed
• I think things look better and simpler for us now
• Larry seemed declined to support some things we thought we might have to support, in particular certain binding features
• we'll clean up the binding syntax in Rakudo over the next couple of months
• Jonathan and I spent one evening reviewing and updating the Copenhagen roadmap
• I'll turn that into some readable prose
• we'll have tha sometime this week
• Jonathan finished his grant review and is waiting review from his grant manager

c:

• oh, I know what that means...
• I'll get to that

Patrick:

• he's busy working on his next grant proposal
• we spent a good amount of time clarifying things so he could finish up his grant
• that largely happened at the hackathon
• we made some PGE improvements
• one participant was interested in quantifiers, so he did some of the hard parts of figuring out how to parse regexp quantifiers
• fixed that in a few hours one day
• a feature for people writing parsers
• that closed a couple of RTs
• I'll post in detail about the hackathon happenings in the next couple of days
• it was very productive
• lots of good energy from the participants
• we had at least five or six people just hacking with Rakudo Perl
• someone put together a pretty good XML parser
• we also talked quite a bit about module distribution for Perl 6
• no concrete plans, just broad brushstrokes
• we're looking at something more distributed, like Git and GitHub
• Masak has something in the works with his Proto module

Nicholas:

• CPAN solves uploading and mirroring
• it doesn't solve the distro problem as much
• mapping uploads to different people and such
• how do you lay things out on disk and figure out how to load them?

Patrick:

• we didn't get to that level of detail
• we're keeping it in mind as we go on
• it'll come up sometime over the summer
• very likely a YAPC::EU topic
• working on Rakudo's release
• expect to get that done in the next few hours
• a little early, as we have several pending patches
• no reason to hold the release in that case

c:

• Perl 6 released early!

Patrick:

• ~3190 new passing tests since the last release

Jerry:

• let's see you do that next month!

c:

• that's 100 a day

Patrick:

• 2100 were Unicode tests
• that's still over 1000 tests in the past month
• several people asked how to port the stuff I did from Rakudo back into PGE
• even without Unicode, it was a significant month
• we'll have trouble matching this month's progress
• we're passing tests people have already written
• but people are adding new tests
• the size of the test suite is growing
• we're catching up
• I'm almost to the point where we should look not at absolute numbers but at the percentage of the test suite

Nicholas:

• that number's more meaningful externally
• are you above or below 50%?

Patrick:

• we're above 60%
• 100% is a receding number though
• some parts of the spec are woefully undertested
• the next big chunk is fixing named argument passing
• we can't handle all of those in the right way all of the time

Will:

• Parrot 1.1 came out yesterday
• general station keeping with coding standards tests and ticket wrangling
• I think I fixed an IMCC bug which blocked lexicals with Unicode names
• was a blocker for Rakudo
• hope to steal some of Allison's energy to get Partcl building against an installed Parrot

c:

• merged in some of the header cleanup work
• have a lot left to go
• will be editing the Parrot book in hopes of publishing it very soon

Will:

• the PASM chapter has a lot of bad code in it

Allison:

• I'm working on it now

The Perl 6 design team met by phone on 15 April 2009. Allison, Will, Nicholas, and chromatic attended.

Allison:

• finished converting Pynie to support Python 3.0
• checked in a big batch of changes to the Parrot book
• added a -X command line option to Parrot to add search paths for dynamic extensions
• still working on the PIR calling conventions
• NCI is more of a bog than the straight PIR stuff was
• it's a matter of cleaning up every single place that calls into a PCCMETHOD
• gradually working my way through those
• did more work on the Parrot Developer's Guide book cover

c:

• moving more Parrot code into the proper subdirectories
• cleaning up the headers
• will merge in what I have by mid-day Saturday
• probably won't change any function names; just their locations

Nicholas:

• happy to see that Dave Mitchell is converging on merging blead to 5.10.1

Will:

• cleaning the cage
• trying to enable more warnings and still be clean
• my biggest current blocker is the language switch to living out of the repository
• can't build Partcl right now with the 1.0 release
• I don't see a way yet to make it work
• will try to get it working with 1.1 this weekend

Allison:

• do you want it to build off of only an installed Parrot
• or also with an installed Parrot?

Will:

• primarily want to work off of an installed release of Parrot
• a monthly release is fine
• no problem lagging on features
• I could build my own copy from HEAD
• it can't build dynops because the link information is not present in the public parrot_config
• it relies on the build directory, which is wrong
• heard some talk recently that was rather incorrect on our current status
• we could use someone doing marketing to keep our message up to date
• it's changed some in the past eight years
• but maybe we need a call for a volunteer to keep our information public and current

Allison:

• we need to work more on communicating what we're doing
• I was amazed at how much more response we had from writing a press release and Twittering our 1.0 release

Will:

• not suggesting any spin
• I just want to get more facts out there
• I worked on this with the "This Week in Perl 6" message
• that sucks from an implementation standpoint
• I'll try to write more prose for the next time

Number of stable Perl 5 releases since the Perl 6 project began in summer 2000: 13.

Number of stable Rakudo (Perl 6 on Parrot) releases since the Perl 6 project began in summer 2000: 17.

Before you tell me how unfair a comparison that is and how you hate my hair and I'm a horrible person, let me add a disclaimer: Perl 5 has several orders of magnitude more users than Rakudo.

Before you get all smug and tell me that I'm correlating meaningless numbers for no good reason, let me revoke part of that disclaimer: in a healthy project, at least some of the users also contribute something back.

The only interesting question to me now is "Why not?"