Stories
Slash Boxes
Comments

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

pmichaud (6013)

pmichaud
  (email not shown publicly)
http://www.pmichaud.com/

Patrick Michaud is the pumpking for the Perl 6 compiler. He holds a Ph.D. in Computer Science and was formerly a Professor of Computer Science at Texas A&M University-Corpus Christi. He is currently a software developer and consultant focused on open source development and applications, including Perl, PmWiki, and Linux.

  Rakudo Perl now passing over 1000 spectests! 2008-06-27 13:19 pmichaud

Submitted by pmichaud on 2008.06.27 13:19
[Rakudo spec regression status: 75 files, 1080 passing tests]

Rakudo Perl is now passing over 1000 tests in the official test suite!

http://www.pmichaud.com/perl6/rakudo-tests-2008-06-27.png

For those who are interested, the history of test results is available in CSV format from http://svn.perl.org/parrot/trunk/languages/perl6/docs/spectest-progress.csv .

Thanks to Jonathan, Moritz, Auzon, bacek, particle, Coke, chromatic, and everyone else who is helping to make this happen!

  Perl 6 on Parrot (Rakudo) progress report 2008-04-02 17:11 pmichaud

Submitted by pmichaud on 2008.04.02 17:11
This is a long-overdue report on the state of Perl 6 on Parrot (now "Rakudo Perl") development. In late 2007 I had been writing progress reports on a regular basis as as part of a grant from the Mozilla Foundation (MF) and The Perl Foundation (TPF), but in early January I had to pause work on the grant and my reporting to take care of a family emergency. I'm very glad that things have now settled down and that I can focus energy back on Perl again.

In spite of my distractions, a lot of progress has been made on Rakudo since last the last major report I wrote in December 2007. The work performed in November and December of last year has opened the door for many others to participate in Perl 6 and Parrot development, and as you'll see in this report, many have done so. I'm very pleased with the team of developers that has arisen and the quality of tools we've created for compiler development on Parrot. If anything, the hiatus has shown that our ability to advance isn't dependent on a single individual.

Progress, December 2007 to March 2008

An early draft of this report started by going into substantial detail about the progress that has been made in recent months, but I soon realized that so much has happened that the resulting report would be far too lengthy to be interesting or useful. Thus for this report I've decided to briefly highlight some of the events and milestones reached to give a good sense of the progress being made, and then write later articles that provide more detail. If I've overlooked an important contributor or event, please accept my apologies and let me know so I can highlight them appropriately.

* In January, the Perl 6 on Parrot compiler was renamed to "Rakudo Perl", or "Rakudo" for short. "Rakudo" is short for "Rakuda-do", or "The Way of the Camel" in Japanese. ("Rakudo" also happens to mean "paradise" in Japanese, which is nice.)

* A tool created by chromatic and Jerry Gay (with contributions from Will Coleda, Bernhard Schmalhofer, Andy Lester, and Francois Perrad) makes it possible to build an executable "perl6" binary. As a result, many people can now use Rakudo to execute Perl 6 programs by simply typing "./perl6 script.pl".

* In December we substantially completed the implementations for the Parrot Compiler Toolkit (PCT) and Not Quite Perl (NQP). The end result is that many of Parrot's HLL compilers are now written using Perl 6 source code instead of PIR.

* Jeff Horwitz continued to extend the implementation of mod_parrot to include scripting in Perl 6 (Rakudo) and PHP (Plumhead).

* We've increased the quantity and quality of documentation on Parrot's compiler tools to make them more accessible to a wider range of programmers. As part of this, we also created a "make a new Parrot language shell" tool that populates a directory with a basic setup for implementing a new compiler in Parrot.

* As an unplanned demonstration of how well the tools work, in January Will Coleda and Simon Cozens used the new tools to create a working LOLCODE implementation (http://lolcode.com/) from scratch with just a few hours' effort. Simon wrote about the experience and LOLCODE has served as a great example for others who are interested in compiler development under Parrot.

* We developed and initiated a plan for reviewing and refactoring the Perl 6 tests of the Pugs repository to become the "official" Perl 6 test suite (http://svn.pugscode.org/pugs/t/spec/).

* Larry Wall and Jerry Gay designed and implemented tools to allow the official test suite to be easily used by multiple Perl 6 implementations, including those that may be incomplete in the early stages of development. Both Pugs and Rakudo are now using these tools (fudge/fudgeall).

* Jonathan Worthington added junctions, classes, objects, roles, given/when statements, and many other significant features to Rakudo. He presented his work at several conferences and workshops, and wrote an article about his positive experience using the new tools to add features to Rakudo.

* Jerry Gay added support for radix conversions, command line argument handling, new test harnesses, many builtin functions and operators, and a perl6doc parser. He has also continued to serve as a mentor and advisor for other Rakudo developers.

* Recently Klaas-Jan Stol completed a draft tutorial for building compilers using the various compiler tools in Parrot.

* We added a compiler overview that describes the architecture and layout of the components of the Rakudo Perl 6 compiler.

* Since December at least twenty individuals contributed patches for new features to Rakudo Perl and the Parrot Compiler Toolkit. In addition, the Parrot subversion repository received a record number of commits in the months of December (1175) and January (1025).

Where things are headed next

The overarching goal for the next couple of weeks will be to again boost development momentum on Rakudo Perl and Parrot compilers to the levels we were seeing in December and early January. Because we now have many of the compiler tools and core code components in place, parallel development efforts for Rakudo are much more possible than they have been in the past. A key task for the next couple of weeks will be to recruit and support additional individuals who can help with the implementation effort.

Specific tasks for the upcoming weeks will include:

* Continue building and refining documentation for the compiler tools and Rakudo implementation
* Review code added to Rakudo in the past couple of months
* Perform a refactor of Rakudo's base classes and namespace handling to better support junctions, classes, and objects in Perl 6.
* Add control exceptions
* Add more hash and list operations
* Continue improving and refactoring the official test suite
* Update test system to provide graphical views of passing/failing tests in Rakudo

Thanks for reading!

Pm

  Parrot 0.5.3 "Way of the Parrot" released! 2008-02-21 03:00 pmichaud

Submitted by pmichaud on 2008.02.21 3:00
pmichaud writes "On behalf of the Parrot team, I'm proud to announce Parrot 0.5.3 "Way of the Parrot." Parrot is a virtual machine aimed at running all dynamic languages.

Parrot 0.5.3 can be obtained via CPAN (soon), or follow the download instructions. For those who would like to develop on Parrot, or help develop Parrot itself, we recommend using Subversion or SVK on our source code repository to get the latest and best Parrot code.

Parrot 0.5.3 highlights:

The Perl 6 on Parrot compiler has been given the name "Rakudo Perl". More details on the new name are available from http://use.perl.org/~pmichaud/journal/35400. In addition, Rakudo now has more support for objects, classes, roles, etc., and a better interface to the official Perl 6 test suite.

More languages are being converted to use the Parrot Compiler Toolkit.

Parrot 0.5.3 News:

- Documentation
    + PDD09 (garbage collection) — approved
    + PDD28 (character sets) — draft started
    + added function documentation to some core functions
    + PCT beginners guide, optable guide and PAST nodes guide, bug fixes
- Compilers
    + IMCC: plugged various memory leaks and other cleanups
    + PCT:
        . add "attribute" as a scope variant to PAST::Var nodes
        . add 'shift' and 'pop' methods to PAST:: nodes
    + NQP: add '=:=' op, tests for scalar and list contextualizers, \x escapes
- Languages
    + APL: reimplementation with PCT
    + Cardinal (Ruby): reimplemention with PCT
    + Ecmascript: reimplementation with PCT
    + lolcode: improved expression parsing, ifthen, IT, YARN
    + lua:
        . aligned with Lua official release 5.1.3.
        . added initial PCT-based implementation.
    + Punie (Perl 1): refactor to use standard PCT-based filenames
    + Pynie (Python): add functions
    + Rakudo (Perl 6):
        . rebranded, formerly known as 'perl6'
        . passes many more official Perl 6 Specification tests
        . added 'perl6doc' utility
        . oo including meta?classes, objects, methods, attributes, role composition
        . match variables, while/until statements, traits
        . many new methods for Str, List, Hash, Junction
- Implementation
- Deprecations
    + PCCINVOKE syntax for named arguments using []; use () instead.
    + see DEPRECATED.pod for details
- Miscellaneous
    + pbc_to_exe refactored for code reduction, portability, and maintainability
    + various bug fixes
    + #line directives added to generated JIT files, improving debugging
    + consting, attribute marking, refactoring, warnings cleanup

The next scheduled Parrot release will be on March 18, 2008.

Thanks to all our contributors for making this possible, and our
sponsors for supporting this project.

Enjoy!"
Submitted by pmichaud on 2007.12.18 18:05
Happy birthday, Perl!

On this 20th birthday of Perl, I'd like to express my appreciation to Larry Wall (and his family) for the many years of leadership, dedication, wit, wisdom, and sacrifice that he has put into Perl. Of course, there are many more people who deserve our thanks for Perl, too many to list here, but Larry has always been a fixture of the community and I'm so thankful for his efforts.

Today Parrot 0.5.1 (Hatachi) was released, and quite frankly it's another big leap in Parrot and Perl 6 development. So much has happened since my last journal post of a week ago that it's hard to know where to begin! The big news is that as of today the perl6 compiler is once again passing all of the 00-parrot and 01-sanity tests, so we can begin attacking the official test suite again. In other news:

* Jerry Gay made even more progress on the Punie (Perl 1) compiler under Parrot, using PCT and NQP he managed to get slurpy arguments working, and all of the tests passing. To me this is a signature achievement, and it shows that the compiler tools (PCT, PAST, and NQP) are working as we had intended.

* Jerry has also been instrumental in getting perl6 to pass its test suites again, particularly in adding constructs to the grammar, including "for", regexes, "END", and the like. So, we're passing the basic test suite again.

For those who might be concerned that we seem to have done a lot of work "just to get back to where we already were", I can only say that from an ease-of-development the perl6 compiler we have now is far ahead of what we had before. Unlike the previous compiler, we're finding that new features are easily added to the system, and that many changes aren't requiring me to shepherd many of the low-level details. For example...

* Flavio Glock joined the Parrot effort this week in a big way, and has been exploring NQP and perl6 and how they can fit in with the KindaPerl 6 (kp6) effort. Flavio has brought us many useful ideas and insights from kp6, and today he added BEGIN blocks to the perl6 compiler. We've also got some initial ideas about handling eval() in perl6, which will be very useful. Right now we're blocking on some issues dealing with lexical variable handling, but Jonathan Worthington has said he'll take a look at that.

* Jonathan Worthington has also been eager to play with junctions in Perl 6, so earlier this week he added an early implementation of a Junction object type. It even works for some simple things, although it doesn't yet do any sort of autothreading. On top of that, Jonathan also added postcircumfix:( ) (subroutine call), postcircumfix:{ } (hash index), and a working implementation of the '&' sigil. Of course, what I really liked best about this were his frequent comments about how easy and fun it was to work with the perl6 compiler. That makes me feel good.

Oh, and by the way, Jonathan also managed the Parrot 0.5.1 release today. Great work!

* chromatic implemented the "copy" opcode that we'll be needing to properly handle assignment in perl6 and other compilers. That was likely to become a huge obstacle to getting things to work properly, and I'm glad the ball for it is back in the PAST/perl6 court so that we can finish that off.

As for me, most of what I've been doing is answering questions on IRC and doing any refactors or cleanups that need to be done to allow others to implement features. So, I worked on things like
      * getting :slurpy to work in PAST (needed for Punie and perl6)
      * getting perl6 to talk directly to PGE again for regex matching
      * refactoring block generation in perl6 to better handle formal params
      * refactoring the PCT::HLLCompiler object and PAST/POST compilers to be a bit saner
      * describing various changes/improvements we're likely to want in Parrot

So, it's been a very productive week, and I'm hoping this next week can be equally productive. The things I plan to focus on for this week:

* some more documentation and articles about PCT, PAST, and NQP, and getting started with perl6 (including more detailed notes for new hackers)

* Scott Duff (PerlJam) has an article half-written about Perl 6 regexes that is waiting for a usable implementation, so I'd like to see perl6 get far enough along that he can publish his article based on perl6

* Jeff Horwitz is doing lots of amazing things with mod_parrot, and he says that if we can get modules working in perl6 that he'll be able to write mod_perl6 in Perl 6. So, 'module' is high on my list of things to accomplish.

* Of course, we're going to return to the effort of getting perl6 to pass as many of the official Perl 6 tests (from the pugs test suite) as we can, and refactoring tests as appropriate.

Stay tuned, take a look at the pieces that are coming together, and if you can, come join the fun!

Pm

  Status of perl6 on Parrot, 2007-12-04 2007-12-05 11:51 pmichaud

Submitted by pmichaud on 2007.12.05 11:51
This is an update on the status of Perl 6 on Parrot development
as of December 4, 2007.

Progress in November 2007
-------------------------
The last report was published in November as a "road map" [1]
for continued development of the Perl 6 on Parrot compiler (perl6).
Since then we've basically been following the steps outlined
on the road map and we are seeing significant progress on the
compiler.

The big news for this month is that we are now using a
lightweight Perl 6 translator ("NQP") to implement the
translation components of the perl6 compiler. Making this
transition means we are currently failing quite a few tests
in the test suite, but Jerry Gay and I are rapidly cleaning
those up. This is also a good place for others to join in if
you're interested -- see "How to hack and contribute" below.

So, as mentioned in the previous status report, the perl6 compiler
has four major components: the parse grammar, specialized parsing
routines, the ast transformation, and the runtime support. Now
that we're using NQP for the ast transformation, this means that
the bulk of the compiler--the parse grammar and the ast
transformation--are written using Perl 6 syntax. If you want to
take a quick peek at these without downloading Parrot, they're
currently available at:

    http://svn.perl.org/parrot/trunk/languages/perl6/src/parser/grammar.pg
    http://svn.perl.org/parrot/trunk/languages/perl6/src/parser/actions.pm

We're also looking into the possibility of rewriting much of
the runtime support as Perl 6. One of the significant things
we added this past week is the ability to embed PIR code directly
inside subroutines compiled using NQP. This has turned out to be
extremely useful for development -- we use Perl 6 syntax for most
things, but can drop in and out of PIR whenever we (temporarily)
need it.

Another major item completed yesterday is a new set of routines
for parsing and transforming Perl 6's quote syntax. Of course, the
primary reference for this is the STD.pm grammar [2], and if
you take a look you'll see that it's a fairly complex component
because of the many options, interpolation forms, and extensions
that are available. It will likely be a while before we have all
of the pieces to implement quotes exactly as STD.pm has them,
so in the meantime I've written up a couple of parse subroutines
(in PIR) that handle most of the quoting forms and options we're
likely to encounter, including interpolation. The only major
forms not available at present are the «...» and heredocs, and
the «...» form is waiting only on some Unicode fixes in PGE
and Parrot (RT#48108).

While I'm on the topic of STD.pm and parsing: I do need to
compliment Larry, Damian, and others who have been building
STD.pm and the language specification for the improvements
made in the language syntax over the past year or so, such
as in quoting. The version of Perl 6 we have now is *much*
easier to parse--by both humans and machines--than what we
had a year ago.

Where things are headed next
----------------------------

There are two major items we will be focusing on for the
next couple of weeks. The first will be continued development
of the perl6 compiler -- getting it back to passing the tests
it was previously passing, as well as adding new features and
tests. Progress on this is already being made and should
accelerate nicely.

The other area of focus will be in writing a lot of prose --
documents that will help new developers figure out where
things are and how it's all put together. For instance, we
really need a "getting started" document for perl6 that
describes how to build the compiler, how to invoke it,
and how/where to report bugs and patches. Much of this
information is currently in the ROADMAP, but would probably
be better placed in a getting started document. We also want
a developer's roadmap of some sort.

I'm also planning to publish a "developer's glossary" somewhere
to identify many of the various terms that get thrown around
in development (PCT, PGE, NQP, STD.pm, regex, PAST, etc.).

Also on the horizon we will start looking at reviewing and
possibly restructuring the Perl 6 test suite, and coming
up with a mechanism to quickly update/publish the status
of various language features in the compiler (hopefully with
references back to the language spec).

How to hack and contribute
--------------------------

If you're interested in hacking on the compiler, this is
a unique time to start getting involved. There are a lot
of relatively easy features to be tested and added, and
the discussions are about the basics of the compiler and
of relatively easy features to be tested and added, and
the discussions are about the basics of the compiler and
how it's put together.

As mentioned above, we need a good "getting started" document,
but for now here's a quick overview. First, you'll want to
download and build Parrot -- instructions for this are at
http://www.parrotcode.org/. Once you have Parrot built,
you can change to the languages/perl6/ directory and type
'make' to build the perl6 compiler. Then use the command

        $ parrot perl6.pbc hello.pl

to run a hello.pl program (written in Perl 6).

To learn more about how compilers are being structured
in Parrot in general, I recommend looking at languages/abc/ [3].
This is an implementation of the bc(1) language using the
same tools we're using to implement perl6, with the advantage
of being a lot simpler language than Perl 6. We've also tried
to put a lot of comments into the source files for abc that
explain what each component is doing.

After that, you can look at perl6's source code files and
running some tests (either from the suite or files that
you build). If you want to see perl6's intermediate compilation
results, use --target=parse, --target=past, or --target=pir
immediately after the "perl6.pbc" part.

In the next couple of weeks we'll be writing up the documents
that describe the details of the compiler implementation, but
in the meantime please ask us questions! We'll use the questions
to help author the documents. The perl6 and Parrot developers
tend to hang out on irc.perl.org/#parrot, and questions are
welcome on the perl6-compiler mailing list [4].

If you find a bug or have a suggested patch, send it to
perl6-bug@perl.org. Please use the moniker [PATCH], [BUG],
or [TODO] (including the brackets) at the start of the subject
so that RT can appropriately tag the item.

Thanks for reading!

Pm

1. http://svn.perl.org/parrot/trunk/languages/perl6/ROADMAP
2. http://svn.pugscode.org/pugs/src/perl6/STD.pm
3. http://svn.perl.org/parrot/trunk/languages/abc/
4. http://dev.perl.org/perl6/lists/

  Perl 6 on Parrot Roadmap 2007-11-26 02:36 pmichaud

Submitted by pmichaud on 2007.11.26 2:36
On Friday I wrote up a brief ROADMAP for Perl 6 on Parrot development, and rather than repeat it all as a journal post I'll just link to it. It really does describe where we've been, and where we're going in the next couple of months as far as Perl 6 development.

If you're wondering where this fits -- right now I'm aware of three Perl 6 implementations underway:

  • Pugs - Haskell-based implementation (http://www.pugscode.org/)
  • perl6 - Parrot-based implementation (http://www.parrotcode.org/)
  • KindaPerl6 - Perl 5/6-based implementation (http://www.pugscode.org/)

So, the implementation I've been focusing on is the one for Parrot, and that's where my journal entries will tend to focus.

More soon...

  Parrot 0.4.15 "Augean Stable" released! 2007-08-22 15:26 pmichaud

Submitted by pmichaud on 2007.08.22 15:26
On behalf of the Parrot team, I'm proud to announce Parrot 0.4.15 "Augean Stable." Parrot is a virtual machine aimed at running all dynamic languages.

Parrot 0.4.15 can be obtained via CPAN (soon), or follow the download instructions. For those who would like to develop on Parrot, or help develop Parrot itself, we recommend using Subversion or SVK on our source code repository to get the latest and best Parrot code.

Parrot 0.4.15 News: Implementation:
  • Lots of code review, many bugs fixed
  • Many more code cleanups and compiler warning levels
  • Started a new jit engine for 64-bit processors
  • Refactored configure process, with tests and new diagnostic options
  • Added new CodeString PMC for dynamic generation of PIR code
  • More pdd15 support for object metamodel
Languages:
  • Added NQP ("Not Quite Perl"), a very lightweight Perl 6-like language
  • Significant improvements and refactors to PCT (Parrot Compiler Toolkit)
  • perl6 passes more spec tests
  • Lua works now with a PGE/TGE/PAST-pm based compiler, lives in one pbc, and the interpreter has same behavior as original
Documentation
  • Added a committers' HOWTO
  • More PIR tutorial examples
  • Added PAUSE guide


Thanks to all our contributors for making this possible, and our sponsors for supporting this project.

Enjoy!