use Perl

Last week I reported that I had a working parser for Java source code implemented as a Perl 6 grammar. Well, a dozen or so bugs later, it actually works now. Out of 7,000+ .java files in the OpenJDK, I fail to parse 3 of them (plus a handful that that are too large to fit in a gig of RAM -- I need to work on decreasing memory consumption). One of the three is a surely a bug, and the other two are unusual syntax that I've never seen before in java.lang.Double and java.lang.Float:

    public static final double MAX_VALUE = 0x1.fffffffffffffP+1023;
    public static final double MIN_NORMAL = 0x1.0p-1022;
    public static final double MIN_VALUE = 0x0.0000000000001P-1022;
    ...
    public static final float MAX_VALUE = 0x1.fffffeP+127f;
    public static final float MIN_NORMAL = 0x1.0p-126f;
    public static final float MIN_VALUE = 0x0.000002P-126f;

Hex notation for rational numbers -- what an interesting idea. I've never encountered that before. It appears that this notation first appeared in the 3rd edition of the Java language specification.

This seems to me like a notation worth adding to Perl 6, given its inherent precision advantages over decimal notation.

My favorite part of Perl 6 is the new grammar syntax. Over the last couple of days, I translated a Java source code grammar from antlr to PGE. After about 4-5 hours of work, I now have a Perl 6 grammar that can parse all of the .java files in the OpenJDK (the Java 7 source code). Well, that may be a lie. It's still crunching at about 5-10 seconds per file so it will be a while before I know if its really true.

Admittedly most of the credit goes to the authors of the antlr grammar I adapted, but this also says good things about the Perl 6 regex implementation in Parrot.

The things that bit me hardest were:

1. negated classes (PGE doesn't understand "<-[abc]>" so I had to make the inner part a separate token)
2. antlr allows character classes with outside of any character group syntax (antlr: "'0'..'9'", perl: "<[0..9]>")
3. longest token on integer vs. float (I had to change the antlr grammar to put float ahead of integer)
4. whitespace (I cribbed from the Pipp implementation)

For the last 20+ years, my favorite sci-fi/fantasy author has been Lawrence Watt-Evans. His books are not always profound, but they just fit my brain. He's currently writing a sequel to his 1989 sci-fi mystery Nightside City.

The catch is that he's writing it one chapter at a time and holding future chapters hostage until he gets enough donations. His goal of $300 per chapter is both reasonable and steep, depending on your point of view. He justifies it thoroughly in his FAQ.

If you've ever read one of his books, or just enjoy sci-fi, or have ever used one of my modules and feel you owe me ;-) then please consider donating.

I've been curious to peek at Padre, Gabor Szabo's new editor, but it's Test::NeedsDisplay prereq always failed for me with an obscure recommendation to install xvfb-run. It turns out all you really need is to run "wxPerl Makefile.PL" instead of "perl Makefile.PL". I added a note to that effect in the Makefile.PL (thanks to AdamK's open repository).

So, any Mac user who wants to try Padre can use this trick (I didn't actually test this exact technique since I got it installed via a much more convoluted trial-and-error process).

cpan install Wx
wxPerl -MCPAN -e install Padre

Now I'm off to report some bugs...

I'm addicted to working on Rakudo. I like Perl6 a lot more than Perl5, so I've been digging into some interesting corners. In the last 4 weeks, I've submitted 13 patches:

• [perl #60356] Rakudo doesn't allow inheriting from classes with :: in the name
• [perl #60358] Rakudo doesn't recognize grammars with :: in the name
• [perl #60384] Remove references to __get_string() and related methods in PGE POD
• [perl #60716] invoke multi-level namespace grammars from rules
• [perl #60218] P6object.new_class 'hll' option is ignored
• [perl #60186] make PGE support {PIR} closures instead of just {{PIR}}
• [perl #60160] recursive "use" causes infinite loop
• [perl #60366] 'does' fails with roles that have '::' in their names
• [perl #57980] Fix bugs with nested ?? !!
• [perl #60164] make Test.pm methods return a boolean, like Test::More
• [perl #60718] better error message for .new on undefined class
• [perl #60446] first draft implementation self.WHO to return package of a class

This represents ~50 hours of learning, reading, coding and patching. That's not much compared to the big contributors, but I'm proud of the work I've done.

I've been learning the Rakudo innards like crazy this past week. I keep staying up past midnight every day -- "just one more bug..."

I suffered some despair over the weekend when I realized that two fundamental features didn't work 1) lexpad variables under recursion and 2) "::" as a namespace separator. For the latter, I had worked around it by using "PDF__Grammar" instead of "PDF::Grammar" (that is, a one-level namespace instead of N-level). But now it's fixed thanks to a big patch from Jonathan Worthington and a little one from me. Yay!

The lexpad bug is a bigger one and it's killing my progress because re-entering a method overwrites the lexical variables of the previous call. I know Patrick Michaud is actively working on it though, so I've satisfied myself by working on the other bugs in the meantime.

It's a little hard to dig through the code because there's so many layers (Perl6->PGE->NQP->PAST->POST->PIR), but I'm generally quite impressed with the Rakudo and Parrot/PCT implementation. PIR is surprisingly readable, despite being an assembler-like language. NQP is deceptively close to Perl, which keeps throwing me off. And PGE just rocks.

The most surprising change I've discovered so far in Perl6 is that the reference operator ('\') is no more. That is, the following code, while valid in both Perl5 and Perl6, behaves differently in each:

   my %hash;
   my $hashref = \%hash;

In Perl6 the \ operator creates Capture instances instead of references. To make Perl6 behave the way the above code does under Perl5 behavior (where a scalar "contains" the hash), do like this:

  my %hash;
  my $hashref = $(%hash); # itemization

Then to use that $hashref, do like this:

  my @keys = $hashref.keys;
  # Perl5 equivalent:
  # my @keys = keys %{$hashref};

So, the $hashref feels less like a pointer and more like a container. The scalar delegates to its value, which is why the ".keys" works in this example without needing to explicitly dereference the scalar.

The funny thing is that Captures of Lists behave a lot like arrayrefs, so I didn't even notice I had misused the \ in the following code:

  my @array;
  my $arrayref = \@array;
  for @($arrayref) -> $val {
     say $val;
  }

The as-list operator, "@(...)", works about the same on Capture instances as it does on scalars that contain Lists -- in both cases, it returns a list of the enclosed values.

w00t! Joe Hudson has submitted a modification to the CAM::PDF encryption module to support 128-bit encryption instead of just basic 40-bit. Along the way, he fixed a LONG standing bug (about 5 years, I think) where I didn't distinguish between owner and user passwords.

I'm hoping to integrate his solution in the next week and release CAM::PDF v1.60.

I love open source!

Yesterday I mentioned that I was experimenting with Perl6. Ovid followed up by suggesting that I open the source. OK: http://svn.chrisdolan.net/perl6/PDF

It's an experiment to see if Perl6 grammars make PDF parsing easier than the Perl5 continued regexes (i.e. //cg) that I used in CAM::PDF. So far the answer is probably yes, but a few Rakudo limitations are slowing me down. Luckily I'm an optimist. :-)

I have gobs of disk and bandwidth, so if others want to use some of my SVN space, I'm willing to share. I'm thinking about moving my Perl5 work from my private repository to that public repository too, but I have to figure out the SVN slicing commands... again.

I wrote my first Perl6 program last night, using the Rakudo implementation on the Parrot virtual machine. Compilation is slow, but runtime is fast. I made a lot of mistakes, and the compiler doesn't print very useful syntax error messages yet, but once I figured out a few gotchas (like you can't have a grammar rule named "null", that must be reserved somewhere) it worked and all of my tests passed.

Take that, false vaporware accusers!