Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Ovid (2709)

Ovid
  (email not shown publicly)
http://publius-ovidius.livejournal.com/
AOL IM: ovidperl (Add Buddy, Send Message)

Stuff with the Perl Foundation. A couple of patches in the Perl core. A few CPAN modules. That about sums it up.

Journal of Ovid (2709)

Wednesday October 31, 2007
05:20 AM

HOPpy Goodness With HOP::Lexer

[ #34799 ]

I recently received a bug report for HOP::Lexer, the lexer described in Higher Order Perl. One concern expressed in the bug report is that the lexer returns an array reference in the form [ TOKEN, VALUE ] with undef as the EOS (End Of Stream) value. However, if one doesn't use HOP::Parser but instead uses Parse::Yapp or similar tools, they expect a list in the form ( TOKEN, VALUE ) with the EOS indicated by ( '', undef ).

The suggested fix was to modify HOP::Lexer to optionally allow this. To my mind, this was the wrong fix. Since HOP is all about functional techniques -- applying functions to functions -- a much cleaner is to not create a special case, but to use a function!

In other words, what we have is this:

my $lexer = make_lexer( $iter, @input_tokens );

while ( my $next = $lexer->() ) {
    my ( $token, $value ) = @$next;
    ...
}

But we can now do this:

my $lexer = make_lexer( $iter, @input_tokens );

my $yapp_lexer = sub {
    my $token = $lexer->();
    return ( '', undef ) unless $token;
    return @$token;
};

use MyParser;

my $parser = MyParser->new;
my $value  = $parser->YYParse( yylex => \&yapp_lexer, yyerror => \&error_sub );

That's the correct solution to this problem and, I think, an elegant one.

Of course, the other part of the bug report was line numbers from the lexer. I'll have to think about that.

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.