Yes, that's the main difficulty : the tokenizer. I can't always figure out the value of the token returned to the parser (the one declared with %token in the yacc source) or the contents of the input source associated to each non-faked-up token. (The tokenizer sometimes makes up tokens to help the parser. Or it may switch two tokens. I suspect it to cast a spell from time to time.)
OK, if this approach doesn't work, I'll try plan B: patch perl to allow pluggable parser backends.
If strain on the lexer were a design criterion, I blew it long ago. -- Larry Wall
And I should avoid sausage factories.