use Perl

• looks_like_number() (Score:2)

  by rafael (2125)

  You might want to look at the perl API function looks_like_number(), accessible from XS. (see perldoc perlapi.)
  • Re: (Score:1)

    by Aristotle (5147)

    Or from Scalar::Util [perl.org].
    • Re: (Score:1)

      by ChrisDolan (2855)

      Aristotle, Rafael,
      
      Much appreciated! I was not aware of that function. In this particular case, PPI is designed to be more lenient than Perl (it must be round-trip-safe even on invalid syntax) so we'll stick to our custom tokenizing. That said, I'll probably look deeply at looks_like_number to see if I can find inconsistencies in our tokenizer.
      
    • Re: (Score:1)

      by ChrisDolan (2855)

      Hmm, I just looked at looks_like_number() in Scalar::Util and it's a different beast entirely. That's used for numification, not tokenization. looks_like_number() does not support '_', '[eE]', binary/octal/hex numbers nor version strings. The internal grok_number() in numeric.c is similarly limited.
      
      Instead, I've discovered scan_num() in toke.c. That's what I want to emulate (leniently).
      
• 5.6 (Score:1)

  by jjore (6662)

  Prior to 5.6 the placement of _ was restricted something like every third character. Now it's completely open. Isn't it? I think so. Probably.
• Underscore (Score:2)

  by bart (450)

  As a summary, am I right in interpreting that an underscore is only accepted, without warning, if there is a digit (in the current base) on either side? And no, the "0" prefix in "0x" or "0b" doesn't count. Ditto for the leading "0" for octal.
  • Re: (Score:1)

    by ChrisDolan (2855)

    Yes, that is my conclusion as well.