use Perl

• My advice… (Score:2)

  by Dom2 (2981)

  Seeing as you didn't ask for it. ;-)

  Always use UTF-8 if you possibly can. It's (more-or-less) a superset of everything else, and it's properly detectable.

  If you're looking for interesting encodings, I'd recommend checking out one of the Shift-JIS [wikipedia.org] things. Just for weirdness. Personally, I've little experience of non-western encodings.

  For more concrete use cases to cover with encoding, you should look at:

  • query parameters coming in from browsers
  • POSTed form parameters coming in from a browser.
  • What e
  • Re: (Score:2)

    by brian_d_foy (44)

    Can you tell me more about the command line and environment variable problem? I think I'll have the other ones covered, but I'd like to know how you solved that one. I don't recall reading anything about how Perl will treat those.
    • Sounds like it would be very platform-specific (Score:2)

      by grantm (164) on 2009.09.25 4:35 (#70669) Homepage Journal

      Personally, I'd just pipe those things through Encoding::FixLatin and enjoy the utf8ness it emits :-)

      Reply to This

      Parent

      • Re: (Score:1)

        by Hinrik (8492)

        Personally, I'd just pipe those things through Encoding::FixLatin and enjoy the utf8ness it emits :-)

        Interesting. I could have used that module a couple of years ago. Since then I've been using this trick to convert UTF-8-or-CP1252 byte strings to UTF-8 text strings:

            use Encode qw(decode);
    use Encode::Guess;

    my $line = <>;
    my $utf8 = guess_encoding($line, 'utf8');
    $line = ref $utf8 ? decode('utf8', $line) : decode('cp1252', $line);

        http://search.cpan.org/perldoc?POE::Component [cpan.org]