All the Perl that's Practical to Extract and Report
use Perl Log In
brian_d_foy (44)
Journal of brian_d_foy (44)
I need some Unicode examples for Effective Perl Programming
Josh McAdams and I are updating Effective Perl Programming, and I'm working on a bunch of items dealing with Unicode.
I need some really nice non-english and especially non-romance language examples for some of the features we want to discuss. I'd love to be able to include sample strings in Chinese, Japanese, Russian, Portugeuse, Arabic, and all sorts of other languages I have no clue about. Most of what I need are the sample phrases. If you don't have something interesting, maybe you can translate "Perl mongers" for me in an example like:
use utf8;
my $phrase = '...'; # fill in your phrase
if( $phrase =~ m/\N{Some charname}/ ) {
say 'I matched a...';
}
I also want to add a couple of examples of other encodings, especially non-Western ones. I have no idea about those encodings, but I don't need anything fancy.
I'm sure that everything is going to get messed up and translated incorrectly, so I'll be sure to let you see the proofs of your example to ensure the typesetters get it right in the end.
help them think they understand.
--Larry Wall
My advice… (Score:2)
Always use UTF-8 if you possibly can. It's (more-or-less) a superset of everything else, and it's properly detectable.
If you're looking for interesting encodings, I'd recommend checking out one of the Shift-JIS [wikipedia.org] things. Just for weirdness. Personally, I've little experience of non-western encodings.
For more concrete use cases to cover with encoding, you should look at:
Re: (Score:2)
Can you tell me more about the command line and environment variable problem? I think I'll have the other ones covered, but I'd like to know how you solved that one. I don't recall reading anything about how Perl will treat those.
Sounds like it would be very platform-specific (Score:2)
Re: (Score:1)
Personally, I'd just pipe those things through Encoding::FixLatin and enjoy the utf8ness it emits :-)
Interesting. I could have used that module a couple of years ago. Since then I've been using this trick to convert UTF-8-or-CP1252 byte strings to UTF-8 text strings:
use Encode qw(decode);use Encode::Guess;my $line = <>;my $utf8 = guess_encoding($line, 'utf8');$line = ref $utf8 ? decode('utf8', $line) : decode('cp1252', $line);http://search.cpan.org/perldoc?POE::Component [cpan.org]
Re: (Score:2)
cmdline handling (Score:1)
Command line arguments come in as raw bytes. So you have to detect the codeset of the user's environment and encode if necessary. Roughly like this:
use I18N::Langinfo qw(langinfo CODESET);
use Encode qw(decode);
my $codeset = langinfo(CODESET);
for (@ARGV) { $_ = decode $codeset, $_ }
Likewise for environment variable values.
Some Swedish Unicode (Score:1)
real world example (Score:1)
My recommendation is to avoid \N and \x escapes except for whitespace and combining characters. Literal characters that can be read immediately and copy-pasted anywhere are much more useful.
»Perl« is a proper name and is not translated (I haven't even seen it transliterated where it would be possible), »monger« is also very difficult to translate because of its multiple denotations in English (of course that word was picked deliberately for this reason). Can you substitute something easi