All the Perl that's Practical to Extract and Report
use Perl Log In
jplindstrom (594)
jplindstrom
(email not shown publicly)
(email not shown publicly)
Journal of jplindstrom (594)
Saturday May 11, 2002
11:30 AM
Unicode transliteration
I'm doing some processing of the OpenDirectory dmoz RDF dump, but couldn't find a decent way to make sense of the Unicode chars.
After having lucked out on Google and CPAN for a week I finally found the way to transcribe utf8 text to Latin-1:
http://groups.google.com/groups?selm=note-18266%40php.net
My previously home grown version is slightly more complete when it comes to e.g. Romanian chars. It's 100% manual though... (I log missing chars and add them by looking at the dmoz.org web site
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Life is ridiculous.
--Larry Wall in Configure from the perl distribution
Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.
Unicode transliteration (Score:2)
Re:Unicode transliteration (Score:1)
The output of unidecode(...) always consists entirely of US-ASCII characters -- i.e., characters 0x0000 - 0x007F.
Which isn't what I need. I need it to map to the URLs used at dmoz.org, which include 8-bit-chars.
Haven't tried the module though.
Text::Iconv (Score:3, Informative)
Reply to This