Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

agent (5836)

agent
  agentzh@yahoo.cn
http://agentzh.spaces.live.com/

Agent Zhang (章亦春) is a happy Yahoo! China guy who loves Perl more than anything else.

Journal of agent (5836)

Tuesday October 17, 2006
08:42 AM

Notes for this fortnight (2006-10-01 ~ 2006-10-16)

Oct 1 (to Sun Xin~)

Please check out your mailbox. i sent one journal and 107 slides to you for proofreading yesterday. remember that i've said i would try my best to keep you relatively busy? :)

Oct 4 (to Sal Zhang~)

I've rewritten your Win32::xul2ppt_mec module using Win32::OLE and stevan's excellent Moose module. Now it's named XUL::Image::PPT and the xul2ppt utility has been divided into two separate tools, xul2img.pl and img2ppt.pl. Please check out http://svn.berlios.de/svnroot/repos/unisimu/Perl/XUL-Image-PPT/ for the source code. :)

Regarding the new xul2img utility, the --count and --title options are required. use --help to see the usage. because the XUL => image part is still based on Win32::GuiTest, the user interface is somewhat fragile and cannot be as nice as that of img2ppt. it's still the user's responsibility to open .xul with firefox and not to enter the full view mode (via F11) before running the xul2img tool.

Delay settings like 0.5 sec should also work now since i've switched to Time::HiRes's sleep function. btw, Moose is so cool that writing perl 5 OO code has been exceedingly enjoyable. you know, perl 5's OO was ever a weak or even boring part in the past. Moose has brought me the feeling of using Perl 6 *today*. So don't hesitate and give it a shot! Enjoy~

(agentzh mooses.)

Oct 5 (to Sun Xin~)

Currently i am making slides for my XML talk. the topic is ``XML in the real world''. will send the slides to you for review once they're ready. :)

Oct 6 (to Cherry Chu~)

I will send you a message when i get up tomorrow morning. please keep your phone on, OK? if you get up earlier than i, would you please inform me via a message? thank you. :)

Oct 6 (to Jack Shen~)

The slides for my XML talk are ready now. please check out your mailbox for details. the slides contain a lot of pretty pictures. i've covered hot topics like RSS and AJAX using Google Reader, the Qzone site, and my GetQzone utility as study cases. these topics are extremely exciting! comments on my slides will be appreciated. :) i hope miss zheng will be kind enough to give me more time to explain everything in my slides...hehe.

Oct 7 (to Cherry Chu~)

cherry: moose. :)

cherry: elk! :D

I'm now heading out. :) 7:15 AM. don't be late, cherry.

...yay! cherry++ i'm already waiting for you. :)

...i am home now, cherry! yay! ...I was walking pretty fast. hehe. have a good rest. hopefully you will regain your strength soon. :)

Take care and sleep early, cherry. gotta run to shower and sleep myself. G'night &

Oct 8 (to Sun Xin~)

Cherry and i rode to the yangzhou city yesterday. we favored small roads in the fields over big ones. as a result, we were often followed by barking dogs and blocked by rivers and fields in our way. it was frustrating but also fun. she was amazingly vigorous and charming yesterday...we talked very happily and laughed a lot. you know, it was quite amusing to see she also talked and laughed very loudly, just like me! yay! hooray for cherry's beauty and the enormous parallels between us! hehe.

we've decided to ride to other cities in the next few times. but it's still undecided which city to go first. what's your opinion, man? ;-)

Oct 8 (to Cherry Chu~)

how are you today? i am still a bit tired. sigh.

...wow, nice to hear that. btw, i'm happy to see my friend laye has replied to your journal. he's a talented programmer and now studying in the Fudan university. :) And your ``journal of 70 kilometers'' post reads very well! :)

...nod nod. he was in ujs when he was an undergraduate student. sadly we have never met in person. :(

Oct 9 (to Sal Zhang and Jack Shen~)

Yay! now i can do Java Swing programming in pure perl 5! furthermore, my perl interpreter can now learn new Java libraries all by itself. so i can manipulate *any* Java classes and objects as if they were implemented directly in perl 5. thanks to Inline::Java and Java::Swing. now i'm trying to get them work with pugs (i.e. perl 6). unfortunately, pugs doesn't do auto-importing for perl 5 modules. sigh. maybe i need to write some perl 5 wrappers and glue code there. oh, well...

Oct 9 (to Sal Zhong~)

huh! google++

i will definitely look into its shiny source code search engine the other day. thanks for the info. :)

Oct 9 (to Sun Xin~)

man, i'll (selectively) translate these notes myself because i don't want to occupy too much of your spare time. anyway, i can do the translation work more easily and more accurately. would you please proofread both my english and chinese transcripts for me? i'll be very grateful to your review! ;)

Oct 10 (to Jack Shen~)

I've nailed down the basic syntax of the SXML language. it looks pretty neat. i'll implement converters for XML <=> SXML and HTML <=> SXML. i believe it's important enough for both XML's human reading and human writing.

Oct 10 (to Cherry Chu~)

Moose. will you come to my class this friday evening? :)

Oct 16 (to Sal Zhong~)

jerry gay (the guy also known as particle) is rewriting my smartlinks.pl using Moose . it's really wonderful! he will commit the code to the parrot repos. He said he would introduce smartlinks to the parrot test suite and link the tests to both the Perl 6 Spec and the parrot PDDs. not sure if he still has the crazy plan to port smartlinks.pl to PIR. anyway, as christopher said, the idea of smartlinking has inspired several add-on hacks. hooray!

btw, pugs 6.2.13 is going to release tomorrow. larry is using pugs for his $work! sweet...

This entry was originally posted to

http://agentzh.spaces.live.com/blog/cns!FF3A735632E41548!125.entry

Sunday October 01, 2006
01:27 AM

Notes for this fortnight (2006-09-21 ~ 2006-09-30)

Sep 21 (to Sun Xin~)

our charming XML instructor today asked me to give one or two talks in her class because she thought i was an expert in this domain. i'm very excited and have decided to make some good- looking slides in both english and chinese. i'll work on the new slides for the coming talks in the next few days. i'll send them to you for review once they are ready.

i'll also send you a bunch of _old_slides_ for the talk given in the last term tomorrow. they're in pure english and most of my classmates liked it. i hope you can proofread the old slides since i'm going to publish them on the web.

Our XML instructor is so beautiful that i dare say a lot of boys in the classroom like her very much. she holds great charm for me. i love to talk to her after the class. talking to her in person is really enjoyable. she is an extremely lovely girl. yay for her beauty and good mood!

Sep 22 (to Sun Xin~)

mails sent. remember to use Firefox to access the .xul URL (i.e. the slides) and don't click too fast while reading the slides since loading images can be slow.

btw, i'll use cherry's qzone blogs as a study case in my new slides for the XML talk. the slide-making process can be fun! stay tuned! :)

Sep 22 (to Jack Shen~)

hey, jack. let me talk about microcosmic stuff, such as interface design for individual classes or small class library and you talk about the handwavy macroscopical things like large OO systems. i'll try my best to put enough basic weapons under the audience's belt before your lectures. what's your opinion?

Sep 23 (to Sun Xin~)

my instructors have asked me to give for total 12 talks this semester. that's really wonderful since i can take a more leisure pace during my talks. but i definitely need many more slides and pictures. i'll be talking about XML, Regular expressions (regexes) and object-oriented modeling and design. what cool stuff!

Sep 27 (to Sun Xin~)

i've produced 82 slides these two days, and i am about to crash... for my first talk, there are still 20 slides to go... it's really exciting!!! man!!!

man, i've sent a weird english poetry to you for translating. i need the chinese transcript for my slides. please get back to me ASAP. it's quite urgent since the talk is scheduled on *this* Friday evening. thank you in advance. :)

Sep 27 (to Cherry Chu~)

our XML teacher has asked me to give one or two talks to my classmates in her class. and i am going to use *your* Qzone blogs as a study case in my slides. :)

will you mind my introduction to your Qzone home? btw, an even shorter URL is working now: http://perlcabal.org/agent/cherry.html. feel free to give it a shot.

...glad to hear that. i have 16 lectures to give out this semester. i've been busy making slides for my talks. it is a hard job but is also fun. :)

Sep 29 (to Sun Xin~)

the talk was a big success. the audience laughed a lot and i was often stopped by the girl students' ``wow''. my instructor said after the talk that he was exceedingly impressed. he told me that it had given him great inspiration and determination. he said he had even been pondering giving up his C.S. career, but my talk completely changed his mind.

...nah. cherry didn't come because she is in a different department and i had not invited her to my talk.

...sorry, i didn't show your transcript explicitly in my slides. Audrey offered a translation in ancient chinese right before the talk and i used hers. but your work had helped me a lot. without your translation, i can't grok that poem to such an extend. thank you!

it's worth mentioning that Larry Wall also provided me with an excellent translation in modern english. you know, he is a great linguist. :)

...LOL. lucky indeed. getting so much help and support is like a dream!

Sep 30 (to Cherry Chu~)

Heh, it will rain tomorrow anyway. hopefully the weather will get better when you come back from shanghai. :)

...nope, not that one. i was talking about the art of naming. the talk mentioning your Qzone space will be given on Oct 14, which is about XML in the real world.

Sun Xin asked me last night if cherry had attended my talk. and i explained to him that you are in a different department and sadly i had not sent you an invitation.

The talk was on this Friday evening. and i'll give talks at that time for every week from now on. we are at Z101. welcome joining us! 9th and 10th classes. :)

...nah, chinese speech mostly. every slide contains two versions of the content, the chinese version and the english version. and yeah, there'll be many students in my class. don't worry. :)

it will simply rock if you can come. for another thing, i really hope you can also attend my XML talk because i'm going to show my classmates your blogs there. he he.

the XML talk has been scheduled at 10504, 7th or 8th class. the concrete date is still undecided. i'll tell you once i had talked to our XML instructor. OK?

...(agent does his happy dance.)

the XML talk should be on Thursday afternoon, btw.

cherry, i have the idea of writing journals based on my cellphone messages. it's a great source of materials, you know. of course, i won't public any messages that i've received from others. for example, your replies will be excluded while my messages to you will probably be shown. what's your opinion? will you mind?

This journal has also been posted at http://agentzh.spaces.live.com/blog/cns!FF3A735632E41548!124.entry

Saturday April 08, 2006
01:18 AM

The Genetic Algorithm Used by Audrey

I've been reading #perl6 IRC logs for more than a year and it's a very good way to sync with the rapid Perl 6 development.

Sometimes I find something really, really interesting so I'd like to quote them here. After all, I know there're many Perl 5 programmers (like me!) who love to learn more about Perl 6 and the future of Perl 5.

2006-04-06
-----------
[04:55] <arcady> by the way, how much of "the perl 6 grammar" exists at the moment?
[04:58] <TimToady> depends on how you count, I suppose.  rule syntax is fairly well characterized by now.  a lot of it is specced pretty well, for some definition of pretty that ain't pretty.
[04:59] <arcady> so at least we can have something like the grammar grammar
[04:59] <TimToady> Most of the operator precedence is not done with rules at all.
[04:59] <TimToady> Yes, the grammar grammar is already bootstrapped approximately twice.
[04:59] <arcady> it's kinda hard to keep track of all the stuff going on...
[05:00] <TimToady> there'e very little top-down grammar over the bottom-up expression parser.
[05:00] <TimToady> more top-down involved in scanning complex tokens containing subexpressions.
[05:00] <TimToady> but the main complication remaining is just making sure all the grammatical categories work as envisioned.
[05:01] <TimToady> Then there's just little detail of attaching semantics to the parse...  :)
[05:01] <TimToady> s/little/the little/
[05:02] <arcady> that can be left as an exercise to the implementors : )
[05:02] <TimToady> But audreyt says that Perl 6 now fits in her head, so that should be finished a day or two after the grammar is done.
[05:04] <arcady> that would be most awesome
[05:05] <TimToady> well, even a month or two would be awesome.  a year or so is more likely before we have something really, really solid.  Still, I'm very happy with how it's going.
[05:05] <arcady> well, I'm happy that you're happy, and that it's going
[05:05] <arcady> I guess it's not entirely obvious from here
[05:06] <arcady> and all the various bootstrap efforts and targets and so on are confusing
[05:07] <TimToady> Hmm, yes.  Audrey
[05:07] <TimToady> Audrey's development methodology resembles a flooding algorithm at times...
[05:08] <TimToady> Or maybe a genetic algorithm.

__END__

Yeah, in the last year, I was also confused by the JavaScript, Perl5, and all other backends for Pugs. I was asking in my mind, "why can't autrijus focus on parrot, which is believed to be the only VM Perl 6 should be run on? Isn't it possible that the various backends will slow down the roaring speed of Pugs?" Now I finally understand the approach Audrey has been taking -- genetic algorithm or something like that. Given that there're always more than one way to do it, how can we figure out the "best" way if we haven't tried others yet? And furthermore, never forget that -Ofun is always the meta goal of the Pugs project. :)

parrot may not be the only choice and may not be the best choice, as evidenced by the following conversation:

2006-04-06
-----------
[05:11] <arcady> what's going on with parrot, by the way?
[05:12] <arcady> and how does any of that connect to any of this?
[05:13] <TimToady> Parrot is sort of the other end of the world from me, so I just follow along in p6i mostly.  I hear conflicting things, but I think it'll get there eventually, for some definition of "there".
[05:14] <TimToady> Whether it will be "the" Perl platform or "a" Perl platform, or somewhere in between, remains to be seen.

__END__

P.S. I must admit, Audrey's "Genetic Algorithm" is funny and helpful even in a general sense. I've successfully applied that to most of my open source projects. Multiple approaches and multiple perspectives often lead to surprisingly deep insights. That may be the most useful "algorithm" I learned from Audrey++. ;-)
Friday January 20, 2006
09:03 AM

use HTTP::Proxy to log my web accessing history

Yeah, I visit many websites everyday. what I'm wanting and what I'm always looking for is a facility to automagically keep a record of the URLs and page titles I've just accessed, so that I can analyse the history some time later to find out the focus of my interest in a particular period of time, for example. And it's very likely that I can come up with even more interesting statistical consequences.

The Mozilla browser doubtlessly gives builtin support for accessing history, but unfortunately exporting that history info is not trivial. what I want is not only the URLs, but also the corresponding page titles (if any!) and the visiting time stamp.

Several weeks ago, I happily found that the CPAN module HTTP::Proxy can come to the rescue. What I need to do is just writing several lines of Perl code using that module, running this script at the background as a local HTTP proxy server, and setting my web browser to simply use that. By doing this, my local proxy has a chance to monitor all the HTTP traffic between my browser and the Internet.

It's fun to see that my local proxy server can also use a remote proxy. so my local one then becomes a secondary proxy, no? ;D

The HTTP::Proxy module also supports logging internally, thus my code is even simpler:

use HTTP::Proxy ':log';
my $logfile = ">>$home/myproxy.log";
open my $log, $logfile or
        die "Can't open $logfile for reading: $!";
my $proxy = HTTP::Proxy->new(
        logmask => STATUS,
        logfh => $log,
);

The logmask parameter here controls what kind of things the proxy should record. the STATUS constant indicates only basic URL and response code will be logged. What I get in the log file is something like this:

[Fri Jan 13 17:25:42 2006] (1888) REQUEST: GET http://www.perl.com/
[Fri Jan 13 17:25:53 2006] (1888) RESPONSE: 200 OK
[Fri Jan 13 17:25:53 2006] (1888) REQUEST: HEAD http://www.google.com/mozilla/google.src
[Fri Jan 13 17:25:54 2006] (1888) RESPONSE: 200 OK
[Fri Jan 13 17:25:54 2006] (1888) REQUEST: GET http://www.perl.com/styles/main.css
[Fri Jan 13 17:25:55 2006] (1888) RESPONSE: 304 Not Modified ...

Hmm...very cute! However, HTTP::Proxy's builtin logging mechanism doesn't respect HTML titles. Thus I need to provide a user agent of my own:

package MyUA;
use HTTP::Proxy ':log';
use base 'LWP::UserAgent';
sub send_request {
        my ($self, $request) = @_;
        my $response;
        eval {
                $response = $self->SUPER::send_request( $request );
        };
        if ($@ and not $response) {
                return HTTP::Response->new(500, $@);
        }
        if ($response->is_success) {
                my $type = $response->header('content-type');
                if ($type and $type =~ m[text/html]i) {
                        if ($response->content =~ m[\s*(.*\S)\s*]si) {
                                $proxy->log( STATUS, 'TITLE', $1);
                        }
                }
        }
        return $response;
}

Now we have HTML titles recorded down as well, as witnessed in my log file:

[Tue Jan 17 20:33:47 2006] (2484) REQUEST: GET http://perladvent.org/2004/20th/
[Tue Jan 17 20:33:50 2006] (2484) TITLE: Perl 2004 Advent Calendar: Filesys::Virtual
[Tue Jan 17 20:33:50 2006] (2484) RESPONSE: 200 OK

Then feed the customized user agent to my HTTP::Proxy instance I created earlier:

my $agent = MyUA->new(
        env_proxy => 1,
        timeout => 100,
);
$proxy->agent( $agent );

At last, we enter an infinite loop as every http proxy server:

while (1) {
        eval { $proxy->start(); };
        warn $@ if $@;
}

That's it!

It already works for me, but there're still several pitfalls in this solution:

  • Images won't display in MS Internet Explorer (Mozilla works fine, however)
  • It seems to me that HTTP::Proxy doesn't support forking by default so it leads to poor performance if I request multiple URLs simultaneously. (BTW, Is there a way to switch to a forking engine? I can't find a word in its POD docs.)
  • SSL connection doesn't work on my box.

Have fun!

Tuesday November 29, 2005
09:23 AM

Tried out Perl6::Attributes

If you have ever written any Perl OO code, I recommend you to take a look at Luke Palmer's CPAN module Perl6::Attributes, which is extremely simple and handy.

Instead of writing something like

    sub method {
        my $self = shift;
        $self->{attr} = '...';
    }

we can now write

    sub method {
        my $self = shift;
        $.attr = '...';
    }

in Perl5.

Another example is replacing the ugly syntax

    sub method {
        my $self = shift;
        @{$self->{attrs}} = qw(a b c);
    }

with the Perl6-ish syntax

    sub method {
        my $self = shift;
        @.attrs = qw(a b c);
    }

You see, it is pretty cool!

I've already had a try in my recent Perl module developements. Perl6::Attributes makes the source of my Perl OO modules amazingly neat.

Saturday November 05, 2005
12:50 AM

MASM Port for Test::More

The upcoming CPAN module, Test::Masm::Dos, will be a MASM port for the excellent Test::More module on Perl's land. The approach is to give special meanings to MASM comments so that we can write something like

        mov ax, 0ffffh
        ; is ax, -1, "Test a negative number";

directly in the context of 16-bit MASM source code. The implementation for these smart comments is straightforward too -- expand each comment with a piece of assembly code that implements exactly what the comment says. Hence most of the amazing power the Perl QA Group owns can be brought to the poor 16-bit MASM, a legacy MS-DOS assembly language developed by Microsoft.

Here is a complete (thought still trivial) example:

        ; foo.asm
        code segment
                assume cs:code
        start:
                ; tests => 5;
                mov ax, 1011011B
                inc ax
                ; is ax, 91;
                mov cx, 5
                next:
                ; say "# cx = ", cx;
                ; ok cx

        code ends
                  end start

Then we could feed it to my masmt script:

        masmt -o ~foo.asm foo.asm

where ~foo.asm is a temporary program with all the smart comments expanded by corresponding assembly instructions. So now we can assemble this to an executable named ~foo.exe, which is a MASM tester that produces TAP report!

At this point, I've only implemented the "say" comment. However, my classmates and I have already found it very helpful, especially when debugging our assembly programs. You know, it's really a painful experience to do even simple I/O in MASM!

With smart comments in hand, we can easily realize our dreams of applying Perl's debugging strategy to MASM: tweak, run, and tweak, run!

Tuesday November 01, 2005
05:46 AM

A flowchart generator using GraphViz

I'm now hacking on a GraphViz application that generates flowcharts. I've designed a tiny scripting language to describe the detailed structure of a program's control flow. This language is very similar to MASM, since flowcharts look pretty close to assembly languages from the point view of branching. Here's a tiny example to illustrate the basic idea:

                 _______
                ( start )
                 -------
                   ||
                   \/
            /----------------/
           / input a string <--------+
          /----------------/         |
                   ||                |
                   \/                |
          +--------------------+     |
          | convert to a number|     |
          +--------------------+     |
                   ||                |
                   \/                |
         /---------------------/     |
        / output the hex form /      |
       /---------------------/       |
                   ||                |
                   ||                |
                   \/                |
                   ||                |
                  /  \               |
                 /    \              |
                /      \             |
               -  quit? +------------+
                \      /
                 \    /
                  \  /
                   ||
                   ||
                   \/
                 ______
                (  end )
                 ------

The corresponding script is as following:

        start   start
     L1:
        io      input a string
        do      convert to number
        io      output  the hex form
        test    quit?
        jno     L1
        end     end

Note that the opcode of each Asm-like instruction is a keyword, such as "start", "io", "do", "test", "jno" and "end". The label "L1" is also critical to the flowchart's structure. Other stuff, such as the long string "convert to a number", is all arbitrarily determined by the user. The compiler just displays these chars in the flowchart verbatim.

Well, it's time for me to name this scripting language formally. Eh, FlowAsm is a good choice. So all FlowAsm source files should be ended with the .fa extension.

FolwAsm is already simple, but it's some kind of immediate languages which are not supposed to be used directly by the user. Hence, the end-user will probably use a more advanced pseudo-language which has saner control flow structures like "if", "while", and "for" statements. Moreover, we may be able to translate real-world C/C++/Perl code to FlowAsm code in the foreseeable future. Converting assembly programs to FolwAsm scripts can be easier.

I started this project because I found no parallel efforts on CPAN except an ASCII version named Text::Flowchart.
Thursday October 06, 2005
08:28 AM

Writing English stuff with help from Google

In this journal I'd make one of my secrets public.

Every time I want to use an English expression that I'm not quite familiar with, I always first have a google for it. If there're a lot of hits, I'll then feel pretty confident to write it down. If not, it's still very likely to find the correct form according to the google results. Google is really an extremely rich source of English language materials. It's able to perfectly answer questions like "Do English speakers usually say this way?"

Frankly speaking, all the successful websites, such as Google, eBay, PayPal, and Yahoo, really has changed our ways of life significantly.

I think it'll be a great idea to write this "secret" into my journal. It's doubtless beneficial to share such experience with people from non-English countries all over the world.

07:30 AM

Tried SVN out

Today I did nothing but try Subversion out. Thanks to the free SVN server provided by OSSF. I've hosted two of my CPAN contributions to that server. I also appreciate TortoiseSVN, which is a very handy SVN client on Win32. Centralized version control seems quite sufficient to me. However, I still have an eager to take a look at the famous SVK. Unfortunately I don't seem to have enough time to try SVK tonight.

Geoff Broadwell said in O'Reilly's weblog that, "If you're not already using a version control system, shame on you." Happily I can announce at this moment that I'm already using such a system for my software development. Whee! :=)

Wednesday October 05, 2005
08:47 AM

Makefile::Parser and Makefile::GraphViz

I managed to release Makefile::Parser 0.06 and Makefile::GraphViz 0.03 to CPAN tonight. Hopefully I can see them appear on search.cpan tomorrow morning.

The former module is a Makefile parser which parses Makefiles to simple ASTs while the latter is a GraphViz application that generates pretty graphs that depicts the building flow directly from Makefiles. The pictures produced by GraphViz are amazingly good. I don't think I'm able to produce better pictures manually with MS Visio. Furthermore, everything in the output pictures are fully parameterized, leading to good flexibility.

Nadim Khemir told me that he was looking forward to use Makefile::Parser in his PerlBuildSystem, which urges me to improve this stuff actively.

More feedbacks are most welcomed. Thank you in advance.