agent's Journal http://use.perl.org/~agent/journal/ agent's use Perl Journal en-us use Perl; is Copyright 1998-2006, Chris Nandor. Stories, comments, journals, and other submissions posted on use Perl; are Copyright their respective owners. 2012-01-25T02:30:10+00:00 pudge pudge@perl.org Technology hourly 1 1970-01-01T00:00+00:00 agent's Journal http://use.perl.org/images/topics/useperl.gif http://use.perl.org/~agent/journal/ SSH::Batch: Treating clusters as maths sets and intervals http://use.perl.org/~agent/journal/38851?from=rss System administration is also part of my $work. Playing with a (big) bunch of &nbsp;machines without a handy tool is painful. So I refactored some of our old scripts and released SSH::Batch, a collection of useful parallel ssh scripts, to CPAN:<br> <br>&nbsp;&nbsp;&nbsp;&nbsp;<a href="http://search.cpan.org/dist/SSH-Batch/">http://search.cpan.org/dist/SSH-Batch/</a> <br> <br>SSH::Batch allows you to name your clusters using variables and interval/set syntax in your ~/.fornodesrc config file. For instance:<br> <br>&nbsp;&nbsp;&nbsp;&nbsp;$ cat ~/.fornodesrc<br>&nbsp;&nbsp;&nbsp;&nbsp;A=foo[01-03].com bar.org<br>&nbsp;&nbsp;&nbsp;&nbsp;B=bar.org baz[a-b,d,e-g].cn foo02.com<br>&nbsp;&nbsp;&nbsp;&nbsp;C={A} * {B}<br>&nbsp;&nbsp;&nbsp;&nbsp;D={A} - {B}<br> <br>where cluster C is the intersection set of cluster A and B while D is those machines in A but not in B.<br> <br>And then you can query machine host list by using SSH::Batch's fornodes script:<br> <br>&nbsp;&nbsp;&nbsp;$ fornodes '{C}'<br> &nbsp;&nbsp;&nbsp;bar.org<br>&nbsp;&nbsp;&nbsp;foo02.com<br> <br>&nbsp;&nbsp;&nbsp;$ fornodes '{D}'<br>&nbsp;&nbsp;&nbsp;foo01.com<br>&nbsp;&nbsp;&nbsp;foo03.com<br> <br> Furthermore, to run a command on a cluster by the concurrency of 6:<br> <br>&nbsp;&nbsp;&nbsp;atnodes 'ls -lh' '{A} + {B}' my.more.com -c 6<br> <br>Or upload a local file to&nbsp;the&nbsp;remote cluster:<br> <br>&nbsp;&nbsp;tonodes ~/my.tar.gz '{A} / {B}'<nobr> <wbr></nobr>:/tmp/<br> <br>There's also a key2nodes script&nbsp;to push SSH public keys&nbsp;to&nbsp;remote machines<nobr> <wbr></nobr>;) <br> <br>A colleague in Alibaba B2B is already using it. And one of my&nbsp;teammates is going to use it to operate on those thousands of&nbsp;machines in our instance of the YST (Yahoo! Search Technology) cluster and I'm&nbsp;ready&nbsp;to&nbsp;receive more feedback from him<nobr> <wbr></nobr>;)<br> <br>Have fun<nobr> <wbr></nobr>:)<p> <i> P.S. This entry was originally posted to my own blog site as <a href="http://blog.agentzh.org/#post-105">http://blog.agentzh.org/#post-105</a> </i> </p> agent 2009-04-23T02:56:36+00:00 releases My VDOM.pm &amp; WebKit Cluster Talk at the April Meeting of Bei http://use.perl.org/~agent/journal/38787?from=rss Last night I gave a talk to our PerlChina folks at <a href="http://wiki.perlchina.org/BJPM-200904">the April meeting</a> in the Flow Bar. Here's the slides that I used:<ul> <li>Firefox XUL format: <a href="http://agentzh.org/misc/slides/BJPW200904/vdomwebkit.xul">Browse in Firefox</a>, <a href="http://agentzh.org/misc/slides/BJPW200904.tar.gz">Download tarball</a> </li><li> <a href="http://agentzh.org/misc/slides/vdomwebkit.pdf">PDF format</a> </li><li> <a href="http://agentzh.org/misc/slides/vdomwebkit.ppt">PPT format</a> </li></ul><p>The XUL format is the best among the three<nobr> <wbr></nobr>;)<br> <br>Just as the topic of the talk suggests, we're migrating from Firefox clusters to WebKit ones. I'll post more details here in the near future.<br> <br>Enjoy! </p><p> <i>P.S. This entry was originally posted to my own blog site as <a href="http://blog.agentzh.org/#post-104">http://blog.agentzh.org/#post-104</a> </i> </p> agent 2009-04-10T04:01:48+00:00 journal The slides for my talk on Firefox cluster &amp; vision-based web http://use.perl.org/~agent/journal/38470?from=rss I gave a talk at the <a href="http://use.perl.org/~Qiang/journal/38413">Beijing Perl Mongers' Feb Meeting</a> last night. It was about my Firefox cluster and vision-based web page extraction technology. I had not expected to see so many people there. Wow. The talk was well received and people asked lots of interesting questions<nobr> <wbr></nobr>:)<br> <br>The slides can be freely downloaded from my site (open the ffcluster.xul file in the tarball via Firefox):<br> <br>&nbsp;&nbsp;&nbsp; <a href="http://agentzh.org/misc/slides/BJPW200902.tar.gz">http://agentzh.org/misc/slides/BJPW200902.tar.gz</a> <br> <br>or browse directly online by Firefox:<br> <br>&nbsp;&nbsp;&nbsp; <a href="http://agentzh.org/misc/slides/BJPW200902/ffcluster.xul">http://agentzh.org/misc/slides/BJPW200902/ffcluster.xul</a> <br> <br>Because it has many big pictures in it, it's recommended to download it to your local side first and display offline<nobr> <wbr></nobr>:)<br> <br>I'll also give this presentation again to those Ruby/Python/Java/C++ guys at Beijing OpenParty's Fox meeting:<br> <br>&nbsp;&nbsp;&nbsp; <a href="http://www.beijing-open-party.org/index.php/2009/02/beijing-open-party-2009-02-fox-event-begin.html">http://www.beijing-open-party.org/index.php/2009/02/beijing-open-party-2009-02-<nobr>f<wbr></nobr> ox-event-begin.html</a> <br> <br>As a side note: recently I'm intrigued by Apache C hacking. My mod_libmemcached_cache is my first Apache module. And I'd love to see more in the near future, such as mod_openresty<nobr> <wbr></nobr>;)<br> <br>Have fun!<p>P.S. This entry was originally posted to my personal blog site: <a href="http://blog.agentzh.org/#post-102">http://blog.agentzh.org/#post-102</a> </p> agent 2009-02-13T05:13:22+00:00 journal Q4 is crazy! http://use.perl.org/~agent/journal/37972?from=rss <p>Yeah, Q4 is really crazy! I've been hacking on several company projects in parallel over the last few weeks. Fortunately they're all very interesting stuffs.<br> <br>We've just kicked <a href="http://search.cpan.org/dist/OpenResty">OpenResty</a> 0.5.2 out of the door and I'm preparing for the 0.5.3 release right now. My teammate xunxin++ has quickly implemented the YLogin handler for OpenResty, via which the users can use Yahoo! ID to login their own applications on OpenResty. Our Yahoo! registeration team helpfully worked out a sane design to allow us to reuse the Yahoo! Login system, which effectively turned Yahoo! ID into something like a passport, at least from the perspective of OpenResty users<nobr> <wbr></nobr>:) Big moment! Lots of company products using Yahoo! IDs could be rewritten in 100% JavaScript! Actually our team is already rewriting the Search DIY product using all the goodies offered by OpenResty.<br> <br>Meanwhile, some guys from Sina.com are doing&nbsp;their personal projects in OpenResty. They said they really appreciated the great opportunities provided by the OpenResty architecture since various kinds of clients (e.g. web sites, cellphones, desktop apps, and etc.) could share the same set of API via OpenResty's web services). They also&nbsp;sent a handful of useful feedbacks and suggestions regarding OpenResty's design and implementation.<br> <br>I've also been working on an intelligent crawler cluster based on Firefox, Apache mod_proxy/mod_cache, and OpenResty. The crawler itself is a plain Firefox extension named List Hunter:<br> <br>&nbsp;&nbsp;&nbsp; <a href="http://agentzh.org/misc/listhunter.xpi">http://agentzh.org/misc/listhunter.xpi</a> <br> <br>It's an enhanced version of the Haiway List Recognization Engine used by my <a href="https://addons.mozilla.org/en-US/firefox/addon/5712">SearchAll extension</a> and also built by my<a href="http://search.cpan.org/perldoc?XUL::App"> XUL::App framework</a>. You can install it to your Firefox and play with it if you like<nobr> <wbr></nobr>;) What this extension does is very simple: recognizing "list regions" and "text regions" in an arbitrary web page and further deciding automatically whether it's a "list page" or a "text page". The latter functionality may sound a bit weird: why is it useful to categorize web pages that way? Anyway, our PM (Product Manager) has crazy ideas about that categorization in our Live Search project and knows better than us<nobr> <wbr></nobr>;)<br> <br>Turning such a Firefox extension into tens or even hundreds of Firefox crawlers running on a bunch of production machines requires a lot of work. I devised a prefetching system which prefetches HTML pages and CSS files included in them, and caches the&nbsp;headers and contents&nbsp;for a fixed amount of time in such a way that Firefox crawlers can later load pages and CSS stuffs directly from the same cache in our local network, thus significantly reducing the page loading time in Gecko. The cache is a <a href="http://agentzh.org/misc/httpd-2.2.10.patch.txt">heavily patched</a> version of Apache2's <a href="http://httpd.apache.org/docs/2.2/mod/mod_cache.html">mod_cache</a> with mod_disk_cache as the backend storage. The way prefetchers and crawlers interact with the&nbsp;Internet and the cache is via HTTP proxies based on Apache2's <a href="http://httpd.apache.org/docs/2.2/mod/mod_proxy.html">mod_proxy</a>. Pipeling the prefetching and crawling processes requires OpenResty&nbsp;with <a href="http://www.pgcon.org/2008/schedule/events/79.en.html">PgQ</a> enabled. Well, I'm still working on this cluster and my goal is 2 pages/sec for&nbsp;every single Firefox process. <a href="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/">Firefox 3.1</a>'s amazing performance boost (more than 30% faster according to my own benchmark) makes me very confident in abusing Gecko to build efficient crawlers that takes advantage of the rich rendering information.<br> <br>Another Firefox crawler project haunting my head is a similar one that automatically recognizes and extracts user comments from arbatrary web pages (if any comments appear, of course). Such tasks would be hard if my code has to run without the geometric informations&nbsp;of every DOM nodes provided by the browser rendering engine (in the form of offsetWidth, offsetHeight, offsetTop, and offsetLeft attributes of DOM elements). Some other collegues in our Alibaba's Search Tech Center are putting their head around Cobra, a pure Java HTML renderer. But I'm doubting that it would run more correctly or more efficiently than Gecko.&nbsp;Oh well, I'm not a Java guy anyway...<br> <br>Finally, just a short note: I had a wonderful time with clkao and Jesse Vincent at Beijing Perl Workshop 2008. I learned pretty a lot about the Prophet internals during the hackation after the conference, and Jesse quickly hacked out a stub OpenResty model API for Prophet. Then we went to the Great Wall the&nbsp;next day. I was amazed to find Jesse hacking crazily on the Great Wall and enjoying the sunshines alone...Wow.<br> <br>Enough blogging...back to hacking<nobr> <wbr></nobr>;)</p><p> <i>P.S. This journal was originally posted to my own blog site as <a href="http://blog.agentzh.org/#post-97">http://blog.agentzh.org/#post-97</a> </i> </p> agent 2008-11-29T13:48:39+00:00 journal Now we have Actions! http://use.perl.org/~agent/journal/37559?from=rss On behalf of the OpenResty team, I'm happy to announce that <a href="http://search.cpan.org/dist/OpenResty-0.5.0">OpenResty 0.5.0</a> has been released to CPAN, which means OpenResty has hit its 5th milestone indicated by a working Action API.<br> <br>I've found Acitons very useful in grouping together concurrent AJAX requests, which will make webpages load much faster. Our blog sites are already taking full advantage of this trick:<br> <br>&nbsp;&nbsp;&nbsp; <a href="http://blog.agentzh.org/">http://blog.agentzh.org</a> <br> &nbsp;&nbsp;&nbsp; <a href="http://www.eeeeworks.org/">http://www.eeeeworks.org</a> <br> <br>Also, Actions ensure cascaded requests run in exactly the expected order and the REST interfaces are called (mostly) in the expected way (e.g. from the end users' web browser). There used to be a serious security hole in the above blog sites in past because I had to expose PUT<nobr> <wbr></nobr>/=/model/Post/~/~ to the Public role for updating the "comments" field in the Post model before we have Actions.<br> <br>The main server for OpenResty, <a href="http://api.openresty.org/">api.openresty.org</a>, has already been upgraded to 0.5.0. If you want to play with OpenResty directly on our servers, feel free to write to me (agentzh at yahoo dot cn) and get an account for free!<br> <br>Enjoy! agent 2008-09-28T10:36:25+00:00 releases pod2html.js: Some JavaScript love for POD in a browser http://use.perl.org/~agent/journal/37558?from=rss It's fun to do POD (<a href="http://en.wikipedia.org/wiki/Plain_Old_Documentation">Plain Old Documentation</a>) in a web browser and I've hacked up a JavaScript implementation for the pod2html utility (actually the output is more like <a href="http://search.cpan.org/perldoc?Pod::Simple::HTML">Pod::Simple::HTML</a>).<br> <br>The pod2html.js script is in OpenResty's SVN repository:<br> <br> &nbsp; &nbsp;<a href="http://svn.openfoundry.org/openapi/trunk/demo/Onccf/js/pod2html.js">http://svn.openfoundry.org/openapi/trunk/demo/Onccf/js/pod2html.js</a> <br> <br>The API is straightforward, for instance,<br> <br> &nbsp; &nbsp;var pod = "=head1 Blah\n\nI&lt;Hello&gt;, C&lt;world&gt;!\n";<br> &nbsp; &nbsp;var html = pod2html(pod);<br> <br>The following web site is already making use of it:<br> <br> &nbsp; &nbsp;<a href="http://agentzh.org/misc/onccf/out/">http://agentzh.org/misc/onccf/out/</a> <br> <br>By sniffing the background AJAX requests (e.g. using Firebug), you can see raw POD is retrieved from the OpenResty server and converted to HTML on-the-fly in your browser.<br> <br>It's worth mentioning that I had a lot of fun combining <a href="http://search.cpan.org/perldoc?Test::Base">Test::Base</a> and <a href="http://search.cpan.org/perldoc?JavaScript::SpiderMonkey">JavaScript::SpiderMonkey</a> to test this piece of JavaScript code in pure Perl. You can checkout the test script here:<br> <br> &nbsp; &nbsp;<a href="http://svn.openfoundry.org/openapi/trunk/demo/Onccf/t/01-pod2html.t">http://svn.openfoundry.org/openapi/trunk/demo/Onccf/t/01-pod2html.t</a> <br> <br>By looking at the (declarative) test cases, it's trivial to see what it can do (and hopefully what it can't)<nobr> <wbr></nobr>:)<br> <br>For the record, as of this writing, the following POD directives are supported:<br> <br> &nbsp; =headN, =over, =item *, =item NUM., =item TEXT, =back, =begin html, =end html, =begin ANY, =end ANY, =cut (it's a no-op), =encoding ANY (it's a no-op)<br> <br>and the following POD markups are implemented:<br> <br> &nbsp; &nbsp;C&lt;...&gt;, I&lt;...&gt;, B&lt;...&gt;, L&lt;...&gt;, F&lt;...&gt;<br> <br>I've also implemented the (non-standard) =image directive for convenience. For example,<br> <br> &nbsp; &nbsp;=image gate.jpg<br> <br>will be converted to<br> <br> &nbsp; &nbsp;&lt;p&gt;&lt;img src="gate.jpg"/&gt;&lt;/p&gt;<br> <br>Have fun!<p> <i>P.S. This journal was originally posted to my personal blog site: <a href="http://blog.agentzh.org/#post-93">http://blog.agentzh.org/#post-93</a> </i> </p> agent 2008-09-28T03:23:08+00:00 journal Filter::QuasiQuote 0.01 is now on CPAN http://use.perl.org/~agent/journal/37097?from=rss After reading Audrey's <a href="http://pugs.blogs.com/pugs/2008/07/pugshs-is-back.html">blog post</a> mentioning GHC's upcoming quasiquoting feature (as well as <a href="http://www.eecs.harvard.edu/~mainland/ghc-quasiquoting/mainland07quasiquoting.pdf">that quasiquoting paper</a>), I quickly hacked up a (simple) quasiquoting mechanism for Perl, hence the Filter::QuasiQuote module already on CPAN:<br> <br> <a href="http://search.cpan.org/perldoc?Filter::QuasiQuote">http://search.cpan.org/perldoc?Filter::QuasiQuote</a> <br> <br>I'm looking forward to using sensible filters in my production code (e.g. <a href="http://search.cpan.org/dist/OpenResty/">OpenResty</a>) and eliminating ugly Perl code for with embedded DSL. For example, instead of writing<br> <br>&nbsp;&nbsp;&nbsp; my $sql = "alter table " . quote_identifer($table) . " drop column " . quote($column) . ";";<br> <br>I can simply write<br> <br>&nbsp;&nbsp;&nbsp; use OpenResty::QuasiQuote::SQL;<br>&nbsp;&nbsp;&nbsp; my $sql = [:sql| alter table $table drop column $column; |];<br> <br>Also, a JSON-like DSL can be used to describe valid Perl data structures and to generate the Perl code doing validation.<br> <br>Filter::QuasiQuote supports subclassing, so the OpenResty::QuasiQuote::SQL module mentioned above could be derived from it. Also, multiple concrete filter classes could be composed in a single Perl source file. Just put a series of use statements together:<br> <br>&nbsp;&nbsp;&nbsp; use MyQuote1;<br>&nbsp;&nbsp;&nbsp; use MyQuote2;<br> <br>and it should work. Because it's required that filters always return Perl source aligned in a single line, line numbers won't get corrupted.<br> <br>Of course, lots of nice consequences of the Haskell quasiquotations will be lost in my implementation, such as type safety. But the Perl version is much more flexible and powerful (by some definition)<nobr> <wbr></nobr>;)<br> <br>It's still in alpha and could be buggy. Feel free to report bugs or send wishlist to the CPAN RT site or directly to me<nobr> <wbr></nobr>;)<br> <br>Enjoy! agent 2008-08-05T10:27:07+00:00 journal UML::Class::Simple 0.10 released http://use.perl.org/~agent/journal/36737?from=rss <p>I've just uploaded UML::Class::Simple 0.10 to CPAN with the highlight of the XMI format support. It will appear on the CPAN mirror near you in the next few hours.</p><p>Thanks Maxim Zenin for contributing this feature<nobr> <wbr></nobr>:) A Japanese user was requesting this in his blog as well. If you're a XMI fanboy, feel free to try it out.</p> agent 2008-06-20T02:26:47+00:00 journal The SearchAll Firefox Plugin and XUL::App framework http://use.perl.org/~agent/journal/34490?from=rss My first $job project is now opensourced. It's a Firefox extension named SearchAll.<br> <br>SearchAll is a simple side-by-side search engine comparing tool which allows you to search at most 3 different search engines simultaneously and benchmark their performance in the status bar. <br> <br>With this extension, you can compare 2 search engines or 3 search engines at a time. There's a long list of default search engines that you can choose from (including <a href="http://search.cpan.org/">search.cpan.org</a>!). And you can also enter search engines' URLs which don't appear in the default list yourself.<br> <br>Currently only the sites' raw HTML pages are shown to the user. We'll add more comprehensive and more intuitive views and graphics for the search results in the near future. Please stay tuned!<br> <br>This project was initiated and has been regulated by the Yahoo! China ( <a href="http://cn.yahoo.com/">http://cn.yahoo.com</a> ) company and opensourced under the MIT license.<br> <br>One of our buzzword (for extension developers) is that there's 0 line of XUL/RDF/XML in our project's source tree. The GUI stuff is totally scripted in Perl. Thanks to Jesse Vincent's Template::Declare module on CPAN.<br> <br>You can always get the latest source code of this project from the following SVN repository:<br> <br>&nbsp;&nbsp; <a href="http://svn.openfoundry.org/searchall/"> http://svn.openfoundry.org/searchall/</a> <br> <br>If you like to help, please let us know. We're very willing to deliver the commit bit like the Pugs team<nobr> <wbr></nobr>;)<br> <br>The XPI file that can be installed directly into Firefox can also be found here: <br> <br>&nbsp;&nbsp; <a href="http://svn.openfoundry.org/searchall/trunk/searchall.xpi">http://svn.openfoundry.org/searchall/trunk/searchall.xpi</a> <br> <br>There's a XUL application framework named XUL::App sitting in the same repos and SearchAll is already using it. I'd expect to move XUL::App to a separate repos and rename it to a cooler name (maybe Xifty or Xufty?). <br> <br>Sorry for the lack of documentation. Please see README for some general ideas<nobr> <wbr></nobr>:)<br> <br>I've already submitted this extension to <a href="http://addons.mozilla.org/">addons.mozilla.org</a> and waiting for the editor's approval. <br> <br>Enjoy! agent 2007-09-20T09:47:25+00:00 journal Notes for this fortnight (2006-10-18 ~ 2006-10-30) http://use.perl.org/~agent/journal/31449?from=rss <div><p>Oct 18 (to Jack Shen~) </p><p>I wrote a UML class diagram generator based on GraphViz. it can parse arbitrary perl OO modules and obtain the inheritance relationships and method/attribute list automatically. it's called <a href="http://search.cpan.org/perldoc?UML::Class::Simple">UML::Class::Simple</a>. And it's much easier to use than <a href="http://www.staruml.com/">StarUML</a> . you know, dragging mouse to draw diagrams is really painful. yay for automatic image generation!</p><p>(Here is one of the sample outputs: <a href="http://svn.berlios.de/svnroot/repos/unisimu/fast.png">http://svn.berlios.de/svnroot/repos/unisimu/fast.png</a>.)</p><div><p>Oct 18 (to Sal Zhong~) </p><p>i'm planning to upload <a href="http://search.cpan.org/perldoc?UML::Class::Simple">UML::Class::Simple</a> to cpan once it's mature enough. will you test it for me? bug reports and patches are most welcome.<nobr> <wbr></nobr>:)</p><p>it's still undecided how to differentiate perl classes' properties from other ordinary methods. i'm also pondering the idea of adding relationships other than inheritance. i'll be delighted if you have some ideas on these matters.</p><p>Note that i'm ignoring the <a href="http://search.cpan.org/perldoc?Autodia">Autodia</a> module on CPAN since i'm not in favor of XML and a quite different approach has been taken in my project. anyway, i have to admit it's wise to talk to <a href="http://search.cpan.org/perldoc?Autodia">Autodia</a> 's author and merge these efforts. at last, i must thank Alias for creating <a href="http://search.cpan.org/perldoc?PPI">PPI</a> and suggesting the use of <a href="http://search.cpan.org/perldoc?Class::Inspector">Class::Inspector</a>. they're invaluable when one wants to extract meta info from the perl world.</p><div><p>Oct 19 (to Jack Shen~) </p><p>I've merely finished the <a href="http://perlcabal.org/agent/slides/naming/naming_recap.pdf">slides</a> for recap. they already reach the amount of 44 and the number is still counting. alas, still wondering what to say in the next talk on the design of methods and subroutines.<nobr> <wbr></nobr>:(</p><div><p>Oct 19 (to Cherry Chu~) </p><p>Thanks. the talk went pretty well. it's interesting to see that i had the feeling just before the talk that you would not come. so i was not very surprised by your absence. no problem, there's always ``the next time''.<nobr> <wbr></nobr>:)</p><p>i've been busy making slides for tomorrow's talk. they're still not finished yet. sigh. have to make more slides during the daytime tomorrow. producing so many slides is quickly getting tedious. hehe, you know that feeling, right?<nobr> <wbr></nobr>;-)</p><div><p>Oct 22 (to He Shan~) </p><p> &gt; hi! I've found a book. IT is so nice that i have been <br> &gt; reading about it all the afternoon. it is great, just <br> &gt; like an extended version of &quot;The Practice of <br> &gt; Programming&quot;. it's named &quot;Code Complete&quot;. </p><p>I've got the feeling that you are currently on the *right* way. you'll definitely become a good hacker if you keep going. hmm, hopefully you'll join us perl camels soon.<nobr> <wbr></nobr>;)</p><div><p>Oct 22 (to Jack Shen~) </p><p>...LOL. apparently you are not a VB guy. inserting images into ppt slides is straightforward once you know how to record down VBA macros in the PowerPoint environment and browsing the generated code in its VB IDE. Another way to get an answer is searching the web. iirc, the method should be AddPicture or something like that. not sure though, computers are out of my reach right now.<nobr> <wbr></nobr>:(</p><p>...Python is even more powerful than MATLAB, Maple, and Haskell? i doubt that.<nobr> <wbr></nobr>:)</p><p>...I was exclusively hacking on the new tokenizer for <a href="http://search.cpan.org/perldoc?Makefile::Parser">Makefile::Parser</a> and completely forgot that i had C# classes tonight. anyway, the next major release of M::P takes precedence over any other things.<nobr> <wbr></nobr>:)</p><div><p>Oct 23 (to Sal Zhong~) </p><p>I've just started to rewrite M::P's codebase (which will hopefully be released as M::P 1.00 soon). Yes, it's long overdue. I've had a pretty good plan for a scalable and extensible gmake implementation based on M::P for long.</p><p>The new M::P API will offer parsing results at two different levels:</p><ul> <li> <p>Makefile DOM tree</p><p>It's a syntax-oriented data structure which preserves every single bit of info in the original makefile (including whitespaces and comments). So one can modify some part of the DOM tree, and write the updated makefile back to disk. I think it's useful to some GUI apps which want to edit makefiles via menus and is also beneficial to the gmake =&gt; <a href="http://khemir.net/sscm_2006/slides/index_slide.html">PBS</a> translator.</p></li><li> <p>Makefile AST</p><p>The AST desugars the handwaving parts of the DOM tree down to a semantic-oriented data structure for make-like tools to ``run'' it or for some visualizer (e.g. my <a href="http://search.cpan.org/perldoc?Makefile::Graphviz">Makefile::Graphviz</a>) to depict the underlying dependency relations. For the <a href="http://khemir.net/sscm_2006/slides/index_slide.html">PBS</a> emitter, I think we should work out a special AST for it since the desugaring must be lossless, much like a program correctness proving system.</p></li></ul><p>I'm currently working on the M::P tokenizer and will finish the DOM tree constructor these days. The process should be going pretty fast since it is mostly test-driven.</p><p>The first goal is to implement the new M::P APIs and get my pgmake utility pass most of the gmake tests so that I can kick M::P 1.00 out of the door.</p><p>I'm stealing a lot of source code and pod from Alias's <a href="http://search.cpan.org/perldoc?PPI">PPI</a> module. I've noticed that the basic structure of PDOM trees can also fit my needs very well. it's called MDOM in my M::P though.<nobr> <wbr></nobr>;-)</p><div><p>Oct 24 (to Sun Xin~) </p><p>Take care. translating may drive you mad some day. just have appropriate amount of fun, dude!</p><div><p>Oct 26 (to Jack Shen and Sal Zhong~) </p><p>my gnu Makefile DOM builder now supports most kinds of rules, 2 flavors of variable assignments, macro interpolations, and various command and comment syntax. Now it's trivial to add new node types and extend the DOM parser.</p><p>i'll add support for double-colon rules, the define/vpath/include/ifeq/ifneq/ifdef/ifndef/... directives, and other missing structures tomorrow. After these additions, the DOM parser will be quite complete and will serve as the solid ground that we keep standing on. constructing the Makefile AST will be much easier if we keep a DOM tree handy.</p><p>yay for test-driven development! without TDD or Alias' <a href="http://search.cpan.org/perldoc?PPI">PPI</a> , i wouldn't have progressed so rapidly.<nobr> <wbr></nobr>;-)</p><div><p>Oct 29 (to Sal Zhong~) </p><p>When and where shall we take the Java exam?</p><p>...Oops, it seems impossible to release <a href="http://search.cpan.org/perldoc?UML::Class::Simple">UML::Class::Simple</a> tonight. still have several missing features to implement and the pod needs loves too. hmm, christopher may be unhappy since i earlier made the promise to him that i would make the release by *this* weekend. sigh. hopefully i'll get some cycles tomorrow.</p><p>...nod nod. but i also gotta review the data mining textbooks for the coming exam. furthermore, i'm planning to hack on two expert systems in the next week. i'll be programming in Prolog, <a href="http://www.ghg.net/clips/CLIPS.html">CLIPS</a> , and Perl simultaneously, which must be a lot of fun! yay!<nobr> <wbr></nobr>:D</p><div><p>Oct 30 (to Sal Zhong~) </p><p>I've just talked to Alias, the author of <a href="http://search.cpan.org/perldoc?PPI">PPI</a> , on <code>#perl</code>. he said that i could borrow as much source code from <a href="http://search.cpan.org/perldoc?PPI">PPI</a> as i would for my <code>Makefile::DOM</code> module. <a href="http://search.cpan.org/perldoc?PPI::Element">PPI::Element</a>, <a href="http://search.cpan.org/perldoc?PPI::Node">PPI::Node</a>, <a href="http://search.cpan.org/perldoc?PPI::Token">PPI::Token</a>, and <a href="http://search.cpan.org/perldoc?PPI::Dumper">PPI::Dumper</a> can be reused by my MDOM directly without many changes. i also briefly introduced the two-level ASTs to him and expressed my appreciation of <a href="http://search.cpan.org/perldoc?PPI">PPI</a> . It has given me plenty of inspiration on how to push my <a href="http://search.cpan.org/perldoc?Makefile::Parser">Makefile::Parser</a> further.</p><p>This journal was originally posted as <a href="http://agentzh.spaces.live.com/blog/cns!FF3A735632E41548!128.entry">http://agentzh.spaces.live.com/blog/cns!FF3A735632E41548!128.entry</a> </p></div></div></div></div></div></div></div></div></div></div></div> agent 2006-10-30T13:59:28+00:00 journal Notes for this fortnight (2006-10-01 ~ 2006-10-16) http://use.perl.org/~agent/journal/31343?from=rss <div><p>Oct 1 (to Sun Xin~) </p><p>Please check out your mailbox. i sent one journal and 107 <a href="http://perlcabal.org/agent/slides/naming/naming.pdf">slides</a> to you for proofreading yesterday. remember that i've said i would try my best to keep you relatively busy?<nobr> <wbr></nobr>:)</p><div><p>Oct 4 (to Sal Zhang~) </p><p>I've rewritten your Win32::xul2ppt_mec module using <a href="http://search.cpan.org/perldoc?Win32::OLE">Win32::OLE</a> and stevan's excellent <a href="http://search.cpan.org/perldoc?Moose">Moose</a> module. Now it's named XUL::Image::PPT and the xul2ppt utility has been divided into two separate tools, xul2img.pl and img2ppt.pl. Please check out <a href="http://svn.berlios.de/svnroot/repos/unisimu/Perl/XUL-Image-PPT/">http://svn.berlios.de/svnroot/repos/unisimu/Perl/XUL-Image-PPT/</a> for the source code.<nobr> <wbr></nobr>:)</p><p>Regarding the new xul2img utility, the --count and --title options are required. use --help to see the usage. because the XUL =&gt; image part is still based on <a href="http://search.cpan.org/perldoc?Win32::GuiTest">Win32::GuiTest</a>, the user interface is somewhat fragile and cannot be as nice as that of img2ppt. it's still the user's responsibility to open<nobr> <wbr></nobr>.xul with firefox and not to enter the full view mode (via F11) before running the xul2img tool.</p><p>Delay settings like 0.5 sec should also work now since i've switched to <a href="http://search.cpan.org/perldoc?Time::HiRes">Time::HiRes</a>'s <code>sleep</code> function. btw, <a href="http://search.cpan.org/perldoc?Moose">Moose</a> is so cool that writing perl 5 OO code has been exceedingly enjoyable. you know, perl 5's OO was ever a weak or even boring part in the past. <a href="http://search.cpan.org/perldoc?Moose">Moose</a> has brought me the feeling of using Perl 6 *today*. So don't hesitate and give it a shot! Enjoy~</p><p>(agentzh mooses.)</p><div><p>Oct 5 (to Sun Xin~) </p><p>Currently i am making slides for my XML talk. the topic is ``XML in the real world''. will send the slides to you for review once they're ready.<nobr> <wbr></nobr>:)</p><div><p>Oct 6 (to Cherry Chu~) </p><p>I will send you a message when i get up tomorrow morning. please keep your phone on, OK? if you get up earlier than i, would you please inform me via a message? thank you.<nobr> <wbr></nobr>:)</p><div><p>Oct 6 (to Jack Shen~) </p><p>The <a href="http://perlcabal.org/agent/slides/xmlapp/xmlapp.pdf">slides</a> for my XML talk are ready now. please check out your mailbox for details. the slides contain a lot of pretty pictures. i've covered hot topics like RSS and AJAX using Google Reader, the Qzone site, and my GetQzone utility as study cases. these topics are extremely exciting! comments on my slides will be appreciated.<nobr> <wbr></nobr>:) i hope miss zheng will be kind enough to give me more time to explain everything in my slides...hehe.</p><div><p>Oct 7 (to Cherry Chu~) </p><p>cherry: moose.<nobr> <wbr></nobr>:)</p><p>cherry: elk!<nobr> <wbr></nobr>:D</p><p>I'm now heading out.<nobr> <wbr></nobr>:) 7:15 AM. don't be late, cherry.</p><p>...yay! cherry++ i'm already waiting for you.<nobr> <wbr></nobr>:)</p><p>...i am home now, cherry! yay!<nobr> <wbr></nobr>...I was walking pretty fast. hehe. have a good rest. hopefully you will regain your strength soon.<nobr> <wbr></nobr>:)</p><p>Take care and sleep early, cherry. gotta run to shower and sleep myself. G'night &amp;</p><div><p>Oct 8 (to Sun Xin~) </p><p>Cherry and i rode to the yangzhou city yesterday. we favored small roads in the fields over big ones. as a result, we were often followed by barking dogs and blocked by rivers and fields in our way. it was frustrating but also fun. she was amazingly vigorous and charming yesterday...we talked very happily and laughed a lot. you know, it was quite amusing to see she also talked and laughed very loudly, just like me! yay! hooray for cherry's beauty and the enormous parallels between us! hehe.</p><p>we've decided to ride to other cities in the next few times. but it's still undecided which city to go first. what's your opinion, man?<nobr> <wbr></nobr>;-)</p><div><p>Oct 8 (to Cherry Chu~) </p><p>how are you today? i am still a bit tired. sigh.</p><p>...wow, nice to hear that. btw, i'm happy to see my friend <a href="http://layesuen.spaces.live.com/">laye</a> has replied to your journal. he's a talented programmer and now studying in the Fudan university.<nobr> <wbr></nobr>:) And your ``journal of 70 kilometers'' <a href="http://cherrychuxinyun.spaces.live.com/Blog/cns!BA706CF728A694B9!302.entry">post</a> reads very well!<nobr> <wbr></nobr>:)</p><p>...nod nod. he was in ujs when he was an undergraduate student. sadly we have never met in person.<nobr> <wbr></nobr>:(</p><div><p>Oct 9 (to Sal Zhang and Jack Shen~) </p><p>Yay! now i can do Java Swing programming in pure perl 5! furthermore, my perl interpreter can now learn new Java libraries all by itself. so i can manipulate *any* Java classes and objects as if they were implemented directly in perl 5. thanks to <a href="http://search.cpan.org/perldoc?Inline::Java">Inline::Java</a> and <a href="http://search.cpan.org/perldoc?Java::Swing">Java::Swing</a>. now i'm trying to get them work with pugs (i.e. perl 6). unfortunately, pugs doesn't do auto-importing for perl 5 modules. sigh. maybe i need to write some perl 5 wrappers and glue code there. oh, well...</p><div><p>Oct 9 (to Sal Zhong~) </p><p>huh! google++</p><p>i will definitely look into its shiny source code search engine the other day. thanks for the info.<nobr> <wbr></nobr>:)</p><div><p>Oct 9 (to Sun Xin~) </p><p>man, i'll (selectively) translate these notes myself because i don't want to occupy too much of your spare time. anyway, i can do the translation work more easily and more accurately. would you please proofread both my english and chinese transcripts for me? i'll be very grateful to your review!<nobr> <wbr></nobr>;)</p><div><p>Oct 10 (to Jack Shen~) </p><p>I've nailed down the basic syntax of the SXML language. it looks pretty neat. i'll implement converters for XML &lt;=&gt; SXML and HTML &lt;=&gt; SXML. i believe it's important enough for both XML's human reading and human writing.</p><div><p>Oct 10 (to Cherry Chu~) </p><p>Moose. will you come to my class this friday evening?<nobr> <wbr></nobr>:)</p><div><p>Oct 16 (to Sal Zhong~) </p><p>jerry gay (the guy also known as particle) is <a href="http://colabti.de/irclogger/irclogger_log/perl6?date=2006-10-16,Mon&amp;sel=193#l328">rewriting</a> my smartlinks.pl using <a href="http://search.cpan.org/perldoc?Moose">Moose</a> . it's really wonderful! he will commit the code to the parrot repos. He said he would introduce smartlinks to the parrot test suite and link the tests to both the Perl 6 Spec and the parrot PDDs. not sure if he still has the crazy plan to port smartlinks.pl to PIR. anyway, as christopher said, the idea of smartlinking has inspired several add-on hacks. hooray!</p><p>btw, pugs 6.2.13 is going to release tomorrow. larry is <a href="http://use.perl.org/~chromatic/journal/31301">using</a> pugs for his $work! sweet...</p><p>This entry was originally posted to</p><p> <a href="http://agentzh.spaces.live.com/blog/cns!FF3A735632E41548!125.entry">http://agentzh.spaces.live.com/blog/cns!FF3A735632E41548!125.entry</a> </p></div></div></div></div></div></div></div></div></div></div></div></div></div></div> agent 2006-10-17T13:42:26+00:00 journal Notes for this fortnight (2006-09-21 ~ 2006-09-30) http://use.perl.org/~agent/journal/31184?from=rss <div><p>Sep 21 (to Sun Xin~) </p><p>our charming XML instructor today asked me to give one or two talks in her class because she thought i was an expert in this domain. i'm very excited and have decided to make some good- looking slides in both english and chinese. i'll work on the new slides for the coming talks in the next few days. i'll send them to you for review once they are ready.</p><p>i'll also send you a bunch of <a href="http://perlcabal.org/agent/slides/patterns/patterns.xul">_old_slides_</a> for the talk given in the last term tomorrow. they're in pure english and most of my classmates liked it. i hope you can proofread the old slides since i'm going to publish them on the web.</p><p>Our XML instructor is so beautiful that i dare say a lot of boys in the classroom like her very much. she holds great charm for me. i love to talk to her after the class. talking to her in person is really enjoyable. she is an extremely lovely girl. yay for her beauty and good mood!</p><div><p>Sep 22 (to Sun Xin~) </p><p>mails sent. remember to use Firefox to access the<nobr> <wbr></nobr>.xul URL (i.e. the slides) and don't click too fast while reading the slides since loading images can be slow.</p><p>btw, i'll use cherry's qzone blogs as a study case in my new slides for the XML talk. the slide-making process can be fun! stay tuned!<nobr> <wbr></nobr>:)</p><div><p>Sep 22 (to Jack Shen~) </p><p>hey, jack. let me talk about microcosmic stuff, such as interface design for individual classes or small class library and you talk about the handwavy macroscopical things like large OO systems. i'll try my best to put enough basic weapons under the audience's belt before your lectures. what's your opinion?</p><div><p>Sep 23 (to Sun Xin~) </p><p>my instructors have asked me to give for total 12 talks this semester. that's really wonderful since i can take a more leisure pace during my talks. but i definitely need many more slides and pictures. i'll be talking about XML, Regular expressions (regexes) and object-oriented modeling and design. what cool stuff!</p><div><p>Sep 27 (to Sun Xin~) </p><p>i've produced 82 slides these two days, and i am about to crash... for my first talk, there are still 20 slides to go... it's really exciting!!! man!!!</p><p>man, i've sent a weird english poetry to you for translating. i need the chinese transcript for my slides. please get back to me ASAP. it's quite urgent since the talk is scheduled on *this* Friday evening. thank you in advance.<nobr> <wbr></nobr>:)</p><div><p>Sep 27 (to Cherry Chu~) </p><p>our XML teacher has asked me to give one or two talks to my classmates in her class. and i am going to use *your* Qzone blogs as a study case in my slides.<nobr> <wbr></nobr>:)</p><p>will you mind my introduction to your Qzone home? btw, an even shorter URL is working now: <a href="http://perlcabal.org/agent/cherry.html">http://perlcabal.org/agent/cherry.html</a>. feel free to give it a shot.</p><p>...glad to hear that. i have 16 lectures to give out this semester. i've been busy making slides for my talks. it is a hard job but is also fun.<nobr> <wbr></nobr>:)</p><div><p>Sep 29 (to Sun Xin~) </p><p>the <a href="http://perlcabal.org/agent/slides/naming/naming.xul">talk</a> was a big success. the audience laughed a lot and i was often stopped by the girl students' ``wow''. my instructor said after the talk that he was exceedingly impressed. he told me that it had given him great inspiration and determination. he said he had even been pondering giving up his C.S. career, but my talk completely changed his mind.</p><p>...nah. cherry didn't come because she is in a different department and i had not invited her to my talk.</p><p>...sorry, i didn't show your transcript explicitly in my slides. Audrey <a href="http://colabti.de/irclogger/irclogger_log/perl6?date=2006-09-29,Fri&amp;sel=144#l220">offered</a> a translation in ancient chinese right before the talk and i used hers. but your work had helped me a lot. without your translation, i can't grok that poem to such an extend. thank you!</p><p>it's worth mentioning that Larry Wall also <a href="http://colabti.de/irclogger/irclogger_log/perl6?date=2006-09-29,Fri&amp;sel=120#l181">provided</a> me with an excellent translation in modern english. you know, he is a great linguist.<nobr> <wbr></nobr>:)</p><p>...LOL. lucky indeed. getting so much help and support is like a dream!</p><div><p>Sep 30 (to Cherry Chu~) </p><p>Heh, it will rain tomorrow anyway. hopefully the weather will get better when you come back from shanghai.<nobr> <wbr></nobr>:)</p><p>...nope, not that one. i was talking about the art of naming. the talk mentioning your Qzone space will be given on Oct 14, which is about XML in the real world.</p><p>Sun Xin asked me last night if cherry had attended my talk. and i explained to him that you are in a different department and sadly i had not sent you an invitation.</p><p>The talk was on this Friday evening. and i'll give talks at that time for every week from now on. we are at Z101. welcome joining us! 9th and 10th classes.<nobr> <wbr></nobr>:)</p><p>...nah, chinese speech mostly. every slide contains two versions of the content, the chinese version and the english version. and yeah, there'll be many students in my class. don't worry.<nobr> <wbr></nobr>:)</p><p>it will simply rock if you can come. for another thing, i really hope you can also attend my XML talk because i'm going to show my classmates your blogs there. he he.</p><p>the XML talk has been scheduled at 10504, 7th or 8th class. the concrete date is still undecided. i'll tell you once i had talked to our XML instructor. OK?</p><p>...(agent does his happy dance.)</p><p>the XML talk should be on Thursday afternoon, btw.</p><p>cherry, i have the idea of writing journals based on my cellphone messages. it's a great source of materials, you know. of course, i won't public any messages that i've received from others. for example, your replies will be excluded while my messages to you will probably be shown. what's your opinion? will you mind?</p><p> This journal has also been posted at <a href="http://agentzh.spaces.live.com/blog/cns!FF3A735632E41548!124.entry">http://agentzh.spaces.live.com/blog/cns!FF3A735632E41548!124.entry</a></p></div></div></div></div></div></div></div></div> agent 2006-10-01T06:27:12+00:00 journal The Genetic Algorithm Used by Audrey http://use.perl.org/~agent/journal/29258?from=rss <tt>I've been reading #perl6 IRC logs for more than a year and it's a very good way to sync with the rapid Perl 6 development.<br><br>Sometimes I find something really, really interesting so I'd like to quote them here. After all, I know there're many Perl 5 programmers (like me!) who love to learn more about Perl 6 and the future of Perl 5.<br><br>2006-04-06<br>-----------<br>[04:55] &lt;arcady&gt; by the way, how much of "the perl 6 grammar" exists at the moment?<br>[04:58] &lt;TimToady&gt; depends on how you count, I suppose.&nbsp; rule syntax is fairly well characterized by now.&nbsp; a lot of it is specced pretty well, for some definition of pretty that ain't pretty.<br>[04:59] &lt;arcady&gt; so at least we can have something like the grammar grammar<br>[04:59] &lt;TimToady&gt; Most of the operator precedence is not done with rules at all.<br>[04:59] &lt;TimToady&gt; Yes, the grammar grammar is already bootstrapped approximately twice.<br>[04:59] &lt;arcady&gt; it's kinda hard to keep track of all the stuff going on...<br>[05:00] &lt;TimToady&gt; there'e very little top-down grammar over the bottom-up expression parser.<br>[05:00] &lt;TimToady&gt; more top-down involved in scanning complex tokens containing subexpressions.<br>[05:00] &lt;TimToady&gt; but the main complication remaining is just making sure all the grammatical categories work as envisioned.<br>[05:01] &lt;TimToady&gt; Then there's just little detail of attaching semantics to the parse...&nbsp;<nobr> <wbr></nobr>:)<br>[05:01] &lt;TimToady&gt; s/little/the little/<br>[05:02] &lt;arcady&gt; that can be left as an exercise to the implementors : )<br>[05:02] &lt;TimToady&gt; But audreyt says that Perl 6 now fits in her head, so that should be finished a day or two after the grammar is done.<br>[05:04] &lt;arcady&gt; that would be most awesome<br>[05:05] &lt;TimToady&gt; well, even a month or two would be awesome.&nbsp; a year or so is more likely before we have something really, really solid.&nbsp; Still, I'm very happy with how it's going.<br>[05:05] &lt;arcady&gt; well, I'm happy that you're happy, and that it's going<br>[05:05] &lt;arcady&gt; I guess it's not entirely obvious from here<br>[05:06] &lt;arcady&gt; and all the various bootstrap efforts and targets and so on are confusing<br>[05:07] &lt;TimToady&gt; Hmm, yes.&nbsp; Audrey<br>[05:07] &lt;TimToady&gt; Audrey's development methodology resembles a flooding algorithm at times...<br>[05:08] &lt;TimToady&gt; Or maybe a genetic algorithm.<br><br>__END__<br><br>Yeah, in the last year, I was also confused by the JavaScript, Perl5, and all other backends for Pugs. I was asking in my mind, "why can't autrijus focus on parrot, which is believed to be the only VM Perl 6 should be run on? Isn't it possible that the various backends will slow down the roaring speed of Pugs?" Now I finally understand the approach Audrey has been taking -- genetic algorithm or something like that. Given that there're always more than one way to do it, how can we figure out the "best" way if we haven't tried others yet? And furthermore, never forget that -Ofun is always the meta goal of the Pugs project.<nobr> <wbr></nobr>:)<br><br>parrot may not be the only choice and may not be the best choice, as evidenced by the following conversation:<br><br>2006-04-06<br>-----------<br>[05:11] &lt;arcady&gt; what's going on with parrot, by the way?<br>[05:12] &lt;arcady&gt; and how does any of that connect to any of this?<br>[05:13] &lt;TimToady&gt; Parrot is sort of the other end of the world from me, so I just follow along in p6i mostly.&nbsp; I hear conflicting things, but I think it'll get there eventually, for some definition of "there".<br>[05:14] &lt;TimToady&gt; Whether it will be "the" Perl platform or "a" Perl platform, or somewhere in between, remains to be seen.<br><br>__END__<br><br>P.S. I must admit, Audrey's "Genetic Algorithm" is funny and helpful even in a general sense. I've successfully applied that to most of my open source projects. Multiple approaches and multiple perspectives often lead to surprisingly deep insights. That may be the most useful "algorithm" I learned from Audrey++.<nobr> <wbr></nobr>;-)</tt> agent 2006-04-08T06:18:49+00:00 journal use HTTP::Proxy to log my web accessing history http://use.perl.org/~agent/journal/28415?from=rss <p>Yeah, I visit many websites everyday. what I'm wanting and what I'm always looking for is a facility to automagically keep a record of the URLs and page titles I've just accessed, so that I can analyse the history some time later to find out the focus of my interest in a particular period of time, for example. And it's very likely that I can come up with even more interesting statistical consequences.</p><p>The Mozilla browser doubtlessly gives builtin support for accessing history, but unfortunately exporting that history info is not trivial. what I want is not only the URLs, but also the corresponding page titles (if any!) and the visiting time stamp.</p><p>Several weeks ago, I happily found that the CPAN module HTTP::Proxy can come to the rescue. What I need to do is just writing several lines of Perl code using that module, running this script at the background as a local HTTP proxy server, and setting my web browser to simply use that. By doing this, my local proxy has a chance to monitor all the HTTP traffic between my browser and the Internet.</p><p>It's fun to see that my local proxy server can also use a remote proxy. so my local one then becomes a secondary proxy, no?<nobr> <wbr></nobr>;D</p><p>The HTTP::Proxy module also supports logging internally, thus my code is even simpler:<br><code><br>use HTTP::Proxy ':log';<br>my $logfile = "&gt;&gt;$home/myproxy.log";<br>open my $log, $logfile or<br> &nbsp; &nbsp; &nbsp; &nbsp; die "Can't open $logfile for reading: $!";<br>my $proxy = HTTP::Proxy-&gt;new(<br> &nbsp; &nbsp; &nbsp; &nbsp; logmask =&gt; STATUS,<br> &nbsp; &nbsp; &nbsp; &nbsp; logfh =&gt; $log,<br>);<br></code><br>The logmask parameter here controls what kind of things the proxy should record. the STATUS constant indicates only basic URL and response code will be logged. What I get in the log file is something like this:<br><code><br>[Fri Jan 13 17:25:42 2006] (1888) REQUEST: GET http://www.perl.com/<br>[Fri Jan 13 17:25:53 2006] (1888) RESPONSE: 200 OK<br>[Fri Jan 13 17:25:53 2006] (1888) REQUEST: HEAD http://www.google.com/mozilla/google.src<br>[Fri Jan 13 17:25:54 2006] (1888) RESPONSE: 200 OK<br>[Fri Jan 13 17:25:54 2006] (1888) REQUEST: GET http://www.perl.com/styles/main.css<br>[Fri Jan 13 17:25:55 2006] (1888) RESPONSE: 304 Not Modified<nobr> <wbr></nobr>...<br></code><br>Hmm...very cute! However, HTTP::Proxy's builtin logging mechanism doesn't respect HTML titles. Thus I need to provide a user agent of my own:<br><code><br>package MyUA;<br>use HTTP::Proxy ':log';<br>use base 'LWP::UserAgent';<br>sub send_request {<br> &nbsp; &nbsp; &nbsp; &nbsp; my ($self, $request) = @_;<br> &nbsp; &nbsp; &nbsp; &nbsp; my $response;<br> &nbsp; &nbsp; &nbsp; &nbsp; eval {<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $response = $self-&gt;SUPER::send_request( $request );<br> &nbsp; &nbsp; &nbsp; &nbsp; };<br> &nbsp; &nbsp; &nbsp; &nbsp; if ($@ and not $response) {<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return HTTP::Response-&gt;new(500, $@);<br> &nbsp; &nbsp; &nbsp; &nbsp; }<br> &nbsp; &nbsp; &nbsp; &nbsp; if ($response-&gt;is_success) {<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; my $type = $response-&gt;header('content-type');<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if ($type and $type =~ m[text/html]i) {<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if ($response-&gt;content =~ m[\s*(.*\S)\s*]si) {<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; $proxy-&gt;log( STATUS, 'TITLE', $1);<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }<br> &nbsp; &nbsp; &nbsp; &nbsp; }<br> &nbsp; &nbsp; &nbsp; &nbsp; return $response;<br>}<br></code><br>Now we have HTML titles recorded down as well, as witnessed in my log file:<br><code><br>[Tue Jan 17 20:33:47 2006] (2484) REQUEST: GET http://perladvent.org/2004/20th/<br>[Tue Jan 17 20:33:50 2006] (2484) TITLE: Perl 2004 Advent Calendar: Filesys::Virtual<br>[Tue Jan 17 20:33:50 2006] (2484) RESPONSE: 200 OK<br></code><br>Then feed the customized user agent to my HTTP::Proxy instance I created earlier:<br><code><br>my $agent = MyUA-&gt;new(<br> &nbsp; &nbsp; &nbsp; &nbsp; env_proxy =&gt; 1,<br> &nbsp; &nbsp; &nbsp; &nbsp; timeout =&gt; 100,<br>);<br>$proxy-&gt;agent( $agent );<br></code><br>At last, we enter an infinite loop as every http proxy server:<br><code><br>while (1) {<br> &nbsp; &nbsp; &nbsp; &nbsp; eval { $proxy-&gt;start(); };<br> &nbsp; &nbsp; &nbsp; &nbsp; warn $@ if $@;<br>}<br></code><br>That's it!</p><p>It already works for me, but there're still several pitfalls in this solution:</p><ul><li>Images won't display in MS Internet Explorer (Mozilla works fine, however)</li><li>It seems to me that HTTP::Proxy doesn't support forking by default so it leads to poor performance if I request multiple URLs simultaneously. (BTW, Is there a way to switch to a forking engine? I can't find a word in its POD docs.)</li><li>SSL connection doesn't work on my box.</li></ul><p>Have fun!</p> agent 2006-01-20T14:03:52+00:00 journal Tried out Perl6::Attributes http://use.perl.org/~agent/journal/27788?from=rss <p>If you have ever written any Perl OO code, I recommend you to take a look at Luke Palmer's CPAN module Perl6::Attributes, which is extremely simple and handy.</p><p>Instead of writing something like<br><code><br> &nbsp; &nbsp; sub method {<br> &nbsp; &nbsp; &nbsp; &nbsp; my $self = shift;<br> &nbsp; &nbsp; &nbsp; &nbsp; $self-&gt;{attr} = '...';<br> &nbsp; &nbsp; }<br></code><br>we can now write<br><code><br> &nbsp; &nbsp; sub method {<br> &nbsp; &nbsp; &nbsp; &nbsp; my $self = shift;<br> &nbsp; &nbsp; &nbsp; &nbsp; $.attr = '...';<br> &nbsp; &nbsp; }<br></code><br>in Perl5.</p><p>Another example is replacing the ugly syntax<br><code><br> &nbsp; &nbsp; sub method {<br> &nbsp; &nbsp; &nbsp; &nbsp; my $self = shift;<br> &nbsp; &nbsp; &nbsp; &nbsp; @{$self-&gt;{attrs}} = qw(a b c);<br> &nbsp; &nbsp; }<br></code><br>with the Perl6-ish syntax<br><code><br> &nbsp; &nbsp; sub method {<br> &nbsp; &nbsp; &nbsp; &nbsp; my $self = shift;<br> &nbsp; &nbsp; &nbsp; &nbsp; @.attrs = qw(a b c);<br> &nbsp; &nbsp; }<br></code><br>You see, it is pretty cool!</p><p>I've already had a try in my recent Perl module developements. Perl6::Attributes makes the source of my Perl OO modules amazingly neat.</p> agent 2005-11-29T14:23:25+00:00 journal MASM Port for Test::More http://use.perl.org/~agent/journal/27458?from=rss <p>The upcoming CPAN module, Test::Masm::Dos, will be a MASM port for the excellent Test::More module on Perl's land. The approach is to give special meanings to MASM comments so that we can write something like<br><code><br> &nbsp; &nbsp; &nbsp; &nbsp; mov ax, 0ffffh<br> &nbsp; &nbsp; &nbsp; &nbsp; ; is ax, -1, "Test a negative number";<br></code><br>directly in the context of 16-bit MASM source code. The implementation for these smart comments is straightforward too -- expand each comment with a piece of assembly code that implements exactly what the comment says. Hence most of the amazing power the Perl QA Group owns can be brought to the poor 16-bit MASM, a legacy MS-DOS assembly language developed by Microsoft.</p><p>Here is a complete (thought still trivial) example:<br><code><br> &nbsp; &nbsp; &nbsp; &nbsp; ; foo.asm<br> &nbsp; &nbsp; &nbsp; &nbsp; code segment<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; assume cs:code<br> &nbsp; &nbsp; &nbsp; &nbsp; start:<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ; tests =&gt; 5;<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mov ax, 1011011B<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; inc ax<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ; is ax, 91;<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; mov cx, 5<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; next:<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ; say "# cx = ", cx;<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ; ok cx <br><code><br> &nbsp; &nbsp; &nbsp; &nbsp; code ends<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; end start<br></code><br>Then we could feed it to my masmt script:<br><code><br> &nbsp; &nbsp; &nbsp; &nbsp; masmt -o ~foo.asm foo.asm<br></code><br>where ~foo.asm is a temporary program with all the smart comments expanded by corresponding assembly instructions. So now we can assemble this to an executable named ~foo.exe, which is a MASM tester that produces TAP report!</code></p><p><code>At this point, I've only implemented the "say" comment. However, my classmates and I have already found it very helpful, especially when debugging our assembly programs. You know, it's really a painful experience to do even simple I/O in MASM!</code></p><p><code>With smart comments in hand, we can easily realize our dreams of applying Perl's debugging strategy to MASM: tweak, run, and tweak, run!</code></p> agent 2005-11-05T05:50:07+00:00 journal A flowchart generator using GraphViz http://use.perl.org/~agent/journal/27403?from=rss <tt>I'm now hacking on a GraphViz application that generates flowcharts. I've designed a tiny scripting language to describe the detailed structure of a program's control flow. This language is very similar to MASM, since flowcharts look pretty close to assembly languages from the point view of branching. Here's a tiny example to illustrate the basic idea:<br><br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;_______<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ( start )<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;-------<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;||<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\/<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<nobr> <wbr></nobr>/----------------/<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;/ input a string &lt;--------+<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<nobr> <wbr></nobr>/----------------/&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;||&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\/&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; +--------------------+&nbsp; &nbsp; &nbsp;|<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; | convert to a number|&nbsp; &nbsp; &nbsp;|<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; +--------------------+&nbsp; &nbsp; &nbsp;|<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;||&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\/&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;/---------------------/&nbsp; &nbsp; &nbsp;|<br>&nbsp; &nbsp; &nbsp; &nbsp; / output the hex form<nobr> <wbr></nobr>/&nbsp; &nbsp; &nbsp; |<br>&nbsp; &nbsp; &nbsp; &nbsp;/---------------------/&nbsp; &nbsp; &nbsp; &nbsp;|<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;||&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;||&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\/&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;||&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<nobr> <wbr></nobr>/&nbsp; \&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;/&nbsp; &nbsp; \&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<nobr> <wbr></nobr>/&nbsp; &nbsp; &nbsp; \&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;|<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;-&nbsp; quit? +------------+<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; \&nbsp; &nbsp; &nbsp;<nobr> <wbr></nobr>/<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\&nbsp; &nbsp;<nobr> <wbr></nobr>/<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; \&nbsp;<nobr> <wbr></nobr>/<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;||<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;||<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;\/<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;______<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; (&nbsp; end )<br>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;------<br><br>The corresponding script is as following:<br><br>&nbsp; &nbsp; &nbsp; &nbsp; start&nbsp; &nbsp;start<br>&nbsp; &nbsp; &nbsp;L1:<br>&nbsp; &nbsp; &nbsp; &nbsp; io&nbsp; &nbsp; &nbsp; input a string<br>&nbsp; &nbsp; &nbsp; &nbsp; do&nbsp; &nbsp; &nbsp; convert to number<br>&nbsp; &nbsp; &nbsp; &nbsp; io&nbsp; &nbsp; &nbsp; output&nbsp; the hex form<br>&nbsp; &nbsp; &nbsp; &nbsp; test&nbsp; &nbsp; quit?<br>&nbsp; &nbsp; &nbsp; &nbsp; jno&nbsp; &nbsp; &nbsp;L1<br>&nbsp; &nbsp; &nbsp; &nbsp; end&nbsp; &nbsp; &nbsp;end<br><br>Note that the opcode of each Asm-like instruction is a keyword, such as "start", "io", "do", "test", "jno" and "end". The label "L1" is also critical to the flowchart's structure. Other stuff, such as the long string "convert to a number", is all arbitrarily determined by the user. The compiler just displays these chars in the flowchart verbatim.<br><br>Well, it's time for me to name this scripting language formally. Eh, FlowAsm is a good choice. So all FlowAsm source files should be ended with the<nobr> <wbr></nobr>.fa extension.<br><br>FolwAsm is already simple, but it's some kind of immediate languages which are not supposed to be used directly by the user. Hence, the end-user will probably use a more advanced pseudo-language which has saner control flow structures like "if", "while", and "for" statements. Moreover, we may be able to translate real-world C/C++/Perl code to FlowAsm code in the foreseeable future. Converting assembly programs to FolwAsm scripts can be easier.<br><br>I started this project because I found no parallel efforts on CPAN except an ASCII version named Text::Flowchart.<br></tt> agent 2005-11-01T10:46:29+00:00 journal Writing English stuff with help from Google http://use.perl.org/~agent/journal/27046?from=rss <p>In this journal I'd make one of my secrets public.</p><p>Every time I want to use an English expression that I'm not quite familiar with, I always first have a google for it. If there're a lot of hits, I'll then feel pretty confident to write it down. If not, it's still very likely to find the correct form according to the google results. Google is really an extremely rich source of English language materials. It's able to perfectly answer questions like "Do English speakers usually say this way?"</p><p>Frankly speaking, all the successful websites, such as Google, eBay, PayPal, and Yahoo, really has changed our ways of life significantly.</p><p>I think it'll be a great idea to write this "secret" into my journal. It's doubtless beneficial to share such experience with people from non-English countries all over the world.</p> agent 2005-10-06T13:28:29+00:00 journal Tried SVN out http://use.perl.org/~agent/journal/27045?from=rss <p>Today I did nothing but try Subversion out. Thanks to the <a href="http://svn.openfoundry.org/">free SVN server</a> provided by <a href="http://rt.openfoundry.org/">OSSF</a>. I've hosted two of my CPAN contributions to that server. I also appreciate <a href="http://tortoisesvn.tigris.org/">TortoiseSVN</a>, which is a very handy SVN client on Win32. Centralized version control seems quite sufficient to me. However, I still have an eager to take a look at the famous <a href="http://svk.elixus.org/">SVK</a>. Unfortunately I don't seem to have enough time to try SVK tonight.</p><p>Geoff Broadwell <a href="http://www.oreillynet.com/pub/wlg/7996">said in O'Reilly's weblog</a> that, "If you're not already using a version control system, shame on you." Happily I can announce at this moment that I'm already using such a system for my software development. Whee!<nobr> <wbr></nobr>:=)</p> agent 2005-10-06T12:30:33+00:00 journal Makefile::Parser and Makefile::GraphViz http://use.perl.org/~agent/journal/27027?from=rss <p>I managed to release <a href="http://search.cpan.org/~agent/Makefile-Parser-0.06/lib/Makefile/Parser.pm">Makefile::Parser 0.06</a> and <a href="http://search.cpan.org/~agent/Makefile-GraphViz-0.03/lib/Makefile/GraphViz.pm">Makefile::GraphViz 0.03</a> to CPAN tonight. Hopefully I can see them appear on search.cpan tomorrow morning.</p><p>The former module is a Makefile parser which parses Makefiles to simple ASTs while the latter is a GraphViz application that generates pretty graphs that depicts the building flow directly from Makefiles. The <a href="http://search.cpan.org/src/AGENT/Makefile-GraphViz-0.03/samples.html">pictures</a> produced by GraphViz are amazingly good. I don't think I'm able to produce better pictures manually with MS Visio. Furthermore, everything in the output pictures are fully parameterized, leading to good flexibility.</p><p>Nadim Khemir told me that he was looking forward to use Makefile::Parser in his <a href="http://search.cpan.org/~nkh/PerlBuildSystem-0.35/">PerlBuildSystem</a>, which urges me to improve this stuff actively.</p><p>More feedbacks are most welcomed. Thank you in advance.</p> agent 2005-10-05T13:47:38+00:00 journal Failed to Run DOS Executables on Win2000 http://use.perl.org/~agent/journal/25018?from=rss <p>=from 2005.6.2.7:50.AM<br>=to<nobr> <wbr></nobr>...6.2.8:20.AM</p><p>A couple of weeks ago, I downloaded Microsoft's Segmented Executable Linker from the web, and built a trivial DOS program that printed out the perimeter of a circle on Windows 2000 Professional. Unfortunately, DOS interrupts</p><p> &nbsp; &nbsp; &nbsp; &nbsp; int 21h</p><p>didn't work at all in the MS-DOS box. So I couldn't use interrupts to output anything to the screen. I suspect that it will work on Win9x platforms anyway.</p><p>I wonder why most Chinese universities still teach 16-bit 8086 assembly language, which is completely out-of-date. I find it very difficult to locate an appropriate environment to experiment with the sample codes provided by our textbooks. Indeed, Win9x is now rarely seen on an ordinary PC.</p><p><a href="http://www.ht.sakura.ne.jp/~delmonta/programmierung/masm-e.html">MASM</a> is not flexible enough in my opinion; <a href="http://nasm.sourceforge.net/">NASM</a> may be a better choice. I plan to learn both when I take the course Assembly Language Programming next semester.</p> agent 2005-06-03T02:52:42+00:00 journal Summarization Completed! http://use.perl.org/~agent/journal/24971?from=rss <p>=from 2005.6.1.9:10.AM<br>=to<nobr> <wbr></nobr>...6.1.9:40.AM</p><p>I am delighted to see that I finally finished summarizing Carl's text "Computer Organization" today. The five summarized chapters are</p><ul><li> <a href="ftp://202.195.168.25/Writings/ComputerOrganization/Chapter2.html">Chapter2</a>: Machine Instructions and Programs</li><li> <a href="ftp://202.195.168.25/Writings/ComputerOrganization/Chapter4.html">Chapter4</a>: Input/Output Organization</li><li> <a href="ftp://202.195.168.25/Writings/ComputerOrganization/Chapter5.html">Chapter5</a>: The Memory System</li><li> <a href="ftp://202.195.168.25/Writings/ComputerOrganization/Chapter6.html">Chapter6</a>: Arithmetic</li><li> <a href="ftp://202.195.168.25/Writings/ComputerOrganization/Chapter7.html">Chapter7</a>: Basic Processing Unit</li></ul><p>which have covered almost all the materials contained in the Chinese textbook used in our department. Perl's POD format is used to produce these pretty pages.</p><p>I wish to express my thanks to many people who have helped during the preparation of these abstracts. My dad is the very person who first suggested that I should do such projects. And my grandpa always gave me a lot of great suggestions and warm-hearted encouragements. Finally I truly appreciate the support of my schoolmates, such as Ben (Dai Xiaoke) and Sal (Zhong Weixiang). Thank you all!</p> agent 2005-06-01T07:39:06+00:00 journal Regard Perl6 as a Super HDL http://use.perl.org/~agent/journal/24954?from=rss <p>=from 2005.5.31.8:00.AM<br>=to<nobr> <wbr></nobr>...5.31.8:45.AM</p><p>In hardware design, it is sometimes desirable to depict the logic circuit diagrams in a constructive manner, just like the way we build a complex data structure using a general-purpose programming language. For example, we might wish to express a cascaded connection of 32 full adders, which is known as a 32-bit carry-ripple adder, by the following pseudo code (I borrowed Perl6 syntax here to illustrate the idea):</p><p> &nbsp; &nbsp; &nbsp; &nbsp; my @adders;<br> &nbsp; &nbsp; &nbsp; &nbsp; for (0..31) -&gt; $i {<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; push @adders, FA.new(<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; name =&gt; "cell$i",<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; X =&gt; "x[$i]", Y =&gt; "y[$i]", Cin =&gt; "c[$i]",<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; S =&gt; "s[$i]", Cout =&gt; "c[" ~ ($i+1) ~ "]"<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; );<br> &nbsp; &nbsp; &nbsp; &nbsp; }</p><p>Unfortunately, so far as I know, earlier versions of Verilog HDL lack the ability to construct such things effectively, so a lot of typing is necessary if we use pure Verilog.</p><p>Is there a way to use Perl6 syntax directly as an HDL? The answer is certainly "yes". The simplest solution may be as follows:</p><ul><li> Write Perl6 codes just like the example we gave above using a predefined class library</li><li> Run this script and generate corresponding Verilog codes.</li></ul><p>In the preceding example, we might write a Perl6 program named adder32.p6, which probably prints out something like the following:</p><p> &nbsp; &nbsp; &nbsp; &nbsp; module adder32(x, y, c_in, c_out, s);<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; input [31:0] x, y;<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; input c_in;<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; output c_out;<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; output [31:0] s;<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; wire [32:0] c;<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; assign c_out = c[32];<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; assign c[0] = c_in;</p><p> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; FA cell0 (.X(x[0]),<nobr> <wbr></nobr>.Y(y[0]),<nobr> <wbr></nobr>.Cin(c[0]),<nobr> <wbr></nobr>.S(s[0]),<nobr> <wbr></nobr>.Cout(c[1]));<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; FA cell1 (.X(x[1]),<nobr> <wbr></nobr>.Y(y[1]),<nobr> <wbr></nobr>.Cin(c[1]),<nobr> <wbr></nobr>.S(s[1]),<nobr> <wbr></nobr>.Cout(c[2]));<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; FA cell2 (.X(x[2]),<nobr> <wbr></nobr>.Y(y[2]),<nobr> <wbr></nobr>.Cin(c[2]),<nobr> <wbr></nobr>.S(s[2]),<nobr> <wbr></nobr>.Cout(c[3]));<nobr> <wbr></nobr>// an awful lot omitted here...<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; FA cell30 (.X(x[30]),<nobr> <wbr></nobr>.Y(y[30]),<nobr> <wbr></nobr>.Cin(c[30]),<nobr> <wbr></nobr>.S(s[30]),<nobr> <wbr></nobr>.Cout(c[31]));<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; FA cell31 (.X(x[31]),<nobr> <wbr></nobr>.Y(y[31]),<nobr> <wbr></nobr>.Cin(c[31]),<nobr> <wbr></nobr>.S(s[31]),<nobr> <wbr></nobr>.Cout(c[32]));<br> &nbsp; &nbsp; &nbsp; &nbsp; endmodule</p><p>The code generation details can be encapsulated into the basic Perl6 library, thus becoming transparent to EDA designers. Some template-driven code generators, such as <a href="http://books.perl.org/book/207">Perl Template Toolkit</a>, are equally helpful.</p><p>It is worth noting that the <i>generate</i> construct provided by VHDL and <a href="http://www.asic-world.com/verilog/verilog2k.html">Verilog-2001</a> can build the 32-bit carry-ripple adder in our example by virtue of loops as well. However, at this moment, they seem to be only applicable to module instantiations. So we can expect the code generation solution based on Perl to have a great future.</p> agent 2005-05-31T03:27:39+00:00 journal Algorithm Simulation Using Haskell and Perl6 http://use.perl.org/~agent/journal/24945?from=rss <p>=from 2005.5.30.8:30.AM<br>=to<nobr> <wbr></nobr>...5.30.9:15.AM</p><p>There are some cases in which we only want to simulate some abstract algorithms without considering the hardware implementation issues. In such situations, Tesla is not very practical. On the other hand, we can perfectly achieve our goals by means of pure "software" methods. For instance, a simple algorithm that converts an unsigned binary number to its octal form can be implemented by the following Haskell codes:</p><p> &nbsp; &nbsp; &nbsp; &nbsp; bin2oct<nobr> <wbr></nobr>:: [Int] -&gt; [Int]<br> &nbsp; &nbsp; &nbsp; &nbsp; bin2oct xs = bin2oct' (replicate (3-r) 0 ++ xs)<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; where r = (length xs) `mod` 3</p><p> &nbsp; &nbsp; &nbsp; &nbsp; bin2oct'<nobr> <wbr></nobr>:: [Int] -&gt; [Int]<br> &nbsp; &nbsp; &nbsp; &nbsp; bin2oct' [] = []<br> &nbsp; &nbsp; &nbsp; &nbsp; bin2oct' (x:y:z:xs) = [x*4 + y*2 + z] ++ (bin2oct' xs)</p><p>As another example, consider the problem of transforming a decimal number to its binary form:</p><p> &nbsp; &nbsp; &nbsp; &nbsp; dec2bin<nobr> <wbr></nobr>:: Int -&gt; [Int]<br> &nbsp; &nbsp; &nbsp; &nbsp; dec2bin 1 = [1]<br> &nbsp; &nbsp; &nbsp; &nbsp; dec2bin 0 = [0]<br> &nbsp; &nbsp; &nbsp; &nbsp; dec2bin x = dec2bin (x `div` 2) ++ [x `mod` 2]</p><p>Perl6 can do this job equally well:</p><p> &nbsp; &nbsp; &nbsp; &nbsp; sub dec2bin(Int $x) {<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return $x if $x == 1|0;<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; dec2bin(int $x/2), $x % 2;<br> &nbsp; &nbsp; &nbsp; &nbsp; }</p><p>Functional programmers tend to think in a recursive way, I think.</p><p>Instead of verify the algorithms involved in logic design "electrically" in Tesla, we can play with such stuff in a complete mathematical sense and take advantage of various advanced programming techniques, such as FP. A lot of joy can be expected then.</p> agent 2005-05-30T13:26:02+00:00 journal Behavioral Tesla, Bus-ize Tesla, and Perl6 Tesla http://use.perl.org/~agent/journal/24944?from=rss <p>=from 2005.5.30.7:45.AM<br>=to<nobr> <wbr></nobr>...5.30.8:30.AM</p><p>Some interesting ideas jumped into my head while I was reading Carl's "Computer Organization" these days.</p><p>I suppose it is possible to translate Verilog's behavioral codes, such as the "always" blocks directly to Perl codes that implement *built-in* circuit elements, just like the AND gates. Such transformation can be mechanical, since Verilog and Perl share many control-flow structures.</p><p>Obviously, Tesla will no longer be a pure gate-level simulator after some built-in blackbox components are introduced. These blackboxes provide sequential logic behaviors to the outside world, but are implemented by pure "software" methods, not logic circuits!</p><p>Tesla's built-in component repertoire will be extensible. We can have complex component types like "Register" and "Counter" that are implemented by ordinary Perl codes. This feature not only eases the pain of writing a Verilog-to-Tesla compiler, but also makes Tesla extremely powerful in high-level simulations.</p><p>A further step can be taken by changing the Tesla's built-in signals from individual wires to multi-channel buses. Oh, no! I think it is more appropriate to use inheritance here. Let the Wire and Bus classes both be derived from an abstract class named Signal. Therefore, Wires and Buses will be distinguished without breaking the existing code that uses Signals.</p><p>Porting the whole Tesla system to Perl6 is even more attractive. Pugs now supports sufficient Perl6 features; this plan thus becomes quite feasible.</p> agent 2005-05-30T11:48:20+00:00 journal Make a Computer of My Own http://use.perl.org/~agent/journal/24830?from=rss <p>=from 2005.5.23.7:50.AM<br>=to<nobr> <wbr></nobr>...5.23.9:05.AM</p><p>The two weeks' efforts on summarizing Carl's "Computer Organization" have been proved fruitful. Now I've got some insight into the internals of computer systems. Maybe it's time for me to *make* a quite complete computer of my own. This idea seems to be a little insane at the first glance; however, it has been one of my goals since high school.</p><p>The current plan is as follows:</p><ol><li>Use standard Verilog HDL to implement the processor, the system bus, RAMs, caches, DMA controllers and some basic I/O device interfaces.</li><li> Test the Verilog design in commercial simulators and synthesizers, such as <a href="http://www.altera.com/products/software/products/quartus2/">Altera Quartus II</a> and <a href="http://www.model.com/">ModelSim</a>.</li><li> Implement a simple Verilog compiler that can convert the Verilog source to my Tesla HDL codes.</li><li> Perform real-time simulation by virtue of my Tesla simulator developed in last term.</li><li> Program the well-tested design down to the devices, such as FPGA, finishing the whole process.</li></ol><p>The step of developing a Verilog compiler may be unnecessary. Transforming the netlist generated by other people's Verilog compiler to Tesla code seems to be much more straightforward. Fortunately, the meta netlist (.vqm file) output by Quartus II is very similar to Verilog syntax, though a little odd. I wonder if there are any specifications for this format.</p><p>In fact, if there's a real-time simulator already available on the web, the steps 3 and 4 can be completely bypassed. I'm guessing the FLI (Foreign Language Interfaces) provided by ModelSim may be the right place to look at.</p><p>However, I still believe that it is doubtless more interesting and more challenging to simulate my design in my own simulator, Tesla, because all the boring principles and algorithms I learned from the course on Logic Design will get an opportunity to be applied to real lift. Yeah, every student has a dream to act as a "practicing theorist".</p><p>Verilog is very concise compared to VHDL, so it's an ideal language to describe large designs. I can't understand why our university chose VHDL in all our hardware courses. Weird.</p> agent 2005-05-23T09:49:23+00:00 journal Sorry for the Lack of Background http://use.perl.org/~agent/journal/24714?from=rss <p>=from 2005.5.16.1:40.PM<br>=to<nobr> <wbr></nobr>...5.16.2:25.PM</p><p>Some kind people pointed to me that my journals seriously lack necessary backgrounds and explanation of mysterious terms, which leads to the terrible fact that no one can understand them even a bit. I feel very sorry and regretful.</p><p>Above all, I suppose it's important to introduce myself a little. I'm a Chinese student majored in Computer Science in <a href="http://www.ujs.edu.cn/eng/ev/">Jiangsu University</a>, who loves perl and gets very interested in Autrijus' journals on this site. I have the honor to be able to upload my own journals too, because there are a lot of great people here.</p><p>In my previous journals, I was mainly talking about two projects I am developing in this term. One is named CirSys, which is a problem solver aimed at dc and ac electric circuits appeared on the textbooks using basic theoretical consequences like the <a href="http://www.physics.uoguelph.ca/tutorials/ohm/Q.ohm.KCL.html">Kirchhoff-Current Law (KCL)</a> and the <a href="http://www.physics.uoguelph.ca/tutorials/ohm/Q.ohm.KVL.html">Kirchhoff-Voltage Law (KVL)</a>. The other one is a toy hacker application named Thief, which can download some info from our university's web sites by means of the LWP module. Developing Thief is totally a proof-test, no other impure intentions at all.</p><p>CirSys is a model-builder written in perl 5, using <a href="http://www.maplesoft.com/products/maple/index.aspx">Maple</a> or <a href="http://www.mathworks.com/products/matlab/">MATLAB</a> as the model-solver at the back-end. But what do we mean by "model" here? The term "model" in context of the CirSys project refers to a mathematical model composed of a list of equations, inequalities, and unknown variables. CirSys accepts a description of the user's circuit problem in the form of CIR scripts. CIR is a tiny language designed to describe the structure of electric circuits, much simpler than the netlist syntax used by <a href="http://bwrc.eecs.berkeley.edu/Classes/IcBook/SPICE/">SPICE</a>. CirSys outputs the Maple commands corresponding to the math model and other solving actions with respect to the CIR input. This output is then fed into the mouth of Maple or MATLAB's solving engine by the user, getting the final solutions.</p><p>There is already an incomplete <a href="ftp://202.195.168.25/Simulation/dc-Circuit/dc1/dc1.html">HTML document</a> for dc CirSys, consisting of a lot of samples. And you can also get the <a href="ftp://202.195.168.25/download/cirsys-src.rar">source code</a> from my FTP site.</p><p>I've also added a lot of hyperlinks to my older journals, which may provide a comprehensible context.</p> agent 2005-05-16T06:26:44+00:00 journal Keeping Journals and Compiling Summaries for Textbooks http://use.perl.org/~agent/journal/24667?from=rss <p>=from 2005.5.12.10:45.PM<br>=to<nobr> <wbr></nobr>...5.13.0:30.AM</p><p>I try to keep writing journals with the aim of getting more and more people interested in what I am doing. Unfortunately, I often find myself too busy to summarize one day's work. It really puts me to shame that Autrijus Tang seldom skips any journals though he currently has his hands full in Pugs' development.</p><p>It should be mentioned that Clover asked for the address of my journals the other night. I was feeling honored and happily provided the URL.</p><p>Life becomes less insane since I released Thief 0.03 and made the decision to compile comprehensible summaries for the textbook "Computer Organization" a few days ago. Yeah, it requires a lot of effort and great patience because a wide variety of principles and techniques need to be memorized. Anyway, I believe it's worth my while.</p><p>I had a plan to learn how to program with Pentium instructions and old 8086 assembly language by virtue of <a href="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vcmasm/html/vcoriMicrosoftAssemblerMacroLanguage.asp">Microsoft Macro Assembler (MASM)</a>. This may be the first step towards the splendid milestone of writing an operating system of my own!</p> agent 2005-05-13T12:12:00+00:00 journal Guessing Password and Developing Thief 0.03 http://use.perl.org/~agent/journal/24650?from=rss <p>=from 2005.5.11.10:00.PM<br>=to<nobr> <wbr></nobr>...5.12.8:10.AM</p><p>I'm sorry for the lack of summaries these days. In the first few days of this month, I focused on the development of Thief 0.03.</p><p>At the very beginning, I found that thief.pl couldn't log in the <a href="http://jwcweb.ujs.edu.cn/">Educational Administration System (EAS)</a> even using my own account. However, I could always do this in a web browser, such as IE. Later I utilized the <a href="http://www.mozilla.org/products/">Mozilla</a> extension <a href="http://livehttpheaders.mozdev.org/">"Live HTTP Headers"</a> to intercept the HTTP requests sent by the Mozilla or Firefox browser locating a missing HTTP attribute named "Referrer" in my perl script, whose value is a URL. I wondered why this peculiar attribute was necessary for login and subsequent page requests in the EAS site. I asked Laye about it, but there was no reply.</p><p>I must thank Laye for his implication in one of his e-mails that there was some wonderful tool which could record down the low-level actions performed by the browser. I found "Live HTTP Headers" on the Internet by myself and solved my problem immediately. However, what he actually used was another independent tool named <a href="http://www.effetech.com/sniffer/">EffeTech HTTP Sniffer</a>, which did not rely on any specific browser and had the ability to keep record of all the HTTP requests and responses taking place over the Windows OS. All I could say was "Wow". It was very kind of him to offer me a serial number for this shareware after he learned that I loved this tool so much.</p><p>Yeah, just as what I said in a mail to Laye, HTTP Sniffer is the very tool that we perl hackers are dreaming of every night! The cost of developing a client-side web application will decrease significantly in virtue of the precise GET or POST headers intercepted by this application. For example, in a project I recently developed for <a href="http://www.jmsoft.cn/">the JMSoft Company</a>, I was asked to develop a fully automated tester for a web-based <a href="http://www.jmsoft.cn/products/oa.asp">JMOA system</a>. If I knew HTTP Sniffer at that time, it would be a trivial task to implement such things.</p><p>Having successfully passed the login stage of the EAS site, I tried to modify the student ID encoded in the GET URL at once, hoping to access other user's pages. Unfortunately, I only confirmed the fact that the EAS team had completely fixed this infamous security leak. Bad luck!</p><p>I shortly realized that it would be good idea to "guess" other students' passwords in EAS making full use of the personal info obtained from <a href="http://202.195.165.18:8080/reader/reader_info.php">the database of our university's public library</a>. I wrote seven similar scripts named trypassword.pl, trypassword2.pl,<nobr> <wbr></nobr>..., and trypassword7.pl respectively, to probe EAS's login system using different password patterns. Basing on student IDs, phone numbers, Chinese names, and birthdays extracted from the Personal ID numbers, I luckily cracked in total 738 accounts (well above 12%) of the students in Grade 03.</p><p>I'd like to write down some typical passwords cracked by my Thief project. I wonder if you can figure out the "meaning" yourself:</p><p> &nbsp; &nbsp; &nbsp; &nbsp; 3030202018<br> &nbsp; &nbsp; &nbsp; &nbsp; 19841101, 198467<br> &nbsp; &nbsp; &nbsp; &nbsp; 841206, 05185763926, 6880313, 13706256959<br> &nbsp; &nbsp; &nbsp; &nbsp; zhoujie, lxb, liu<br> &nbsp; &nbsp; &nbsp; &nbsp; zhangmu1985, zhou1985, py1983<br> &nbsp; &nbsp; &nbsp; &nbsp; lxx3482858, 7301572huyewen</p><p>Thanks to the <a href="http://tech.sina.com.cn/c/2001-10-08/6239.html">Imegen.exe</a> program shipped with Windows. I got a GBK-PinYin mapping list within only a few minutes by using the Reverse-Conversion function provided by this tool. The data were actually extracted from the WINPY.MB file used by the Quan Pin IME (Input Method Editor). So we have no reason to worry about the correctness and completeness of our mapping list.</p> agent 2005-05-12T09:45:49+00:00 journal Pinginfo.pl Crashed! http://use.perl.org/~agent/journal/24444?from=rss <p>=from 2005.4.29.2:05.PM<br>=to<nobr> <wbr></nobr>...4.29.2:45.PM</p><p>I really got annoyed when I first ran pinginfo.pl only to find out that <a href="http://jwcweb.ujs.edu.cn/">the Educational Administration System</a> had changed its web interface and thus caused my hacker script to crash completely. Furthermore, the new version seemed to have fixed the well-known "bug" associated with account secrecy. So we might no longer be able to check other people's info.</p><p>I should have written this student-id generator down a little earlier. It couldn't be too late as long as the EAS web site didn't change its face. But now I really have nothing to do with it! Bang!</p><p>Fortunately, the web site of <a href="http://202.195.165.18:8080/reader/reader_info.php">our university's library</a> didn't change its page templates. I promptly migrated pinginfo.pl to fit the user system's web interface, and brought libping.pl into being, which worked perfectly fine. I have already downloaded thousands of user's account info, including the user's real-name, profession, sex, Personal ID number, record of the books borrowed, peccancy history, and so on. My little libping.pl is still crazily capturing more reader's materials at the time of this writing. Oh, no problem, There was till more than 600 MB free room on my disk when I left my machine in the noon.</p><p>How should we process the huge amount of data gained from the web? The current plan is as follows:</p><ul><li> Use HTML parser (written in perl) to filter out the key info from the HTML tables</li><li> Save it in a customized compact format</li><li> Convert this form of data to more specialized and more normative formats, such as CSV or even SQL commands</li><li> Import all the stuff to relational database system, such as <a href="http://office.microsoft.com/en-us/FX010857911033.aspx">MS Access</a> or SQL Server 2000</li></ul><p>The whole process can't be too difficult from the viewpoint of us perl programmers! Right?</p> agent 2005-04-29T09:51:36+00:00 journal