use Perl

=from 2005.6.2.7:50.AM
=to ...6.2.8:20.AM

A couple of weeks ago, I downloaded Microsoft's Segmented Executable Linker from the web, and built a trivial DOS program that printed out the perimeter of a circle on Windows 2000 Professional. Unfortunately, DOS interrupts

        int 21h

didn't work at all in the MS-DOS box. So I couldn't use interrupts to output anything to the screen. I suspect that it will work on Win9x platforms anyway.

I wonder why most Chinese universities still teach 16-bit 8086 assembly language, which is completely out-of-date. I find it very difficult to locate an appropriate environment to experiment with the sample codes provided by our textbooks. Indeed, Win9x is now rarely seen on an ordinary PC.

MASM is not flexible enough in my opinion; NASM may be a better choice. I plan to learn both when I take the course Assembly Language Programming next semester.

=from 2005.6.1.9:10.AM
=to ...6.1.9:40.AM

I am delighted to see that I finally finished summarizing Carl's text "Computer Organization" today. The five summarized chapters are

• Chapter2: Machine Instructions and Programs
• Chapter4: Input/Output Organization
• Chapter5: The Memory System
• Chapter6: Arithmetic
• Chapter7: Basic Processing Unit

which have covered almost all the materials contained in the Chinese textbook used in our department. Perl's POD format is used to produce these pretty pages.

I wish to express my thanks to many people who have helped during the preparation of these abstracts. My dad is the very person who first suggested that I should do such projects. And my grandpa always gave me a lot of great suggestions and warm-hearted encouragements. Finally I truly appreciate the support of my schoolmates, such as Ben (Dai Xiaoke) and Sal (Zhong Weixiang). Thank you all!

=from 2005.5.31.8:00.AM
=to ...5.31.8:45.AM

In hardware design, it is sometimes desirable to depict the logic circuit diagrams in a constructive manner, just like the way we build a complex data structure using a general-purpose programming language. For example, we might wish to express a cascaded connection of 32 full adders, which is known as a 32-bit carry-ripple adder, by the following pseudo code (I borrowed Perl6 syntax here to illustrate the idea):

        my @adders;
        for (0..31) -> $i {
                push @adders, FA.new(
                        name => "cell$i",
                        X => "x[$i]", Y => "y[$i]", Cin => "c[$i]",
                        S => "s[$i]", Cout => "c[" ~ ($i+1) ~ "]"
                );
        }

Unfortunately, so far as I know, earlier versions of Verilog HDL lack the ability to construct such things effectively, so a lot of typing is necessary if we use pure Verilog.

Is there a way to use Perl6 syntax directly as an HDL? The answer is certainly "yes". The simplest solution may be as follows:

• Write Perl6 codes just like the example we gave above using a predefined class library
• Run this script and generate corresponding Verilog codes.

In the preceding example, we might write a Perl6 program named adder32.p6, which probably prints out something like the following:

        module adder32(x, y, c_in, c_out, s);
                input [31:0] x, y;
                input c_in;
                output c_out;
                output [31:0] s;
                wire [32:0] c;
                assign c_out = c[32];
                assign c[0] = c_in;

                FA cell0 (.X(x[0]), .Y(y[0]), .Cin(c[0]), .S(s[0]), .Cout(c[1]));
                FA cell1 (.X(x[1]), .Y(y[1]), .Cin(c[1]), .S(s[1]), .Cout(c[2]));
                FA cell2 (.X(x[2]), .Y(y[2]), .Cin(c[2]), .S(s[2]), .Cout(c[3])); // an awful lot omitted here...
                FA cell30 (.X(x[30]), .Y(y[30]), .Cin(c[30]), .S(s[30]), .Cout(c[31]));
                FA cell31 (.X(x[31]), .Y(y[31]), .Cin(c[31]), .S(s[31]), .Cout(c[32]));
        endmodule

The code generation details can be encapsulated into the basic Perl6 library, thus becoming transparent to EDA designers. Some template-driven code generators, such as Perl Template Toolkit, are equally helpful.

It is worth noting that the generate construct provided by VHDL and Verilog-2001 can build the 32-bit carry-ripple adder in our example by virtue of loops as well. However, at this moment, they seem to be only applicable to module instantiations. So we can expect the code generation solution based on Perl to have a great future.

=from 2005.5.30.8:30.AM
=to ...5.30.9:15.AM

There are some cases in which we only want to simulate some abstract algorithms without considering the hardware implementation issues. In such situations, Tesla is not very practical. On the other hand, we can perfectly achieve our goals by means of pure "software" methods. For instance, a simple algorithm that converts an unsigned binary number to its octal form can be implemented by the following Haskell codes:

        bin2oct :: [Int] -> [Int]
        bin2oct xs = bin2oct' (replicate (3-r) 0 ++ xs)
                where r = (length xs) `mod` 3

        bin2oct' :: [Int] -> [Int]
        bin2oct' [] = []
        bin2oct' (x:y:z:xs) = [x*4 + y*2 + z] ++ (bin2oct' xs)

As another example, consider the problem of transforming a decimal number to its binary form:

        dec2bin :: Int -> [Int]
        dec2bin 1 = [1]
        dec2bin 0 = [0]
        dec2bin x = dec2bin (x `div` 2) ++ [x `mod` 2]

Perl6 can do this job equally well:

        sub dec2bin(Int $x) {
                return $x if $x == 1|0;
                dec2bin(int $x/2), $x % 2;
        }

Functional programmers tend to think in a recursive way, I think.

Instead of verify the algorithms involved in logic design "electrically" in Tesla, we can play with such stuff in a complete mathematical sense and take advantage of various advanced programming techniques, such as FP. A lot of joy can be expected then.

=from 2005.5.30.7:45.AM
=to ...5.30.8:30.AM

Some interesting ideas jumped into my head while I was reading Carl's "Computer Organization" these days.

I suppose it is possible to translate Verilog's behavioral codes, such as the "always" blocks directly to Perl codes that implement *built-in* circuit elements, just like the AND gates. Such transformation can be mechanical, since Verilog and Perl share many control-flow structures.

Obviously, Tesla will no longer be a pure gate-level simulator after some built-in blackbox components are introduced. These blackboxes provide sequential logic behaviors to the outside world, but are implemented by pure "software" methods, not logic circuits!

Tesla's built-in component repertoire will be extensible. We can have complex component types like "Register" and "Counter" that are implemented by ordinary Perl codes. This feature not only eases the pain of writing a Verilog-to-Tesla compiler, but also makes Tesla extremely powerful in high-level simulations.

A further step can be taken by changing the Tesla's built-in signals from individual wires to multi-channel buses. Oh, no! I think it is more appropriate to use inheritance here. Let the Wire and Bus classes both be derived from an abstract class named Signal. Therefore, Wires and Buses will be distinguished without breaking the existing code that uses Signals.

Porting the whole Tesla system to Perl6 is even more attractive. Pugs now supports sufficient Perl6 features; this plan thus becomes quite feasible.

=from 2005.5.23.7:50.AM
=to ...5.23.9:05.AM

The two weeks' efforts on summarizing Carl's "Computer Organization" have been proved fruitful. Now I've got some insight into the internals of computer systems. Maybe it's time for me to *make* a quite complete computer of my own. This idea seems to be a little insane at the first glance; however, it has been one of my goals since high school.

The current plan is as follows:

1. Use standard Verilog HDL to implement the processor, the system bus, RAMs, caches, DMA controllers and some basic I/O device interfaces.
2. Test the Verilog design in commercial simulators and synthesizers, such as Altera Quartus II and ModelSim.
3. Implement a simple Verilog compiler that can convert the Verilog source to my Tesla HDL codes.
4. Perform real-time simulation by virtue of my Tesla simulator developed in last term.
5. Program the well-tested design down to the devices, such as FPGA, finishing the whole process.

The step of developing a Verilog compiler may be unnecessary. Transforming the netlist generated by other people's Verilog compiler to Tesla code seems to be much more straightforward. Fortunately, the meta netlist (.vqm file) output by Quartus II is very similar to Verilog syntax, though a little odd. I wonder if there are any specifications for this format.

In fact, if there's a real-time simulator already available on the web, the steps 3 and 4 can be completely bypassed. I'm guessing the FLI (Foreign Language Interfaces) provided by ModelSim may be the right place to look at.

However, I still believe that it is doubtless more interesting and more challenging to simulate my design in my own simulator, Tesla, because all the boring principles and algorithms I learned from the course on Logic Design will get an opportunity to be applied to real lift. Yeah, every student has a dream to act as a "practicing theorist".

Verilog is very concise compared to VHDL, so it's an ideal language to describe large designs. I can't understand why our university chose VHDL in all our hardware courses. Weird.

=from 2005.5.16.1:40.PM
=to ...5.16.2:25.PM

Some kind people pointed to me that my journals seriously lack necessary backgrounds and explanation of mysterious terms, which leads to the terrible fact that no one can understand them even a bit. I feel very sorry and regretful.

Above all, I suppose it's important to introduce myself a little. I'm a Chinese student majored in Computer Science in Jiangsu University, who loves perl and gets very interested in Autrijus' journals on this site. I have the honor to be able to upload my own journals too, because there are a lot of great people here.

In my previous journals, I was mainly talking about two projects I am developing in this term. One is named CirSys, which is a problem solver aimed at dc and ac electric circuits appeared on the textbooks using basic theoretical consequences like the Kirchhoff-Current Law (KCL) and the Kirchhoff-Voltage Law (KVL). The other one is a toy hacker application named Thief, which can download some info from our university's web sites by means of the LWP module. Developing Thief is totally a proof-test, no other impure intentions at all.

CirSys is a model-builder written in perl 5, using Maple or MATLAB as the model-solver at the back-end. But what do we mean by "model" here? The term "model" in context of the CirSys project refers to a mathematical model composed of a list of equations, inequalities, and unknown variables. CirSys accepts a description of the user's circuit problem in the form of CIR scripts. CIR is a tiny language designed to describe the structure of electric circuits, much simpler than the netlist syntax used by SPICE. CirSys outputs the Maple commands corresponding to the math model and other solving actions with respect to the CIR input. This output is then fed into the mouth of Maple or MATLAB's solving engine by the user, getting the final solutions.

There is already an incomplete HTML document for dc CirSys, consisting of a lot of samples. And you can also get the source code from my FTP site.

I've also added a lot of hyperlinks to my older journals, which may provide a comprehensible context.

=from 2005.5.12.10:45.PM
=to ...5.13.0:30.AM

I try to keep writing journals with the aim of getting more and more people interested in what I am doing. Unfortunately, I often find myself too busy to summarize one day's work. It really puts me to shame that Autrijus Tang seldom skips any journals though he currently has his hands full in Pugs' development.

It should be mentioned that Clover asked for the address of my journals the other night. I was feeling honored and happily provided the URL.

Life becomes less insane since I released Thief 0.03 and made the decision to compile comprehensible summaries for the textbook "Computer Organization" a few days ago. Yeah, it requires a lot of effort and great patience because a wide variety of principles and techniques need to be memorized. Anyway, I believe it's worth my while.

I had a plan to learn how to program with Pentium instructions and old 8086 assembly language by virtue of Microsoft Macro Assembler (MASM). This may be the first step towards the splendid milestone of writing an operating system of my own!

=from 2005.5.11.10:00.PM
=to ...5.12.8:10.AM

I'm sorry for the lack of summaries these days. In the first few days of this month, I focused on the development of Thief 0.03.

At the very beginning, I found that thief.pl couldn't log in the Educational Administration System (EAS) even using my own account. However, I could always do this in a web browser, such as IE. Later I utilized the Mozilla extension "Live HTTP Headers" to intercept the HTTP requests sent by the Mozilla or Firefox browser locating a missing HTTP attribute named "Referrer" in my perl script, whose value is a URL. I wondered why this peculiar attribute was necessary for login and subsequent page requests in the EAS site. I asked Laye about it, but there was no reply.

I must thank Laye for his implication in one of his e-mails that there was some wonderful tool which could record down the low-level actions performed by the browser. I found "Live HTTP Headers" on the Internet by myself and solved my problem immediately. However, what he actually used was another independent tool named EffeTech HTTP Sniffer, which did not rely on any specific browser and had the ability to keep record of all the HTTP requests and responses taking place over the Windows OS. All I could say was "Wow". It was very kind of him to offer me a serial number for this shareware after he learned that I loved this tool so much.

Yeah, just as what I said in a mail to Laye, HTTP Sniffer is the very tool that we perl hackers are dreaming of every night! The cost of developing a client-side web application will decrease significantly in virtue of the precise GET or POST headers intercepted by this application. For example, in a project I recently developed for the JMSoft Company, I was asked to develop a fully automated tester for a web-based JMOA system. If I knew HTTP Sniffer at that time, it would be a trivial task to implement such things.

Having successfully passed the login stage of the EAS site, I tried to modify the student ID encoded in the GET URL at once, hoping to access other user's pages. Unfortunately, I only confirmed the fact that the EAS team had completely fixed this infamous security leak. Bad luck!

I shortly realized that it would be good idea to "guess" other students' passwords in EAS making full use of the personal info obtained from the database of our university's public library. I wrote seven similar scripts named trypassword.pl, trypassword2.pl, ..., and trypassword7.pl respectively, to probe EAS's login system using different password patterns. Basing on student IDs, phone numbers, Chinese names, and birthdays extracted from the Personal ID numbers, I luckily cracked in total 738 accounts (well above 12%) of the students in Grade 03.

I'd like to write down some typical passwords cracked by my Thief project. I wonder if you can figure out the "meaning" yourself:

        3030202018
        19841101, 198467
        841206, 05185763926, 6880313, 13706256959
        zhoujie, lxb, liu
        zhangmu1985, zhou1985, py1983
        lxx3482858, 7301572huyewen

Thanks to the Imegen.exe program shipped with Windows. I got a GBK-PinYin mapping list within only a few minutes by using the Reverse-Conversion function provided by this tool. The data were actually extracted from the WINPY.MB file used by the Quan Pin IME (Input Method Editor). So we have no reason to worry about the correctness and completeness of our mapping list.

=from 2005.4.29.2:05.PM
=to ...4.29.2:45.PM

I really got annoyed when I first ran pinginfo.pl only to find out that the Educational Administration System had changed its web interface and thus caused my hacker script to crash completely. Furthermore, the new version seemed to have fixed the well-known "bug" associated with account secrecy. So we might no longer be able to check other people's info.

I should have written this student-id generator down a little earlier. It couldn't be too late as long as the EAS web site didn't change its face. But now I really have nothing to do with it! Bang!

Fortunately, the web site of our university's library didn't change its page templates. I promptly migrated pinginfo.pl to fit the user system's web interface, and brought libping.pl into being, which worked perfectly fine. I have already downloaded thousands of user's account info, including the user's real-name, profession, sex, Personal ID number, record of the books borrowed, peccancy history, and so on. My little libping.pl is still crazily capturing more reader's materials at the time of this writing. Oh, no problem, There was till more than 600 MB free room on my disk when I left my machine in the noon.

How should we process the huge amount of data gained from the web? The current plan is as follows:

• Use HTML parser (written in perl) to filter out the key info from the HTML tables
• Save it in a customized compact format
• Convert this form of data to more specialized and more normative formats, such as CSV or even SQL commands
• Import all the stuff to relational database system, such as MS Access or SQL Server 2000

The whole process can't be too difficult from the viewpoint of us perl programmers! Right?