alanhaggai's Journal

Back to coding

alanhaggai — 2009-06-08T03:59:43+00:00

I am glad to be back on track after some time. Started to write tests first. It is interesting.

I am rewriting support for named arguments in Archive-Zip (which I had added earlier), so that this time, there will be automated tests for them. Tests were manual earlier, of which I got tired soon due to one-thing-breaking-the-other, and the chain-of-reactions!

Update on General Purpose Bit Flag bug

alanhaggai — 2008-10-25T12:38:20+00:00

bitFlag() sub was modified with the addition of the following conditions:

Desired Compression Level | Speed / Compression
1, 2 | Fast
3, 4, 5, 6, 7 | Normal
8, 9 | Maximum

Thus, Archive::Zip will set the bits exactly the same way that the GNU/Linux zip utility does.

Archive::Zip does not set General Purpose Bit Flag

alanhaggai — 2008-10-24T18:15:55+00:00

After analysing few .zip files with the .Zip file parser (http://code.google.com/p/zipparser/), I was able to find out that Archive::Zip does not (for some reason) set the bits of General Purpose Bit Flag.

This means that .zip file members' compression levels will not be stored in the archive.

Let us reproduce it by creating two .zip files named hello-zip.zip (created with the GNU/Linux zip utility) and hello-az.zip created with Archive::Zip for comparison. Both the files contain:

Filename: hello Content: Hello world. Hello world. Hello world

Let us compress:

[alanhaggai@archer zip]$ zip -9 hello-zip.zip hello adding: hello (deflated 54%) #!/usr/bin/perl -w use strict; use Archive::Zip qw( :ERROR_CODES :CONSTANTS ); my $zip = Archive::Zip->new; my $member = $zip->addFile('hello'); $member->desiredCompressionLevel(9); $zip->writeToFileNamed('hello-az.zip');

Now, let us compare relevant parts of the .Zip files (several of the zip_parser.conf elements were commented out for brevity):

[alanhaggai@archer zip]$ ./zip_parser hello-zip.zip END CENTRAL DIRECTORY RECORD -------------------------------------------------- LOCAL FILE HEADER -------------------------------------------------- Compressed_Size -> 18 Compression_Method -> The file is Deflated Filename -> hello General_Purpose_Bit_Flag -> Maximum (-exx/-ex) compression option was used Uncompressed_Size -> 39 CENTRAL DIRECTORY RECORD -------------------------------------------------- Compressed_Size -> 18 Compression_Method -> The file is Deflated Filename -> hello General_Purpose_Bit_Flag -> Maximum (-exx/-ex) compression option was used Uncompressed_Size -> 39 [alanhaggai@archer zip]$ ./zip_parser hello-az.zip END CENTRAL DIRECTORY RECORD -------------------------------------------------- LOCAL FILE HEADER -------------------------------------------------- Compressed_Size -> 18 Compression_Method -> The file is Deflated Filename -> hello General_Purpose_Bit_Flag -> Normal (-en) compression option was used Uncompressed_Size -> 39 CENTRAL DIRECTORY RECORD -------------------------------------------------- Compressed_Size -> 18 Compression_Method -> The file is Deflated Filename -> hello General_Purpose_Bit_Flag -> Normal (-en) compression option was used Uncompressed_Size -> 39

Both the files were compressed at level 9 (maximum compression). However, Archive::Zip did not set the `General Purpose Bit Flag', and so, suggest: `Normal (-en) compression option was used'.

In the case of hello-zip.zip produced by the zip utility, the flag's bits were set and suggest: `Maximum (-exx/-ex) compression option was used'.

Zip Parser

alanhaggai — 2008-10-18T06:08:41+00:00

I have written a .Zip file parser. This parser can be (hopefully) used to identify buggy areas of Archive::Zip and possibly other .Zip file writers. Currently, it lacks the capability to parse Zip64 and encrypted .Zip files. I wish to add support for those soon.

Study of Data::ParseBinary

alanhaggai — 2008-10-10T06:49:45+00:00

I have been studying Data::ParseBinary for a while. I sincerely hope to be able to build a dissector for .zip file format.

Work resumed

alanhaggai — 2008-08-22T01:40:34+00:00

I have resumed work on the Archive::Zip grant.

Work on Archive::Zip Stopped for Two Weeks

alanhaggai — 2008-08-11T17:15:50+00:00

My work on Archive::Zip has been stopped for two weeks as my right-eye contracted a viral infection. I am applying eye-drops and ointments. However, it is difficult to concentrate in work as I have to take frequent naps. Also, the right eye has got blurred in vision due to the disease. The blurriness interferes a lot. So, I thought of stopping the work for two weeks, which is the period within which I hope the eye will cure ( the doctor said so ).

I will post as soon as I get back to work.

.zip file format

alanhaggai — 2008-07-06T17:57:43+00:00

My study of the .zip file format:

Magic bytes

Magic numbers are common in programs across many operating systems. Magic numbers implement strongly typed data and are a form of in-band signaling to the controlling program that reads the data type(s) at program run-time. Many files have such constants that identify the contained data. Detecting such constants in files is a simple and effective way of distinguishing between many file formats and can yield further run-time information.

That is Wikipedia's definition of magic bytes / numbers. .zip magic bytes are:
0x50 0x45 at offset 0x00.
0x50 = P,
and 0x45 = K in ASCII.

PK stands for Phil Katz, the creator of the .zip file format.

Meta Data

Optional archive comment
Optional comment per entry
All kinds of system-specific data ( like file attributes ) can be added per file using so called extra fields

Limitations

Internal offset values are 32-bits large, so only files upto 4 GB can be stored
No support for extended character sets in file names

Grant proposal accepted!

alanhaggai — 2008-06-02T19:06:18+00:00

My proposal `Fixing Bugs in the Archive::Zip Perl Module' has been accepted by The Perl Foundation. Shlomi Fish ( http://www.shlomifish.org ) helped me with the proposal, supported me and also encouraged me.

Thank you very much Shlomi. :-)

I am having examinations at college now. I will start working as soon as the examinations are over.

Fixing Bugs in the Archive::Zip Perl Module

alanhaggai — 2008-04-22T05:05:41+00:00

Abstract: Perl programs often need to manipulate .zip files. Archive::Zip (http://search.cpan.org/dist/Archive-Zip/) is a Perl module that allows a Perl program to manage Zip archive files without calling an external utility.

The Archive::Zip module, however, has some bugs which prevent it from generating fully-portable .zip files, that are handled correctly by all .zip file readers and manipulators. The project's main aim is to address the outstanding bug reports (http://rt.cpan.org/Public/Dist/Display.html?Name=Archive-Zip), by using pyconstruct (http://pyconstruct.wikispaces.com/), which is a flexible framework for defining dissectors for binary formats in a declarative way.

I have to start working on the project soon. I, most probably, will be having the time to work from the start of May 2008. I am very thrilled to work on it and be part of the Perl community.

Hope the project goes well. I am also applying for `The Perl Foundation Grants'.

First write up

alanhaggai — 2008-04-06T05:00:43+00:00

This is my first write up at `use Perl'. Hope to write more soon.