Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

alanhaggai (8497)

alanhaggai
  (email not shown publicly)
http://alanhaggai.org/
Yahoo! ID: alan_haggai (Add User, Send Message)
Jabber: alanhaggai@gmail.com

Journal of alanhaggai (8497)

Sunday June 07, 2009
10:59 PM

Back to coding

I am glad to be back on track after some time. Started to write tests first. It is interesting.

I am rewriting support for named arguments in Archive-Zip (which I had added earlier), so that this time, there will be automated tests for them. Tests were manual earlier, of which I got tired soon due to one-thing-breaking-the-other, and the chain-of-reactions!

Saturday October 25, 2008
07:38 AM

Update on General Purpose Bit Flag bug

bitFlag() sub was modified with the addition of the following conditions:

Desired Compression Level | Speed / Compression
1, 2 | Fast
3, 4, 5, 6, 7 | Normal
8, 9 | Maximum

Thus, Archive::Zip will set the bits exactly the same way that the GNU/Linux zip utility does.

Friday October 24, 2008
01:15 PM

Archive::Zip does not set General Purpose Bit Flag

After analysing few .zip files with the .Zip file parser (http://code.google.com/p/zipparser/), I was able to find out that Archive::Zip does not (for some reason) set the bits of General Purpose Bit Flag.

This means that .zip file members' compression levels will not be stored in the archive.

Let us reproduce it by creating two .zip files named hello-zip.zip (created with the GNU/Linux zip utility) and hello-az.zip created with Archive::Zip for comparison. Both the files contain:

Filename: hello
Content:
Hello world.
Hello world.
Hello world


Let us compress:

[alanhaggai@archer zip]$ zip -9 hello-zip.zip hello
adding: hello (deflated 54%)


#!/usr/bin/perl -w


use strict;
use Archive::Zip qw( :ERROR_CODES :CONSTANTS );


my $zip = Archive::Zip->new;
my $member = $zip->addFile('hello');
$member->desiredCompressionLevel(9);
$zip->writeToFileNamed('hello-az.zip');

Now, let us compare relevant parts of the .Zip files (several of the zip_parser.conf elements were commented out for brevity):

[alanhaggai@archer zip]$ ./zip_parser hello-zip.zip END CENTRAL DIRECTORY RECORD
--------------------------------------------------



LOCAL FILE HEADER
--------------------------------------------------


Compressed_Size ->
18
Compression_Method ->
The file is Deflated
Filename ->
hello
General_Purpose_Bit_Flag ->
Maximum (-exx/-ex) compression option was used
Uncompressed_Size ->
39


CENTRAL DIRECTORY RECORD
--------------------------------------------------


Compressed_Size ->
18
Compression_Method ->
The file is Deflated
Filename ->
hello
General_Purpose_Bit_Flag ->
Maximum (-exx/-ex) compression option was used
Uncompressed_Size ->
39


[alanhaggai@archer zip]$ ./zip_parser hello-az.zip
END CENTRAL DIRECTORY RECORD
--------------------------------------------------



LOCAL FILE HEADER
--------------------------------------------------


Compressed_Size ->
18
Compression_Method ->
The file is Deflated
Filename ->
hello
General_Purpose_Bit_Flag ->
Normal (-en) compression option was used
Uncompressed_Size ->
39


CENTRAL DIRECTORY RECORD
--------------------------------------------------


Compressed_Size ->
18
Compression_Method ->
The file is Deflated
Filename ->
hello
General_Purpose_Bit_Flag ->
Normal (-en) compression option was used
Uncompressed_Size ->
39

Both the files were compressed at level 9 (maximum compression). However, Archive::Zip did not set the `General Purpose Bit Flag', and so, suggest: `Normal (-en) compression option was used'.

In the case of hello-zip.zip produced by the zip utility, the flag's bits were set and suggest: `Maximum (-exx/-ex) compression option was used'.

Saturday October 18, 2008
01:08 AM

Zip Parser

I have written a .Zip file parser. This parser can be (hopefully) used to identify buggy areas of Archive::Zip and possibly other .Zip file writers. Currently, it lacks the capability to parse Zip64 and encrypted .Zip files. I wish to add support for those soon.
Friday October 10, 2008
01:49 AM

Study of Data::ParseBinary

I have been studying Data::ParseBinary for a while. I sincerely hope to be able to build a dissector for .zip file format.
Thursday August 21, 2008
08:40 PM

Work resumed

I have resumed work on the Archive::Zip grant.
Monday August 11, 2008
12:15 PM

Work on Archive::Zip Stopped for Two Weeks

My work on Archive::Zip has been stopped for two weeks as my right-eye contracted a viral infection. I am applying eye-drops and ointments. However, it is difficult to concentrate in work as I have to take frequent naps. Also, the right eye has got blurred in vision due to the disease. The blurriness interferes a lot. So, I thought of stopping the work for two weeks, which is the period within which I hope the eye will cure ( the doctor said so ).

I will post as soon as I get back to work.

Sunday July 06, 2008
12:57 PM

.zip file format

My study of the .zip file format:

Magic bytes

Magic numbers are common in programs across many operating systems. Magic numbers implement strongly typed data and are a form of in-band signaling to the controlling program that reads the data type(s) at program run-time. Many files have such constants that identify the contained data. Detecting such constants in files is a simple and effective way of distinguishing between many file formats and can yield further run-time information.

That is Wikipedia's definition of magic bytes / numbers. .zip magic bytes are:
0x50 0x45 at offset 0x00.
0x50 = P,
and 0x45 = K in ASCII.

PK stands for Phil Katz, the creator of the .zip file format.

Meta Data

  • Optional archive comment
  • Optional comment per entry
  • All kinds of system-specific data ( like file attributes ) can be added per file using so called extra fields

Limitations

  • Internal offset values are 32-bits large, so only files upto 4 GB can be stored
  • No support for extended character sets in file names
Monday June 02, 2008
02:06 PM

Grant proposal accepted!

My proposal `Fixing Bugs in the Archive::Zip Perl Module' has been accepted by The Perl Foundation. Shlomi Fish ( http://www.shlomifish.org ) helped me with the proposal, supported me and also encouraged me.

Thank you very much Shlomi. :-)

I am having examinations at college now. I will start working as soon as the examinations are over.

Tuesday April 22, 2008
12:05 AM

Fixing Bugs in the Archive::Zip Perl Module

Abstract: Perl programs often need to manipulate .zip files. Archive::Zip (http://search.cpan.org/dist/Archive-Zip/) is a Perl module that allows a Perl program to manage Zip archive files without calling an external utility.

The Archive::Zip module, however, has some bugs which prevent it from generating fully-portable .zip files, that are handled correctly by all .zip file readers and manipulators. The project's main aim is to address the outstanding bug reports (http://rt.cpan.org/Public/Dist/Display.html?Name=Archive-Zip), by using pyconstruct (http://pyconstruct.wikispaces.com/), which is a flexible framework for defining dissectors for binary formats in a declarative way.

I have to start working on the project soon. I, most probably, will be having the time to work from the start of May 2008. I am very thrilled to work on it and be part of the Perl community.

Hope the project goes well. I am also applying for `The Perl Foundation Grants'.