Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

schwern (1528)

schwern
  (email not shown publicly)
http://schwern.net/
AOL IM: MichaelSchwern (Add Buddy, Send Message)
Jabber: schwern@gmail.com

Schwern can destroy CPAN at his whim.

Journal of schwern (1528)

Thursday October 13, 2005
09:41 PM

What is the sound of one head thudding?

[ #27163 ]

Is it not the sound of frustration?

I'm trying to write a module that given a filehandle which is zero or more of zipped, gzipped or pgp (and possibly more in the future) encoded it unpacks the contents of the filehandle. The trick is everything has to be done with filehandles. Filehandles in and filehandles out. At no time should the files exist on disk (security reasons, very sensitive data). The moral equivalent to:

    system("gunzip foo.pgp.gz | ungpg | myprogram");

Except its more complicated because it involves zip programs with multiple files. Ultimately, I want to be able to write this:

    # Typically $input_fh comes from Net::SFTP.
    my %files = Extractor->extract($input_fh);
 
    while( my($filename, $fh) = each %files) {
        _filter_out_sensitive_information_and_write($fh, $filename);
    }

For example, if $input_fh is to a zip file which contains "foo" and "bar" %files will contain the keys "foo" and "bar" and the values will be open filehandles to their contents. The filter reads and munges the contents of the filehandles and outputs to the associated file.

What's kicking my ass is that many of the existing unarchiving modules don't return filehandles. Archive::Zip and Compress::Zlib both return objects. I'm using the less than desirable GnuPG::Interfaces instead of Crypt::GPG simply because Interfaces uses filehandles. I tried tying the objects to a handle but the tying seems to get lost once the handle is copied. I played a bit with wrapping them inside IO::Handle glob objects but that's just too weird even for me.

My final thought is that since I control the filters I can just write an adaptor around each nearly-a-filehandle object which Archive::Zip or Compress::Zlib returns and have the filters use that, but it seems like a lot of work when filehandles would do everything I want.

Am I missing something?

UPDATE:

Yes I am. And it is this.

    assert tied $fh;

That returns false. THIS returns true.

    assert tied *$fh;

*sigh* I was doing the former. It made me think the returned filehandle was not tied. Trouble is, the internal testing system we use is XUnit style. It dies on failure. So I never saw the next test which checks the contents of the filehandle.

Reason #139892 why aborting on failure is EVIL!

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • At no time should the files exist on disk (security reasons, very sensitive data).

    What about using an encrypted filesystem? Or "plain" encrypted files for every step?

    --
    mirod
    • What about using an encrypted filesystem?

      I don't have that level of control over the system. One trick I can use, and is being used elsewhere in the code, is unlinked temp files (open a temp file and unlink it). The file is on disk but no other process can see or read it except my open filehandle. Once my filehandle goes away the file goes away (yes, it might hang around on the disk for a while before its wiped. This is ok.). It won't get backed up.

      But this involves each extraction step to read the f

  • Surely you can make it work using tied filehandles. It won't be dead easy, it won't be the fastest, but at least you can make it behave like filehandles.

    Another option would be to make the readers separate processes, which print the virtual file contents through their STDOUT, which you can then pipe back into your program.
    • Surely you can make it work using tied filehandles. It won't be dead easy, it won't be the fastest, but at least you can make it behave like filehandles.
      BLALHRARHALHBLAHGH!

      I just figured it out. See edited post.