Once you start using the P5I script support in pip, one of the next problems you hit fairly quickly is that you have to keep all the different files seperate, you can't just pack them up into a single file.
Well now you can! (sorta)
The new pip 0.10 release introduces experimental support for "P5Z" archives.
A P5Z archive is a gzipped tarball (tgz) that contains a default.p5i script at the root of the tarball (or one level down if the tarball contains a single directory at the root, as with CPAN tarballs).
When you call the following...
> pip some/program.p5z
Think of it as sort of like a PAR file, except that instead of binary packages, you can bundle together a whole bunch of arbitrary source distributions and install them in order, including any of the normal features pip support (URIs for example).
Unfortunately, at the present time you need to build the p5z files by hand.
However, coming up next will be support for taking any arbitrary P5I file and converting it into a P5Z archive, slurping in distributions from URLs and rewriting the P5I file on the fly.
Beyond that, it should be possibly to add a method for you to "record" an entire series of CPAN installations, and then grab all the tarballs that got installed (in the right order) and assemble a P5Z archive out of them, so you can replicate the same installation offline.
Exciting times ahead for custom module installation!