Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

jesse (2531)

jesse
  (email not shown publicly)

Journal of jesse (2531)

Thursday February 14, 2008
04:12 PM

Shipwright - our new code distribution system

[ #35658 ]

Like any opensource software shop, we distribute the source code
for our software. How's that for the obvious statement of the decade?

Actually, I can beat it. It's a pain in the neck for end users to
collect and install all of the dependencies for our software.

And now I'm going to one-up myself again. Customers often build
our software against untested versions of libraries, making debugging
'frustrating.'

RT, our flagship product, depends on 124 separate packages, 114 of
them CPAN libraries. While CPAN has pretty good support for recursively
installing dependencies, it's not perfect and can be time consuming
and confusing for end users. And when it doesn't work right, as can
happen when a module author makes an incompatible change, debugging
requires a wizard.

We've built a new source (and binary) packaging system called
Shipwright. Shipwright allows you to track all of your package's
dependencies in a version control repository like SVN or SVK as
well as build order and build instructions.

It comes with tools for importing Perl modules, C libraries and
other dependencies from CPAN, upstream version control repositories
and tarballs. When it can discover dependency information (as it
can for Perl modules), Shipwright will automatically import the
current versions of all listed dependencies if the repository doesn't
already contain sufficient versions.

Shipwright can automatically set up build instructions for projects
using autoconf as well as projects using Perl's MakeMaker,
Module::Install and Module::Build mechanisms. If necessary, you
can customize the build instructions and dependency ordering after
you import a package.

When it's time to ship your project to your end users, all you need
to do is take a snapshot of your Shipwright repository and send it
out. To build your project, an end user just needs to run
"./bin/shipwright-build". If they want to, your users can choose
to skip certain dependencies (if they want to use system versions)
or specify an installation path. By default, Shipwright builds
fully relocatable binary distributions into a temporary directory
and users can move them into place or copy them to any number of
hosts with the same base system libraries -- Shipwright automatically
wraps all your binaries and scripts so that they can find the
Shipwright versions of their dependencies, no matter where you move
the installed distribution. Shipwright also comes with sh and tcsh
scripts you can 'source' to add an installed distribution's libraries
to your current environment.

At Best Practical, we've configured most of our Shipwright distributions
to bundle everything above libc. Perl, Subversion and GD are just
some of the packages we distribute as part of these relocatable
builds. We now have a single-command tool to build, link and install
Subversion, SVK and all their dependencies (including APR, Neon,
Perl and a bunch of others). With a single command, we downloaded,
extracted, checked in and tested Tatsuhiko Miyagawa's Plagger Feed
Aggregator and all 134 perl modules it depends on. After that, a
single command built a full binary distribution of Plagger, ready
for deployment on any Mac OS X system.

I'm quite proud to release Shipwright 1.0 today. I designed Shipwright
with sunnavy, one of the hackers here at Best Practical. He's
responsible for almost all of the project's implementation to date,
though we're eager to have additional developers join us going forward.

If you're interested in Shipwright, download 1.0 from
http://search.cpan.org/dist/Shipwright and subscribe to the Shipwright
mailing list by emailing shipwright-subscribe@lists.bestpractical.com

You can always get the latest version of the Shipwright source code
with this command:

svn co svn://svn.bestpractical.com/Shipwright/trunk

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • This sounds awesome, I shall experiment.
  • There's a remarkable lack of documentation in the CPAN release for a v1.0. Your journal post contains dramatically more information than the POD.

    It sounds like a great idea and perhaps a nice step beyond PAR, but it looks like I'll have to go read source code to figure out how to use it.
  • Well, it's very much a first release. We picked 1.0 instead of 0.1 to indicate that we consider it to be largely feature complete and to, well, work. There's a tutorial in the docs/ directory, but we'd really appreciate help improving the state of documentation.
  • I think we may use it for our deployment of the 4e OpenAPI servers :)