"An object persistence tool?" You ask, "There's hundreds of 'em, aren't you just reinventing the wheel?"
Well, yes and no. We've looked at a whole heap of tools that are available and none of them fit the bill for what we want. Our design goals have been, in no particular order (I've numbered the list so I can refer back):
And do you know, it's starting to come together. We're using James' trick with Data::Dumper and
bless as a way of walking object trees (which hurts your head the first few times you look at it, but which gets rid of a whole pile of treewalking code). Once you get your head wrapped 'round how it works, it all starts to seem remarkably simple...
So, what do we have working. Part 1 is really, really hard in the general case. For now we can reliably store most 'pure perl' objects that aren't based on CODE refs. Storage is more efficient (ie: We can do deferred fetching tricks) if the classes support an
_oid method. Things fall apart (badly) in the case where we have XS classes that use the blessed reference as a key to some perl inaccessible data structure out in C space. The general rule of thumb is that, if an object has state that Data::Dumper can't see, then neither can we. It'd be really cool if Data::Dumper were a little more 'hooky', and checked for, say, '_Dumper' method on all classes and could hand off the responsibilitly for serializing to perl code to those objects that wanted it...
Point 2 is pretty much covered (modulo what was discussed in point 1). Objects can help by providing an
_oid method, but we can generally cope if they don't.
Point 3, yup, we do that, we have Pixie::Store::* objects for DBI, BerkeleyDB and Memory (that last is a hangover from early algorithm testing...). The DBI store uses the very lovely DBIx::AnyDBD to cope with database inconsistencies. (If you've not looked at DBIx::AnyDBD and you do anything with databases then I strongly recommend you take a look. Matt Sergeant is a genius.) Currently the only database we have a 'specific' module for is MySQL, but that's just to get round the lack of real transactions by using their atomic
Point 4. Yup, we do that. We not only don't require a schema, we wouldn't know what to do with one if we had it.
Point 5. Yup. It's simple all right.
Point 6. Yup. If an object contains other objects that are stored seperately (objects that have oids) then we don't pull them in immediately, but instead provide a proxy.
Point 7. Sort of. A lot of this falls out as a result of deferred loading. When we store an object we store all the 'real' objects that are 'reachable' from it, but we stop when we reach a proxy object. We could probably be more efficient, but I've not had a hard think about pathological cases yet.
We've now reached the point where I'm thinking of deploying it for an internal application. There's still issues though. Here's a few that are making me think at the moment.
"So, where can I find this paragon of perl persistency? And what's it called?"
Well, it's called 'Pixie' (ask Acme), and right now you can't have it. Some of the current tests are adapted from Tangram and are only distributable under the GPL, and we want Pixie to be destributed under the same terms as Perl. So, we need to either get permission to distribute them under a dual license, or get rid of them entirely.
I'll be off to ask for permission as soon as I've hit 'save' on this form, so we should know one way or the other soon.