Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Purdy (2383)

Purdy
  reversethis-{ofni.ydrup} {ta} {nosaj}
http://purdy.info/
AOL IM: EmeraldWarp (Add Buddy, Send Message)
Yahoo! ID: jpurdy2 (Add User, Send Message)

Bleh - not feeling creative right now. You can check me out on PerlMonks [perlmonks.org].

Journal of Purdy (2383)

Thursday September 28, 2006
08:45 AM

Free lesson for you...

[ #31150 ]

I have been working pretty hard on a work project for about a month now (off & on and for the last week, mostly on) and I've come to a realization that perhaps most of you already have.

For the impatient, here's the lesson up-front:

When tasked with importing data from an external source, consider importing it into a separate db/table and then building/extending the necessary functionality on top of that (versus importing the data right into your existing data).

We run circulation data for two magazines, both on systems we built ourselves. It was decided to outsource circulation of one of these to another vendor. Several months later, I was asked to import their data for some functionality that the vendor doesn't provide. As it turns out, there are some similarities between the schemas of their data and ours, but mostly differences. A big one being when a user renews their subscription, I treat that as a separate subscription and they treat that as an extension of their existing subscription.

Anyway, like I said, I've been working hard and feel like I'm currently at 85 or 90% completion, but to nail the final non-conformities would require user-specific code, which makes my eyes bleed when I'm already facing some messed-up code (5 or 6 main IF branches and a few places that could be re-factored).

Perhaps this is when I should move to logic programming vs. functional programming, but I still don't have my head around that one. :(

Perhaps also, I'm just at a point that all programmers reach when they've invested too much time in a project and question "WHY?" and I should just buckle down and knock out the remaining 10-15% cases.

Cheers,

Jason

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • When tasked with importing data from an external source, consider importing it into a separate db/table and then building/extending the necessary functionality on top of that (versus importing the data right into your existing data).

    Data import is a large part of what we do. We collect property appraisal data, tax collection data from banks.... the list goes on. There are so many reasons this lesson is valid. If nothing else, if there is a bug in the import, you don't have to go through the hassle of act