See, there was a hacked version of expat that didn't do entity translation. When you're doing a pass-through parse, sometimes you really do want the entities to remain as they are. For example, it's bad to convert numeric character entities coming from the database as you parse them, right before you spit them out to the browser. The browser can deal with tokens like �, but it might not be able to recognize a multibyte Unicode character as a multibyte Unicode character. So keeping the entitites unparsed is sometimes both the right thing to do and the tasty way to do it[*].
So now there's this situation where we have dueling copies of expat integrated into the project. One of the things that depends on expat will be broken, and as luck would have it, the thing that broke is the more important subsystem that has worked reliably for ~5 years (until I integrated that XSLT engine). As expected, my changes broke the build, so I rushed out some hacks to prevent the offending parts of expat to stop colliding. (Basically, some #define Token new_Token hacks for all of the functions that need two definitions.)
Do a clean checkout and a full build, and everything works. Do a make clean and make all on the offending component, and everything works.
Fast forward a couple of months, and things are acting wonky. The other component is now broken in some hard-to-describe circumstances. The only way to fix it is to build the entire project, including a broken XSLT component, then make clean && make install on the XSLT component to get everything working. Very strange indeed.
After about an hour of hacking around to understand how something that simply shouldn't be possible was happening, I found the cause of the problem. The C files that contained the definitions of the redefined symbols were updated from CVS and recompiled. But the C files that contained the invocations were never changed, so they were never recompiled, and the
And here was the root cause of the problem. In development, we do a 'make clean' all the time. But in testing, they do a simple 'cvs update && make' instead. So the stale
Funny. After years of using a real language, I had nearly forgotten that these kinds of partially-stale configurations were possible.
*: Thank you Quaker Oats for one of the most memorable ad campaigns from the 1980s.