Taskboy.com and Pudge.net are on the same server. This machine has an unusually apache set up: one apache server answers on port 80 and uses name-based virtual servers to handle Taskboy and Pudge.net. The virtual server conf actually palms the requests off to another apache process (configured with mod_perl) running on local host. Responses are then proxied back through the first apache process. Sound confusing? It is. Now, let's throw XML-RPC into the mix.
mod_perl, Apache and XML::Parser don't mix without special care. Why? Because XML::Parser and Apache each have their own expat libraries. Since mod_perl jams these two things into the same process space, seg faults happen whenever a mod_perl script (like an XML-RPC client) tries to use XML::Parser.
The solution is to compile Apache without expat, which isn't hard.
The server config mentioned above made this problem really obscure. I was getting 502 proxy errors from the server on port 80. This was bugging me out until I check its error_log and noticed the seg faulting children. Then it clicked.
I'll resolve this problem tomorrow (there are also two versions of Perl on the system, so I need to be awake when compiling).