A quick search by one of our managers led to the ultimate proof why we should not use SOAP::Lite at all. The report Latency Performance of SOAP Implementations shows that SOAP::Lite is far the slowest SOAP implementation on the market.
While probably they did not use mod_perl when doing the experiments for this article, it looks really bad that SOAP::Lite came out 10 times slower than other solutions.
Returning to our case: For now it seems we have a 10 fold increase in time when transfering the same data via SOAP instead of reading from an NFS share, even though we are using mod_perl and the data is already in the memory of the SOAP server.
Maybe buying bigger hardware will solve this problem as well.