[T]here's a lot of envy -- perhaps animosity -- surrounding Perl because it's ugly, has all of these weird hacks, isn't "pure," and yet is enormously successful. At the same time, the Smalltalk and Scheme guys were trying to explain to the other camp that they solved our sorts of problems decades ago, so we should really listen to them.
One reason why the academic languages never take off is because they are tied to a research agenda. When the research folks were worried about virtual machines, garbage collection, JIT compilation, functional programming, type safety, generic programming, objects, patterns and whatnot, the average workaday programmer was worried about bytesexing, 16-/32-bit integer conversions, and just getting a program to fit in a PC with crazy small memory footprints. It's only within the last few years that machines have gotten fast enough that working programmers can afford the indirection that many of these research languages bring with them.
It's obvious now that the academics were more focused on building the platform that people would use 5-, 10- and 15- years hence than the current limitations of whatever hardware came out last month. (This has been Alan Kay's mantra for ~40 years.) So it's no surprise now that hardware has caught up to where the academics planned, they're pissed off the rest of the world is still programming much as we did when we were using 286s and 68000s with 1MB of memory.
On the flip side, a lot of the kinds of problems that were interesting to academia decades ago were trivial compared to the kinds of problems working programmers faced, or at least they didn't have the same kinds of constraints. So it was OK to use recursion, VMs and garbage collection in 1980 with "research languages", even if those languages were considered didn't scale up to solve the problems of the day. (Or, rather, didn't scale up when used by average programmers instead of great hackers).
By and large, the features that the academic community pioneered are being adopted in bits and pieces. Perl got reference counting memory management before it got closures and objects. That trend will likely continue into the future; Parrot is being built with JIT in mind from day zero. We're catching up with the past, albeit slowly. (Guy Steele doesn't look at Java as a sucky language; he looks at it as slipping about half of Lisp into the minds of C/C++ programmers that were otherwise hostile to the lessons of Lisp.)
Finally, there is this issue as to why a popular, impure language like Perl is so successful, where pure, academic languages like Scheme never get widespread use. My take is that there is a schism between the purity of the ivory tower, and the reality of the everyday world. Perl is a dirty language because it's problem domain is dirty. Better to adapt to the problem than impose a simulated simplicity on it. And that's a bitter pill for academics to swallow. I've looked and looked and then looked some more, but the applicative order Y-Combinator and ((call/cc call/cc) (call/cc call/cc)) in Scheme still makes no sense to me. But when I see a gnarly piece of Perl syntax, I recognize it as a regular expression that simply and concisely captures exactly what I mean. And I'll take a working regex over an elegant definition of recursion any day of the week.