Michael Schwern created a detailed proposal for a project called CPANTS. I have long felt that CPAN would be benefit from some kind of rating and comment system, and this sounds like a great solution.
Personally, I'm in. I'll help.
Read it and see what you think.
update 8/6/2003: There's also an aspiring search interface that has sprung up based on CPANTS.