Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

Aristotle (5147)

Aristotle
  pagaltzis@gmx.de
http://plasmasturm.org/
Blah blah blah blah blah [technorati.com]

Journal of Aristotle (5147)

Saturday April 09, 2005
01:28 PM

CPAN Ratings: the social failings of a useful system

[ #24091 ]

My personal overall opinion is that CPAN Ratings has been a mild success. Key to its usefulness is that all these reviews, noise or signal, are easily reachable directly from the CPAN search itself. This close association makes it better equipped to be relevant than any other venue we currently have.

Of course, most reviews are just about useless and don’t manage to tell you more than that the author thinks “this module is great!” This is no surprise; reviewing well requires empathy, which is a talent few people have; particularly relating to an audience. Reviews which show that the reviewer has put the module through its paces or at least taken an in-depth glance at it, however short, are relatively few and far between. Still, their absolute number is high enough for the system to be useful, in my opinion. I’ve spent quite a bit of my own time trying to write reviews I hope are useful, in hopes of contributing positively to the system.

What really annoys me are the cases of which this one or that are only the latest examples: people use the rating system to write bug reports or to respond to a review. These things are not reviews! They don’t belong where they are posted.

With bug reports, the case is clear: they belong on RT, not on CPAN Ratings. One might argue that grave bugs might warrant a rating (and thus review) all by themselves, but then how many people are there who will actually go back once the bug has been fixed and amend or retract such a “bug review?” Better to keep these things were they belong. Let’s not forget that someone who checks the ratings can easily take a look at the bug tracker as well.

With commentary on reviews, the case gets murkier. CPAN Ratings does not provide a way to contact the author of a review nor does it offer a way to comment on a review. What’s worse is that the lack of a discussion platform is compounded by the abuse as a bug reporting system. I do not have a link handy, but I’ve seen authors respond to bug reports on the rating system by… writing a post on the rating system. Frankly, I have no good ideas on how to solve this problem. The most practical approach (note: not solution) might be to allow posting “reviews” which carry no rating so that they can be used for responses and comments. Still, the rating system is not a discussion forum (and in my opinion, rightly so), so the format will always be ill-suited to discussion.

And of course, the question which remains is: how do we get people to direct comments (even if we knew where those should go) and bug reports to their respective, correct venues?


List all Journal entries
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

CPAN Ratings: the social failings of a useful system 10 Comments More | Login | Reply /

 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.
  • I absolutely agree. Worst thing happening to me so far are bad reviews from people who don't even understand what that module in question does...wish i could at least comment. :(

  • Maybe one should construct a commenting system for the reviews. I.e: be able to comment on the review and hold a discussion about it. Either a nested discussion, but it's possible a flat one will also do.

    This will resolve most of the problems with the current system. Gabor Szabo has already implemented CPAN-Forum, so one way would be to post the review on this module forum, and place a link from the review in cpanratings to ths discussion in the forum, with a text of "comment".

  • how do we get people to direct comments (even if we knew where those should go) and bug reports to their respective, correct venues?

    How about some review guidelines just above the "Review" box? Such as:

    "Do not use this to report bugs, it is likely the author will never see it and it won't get fixed. Please report bugs via [link to the distribution's rt queue]."

    Ironically, use.perl is not the bug report forum for cpanratings. :) I'll mention this thread to Ask so he sees it.

    While I'm at it:

    "Think be

    • Hey, thanks for the comments. I am aware that this is not the bugreport forum for cpanratings :) (though I’m not sure what is), but I wasn’t reporting bugs so much as hoping to kick off a bit of discussion. (Maybe I should post to Perlmonks as well.)

      Some explanatory text would certainly help. It won’t suffice, I think, but it sure would be an improvement over the current situation. Having ratings mailed to authors is a good suggestion, but I’m not sure how it relates to my points;

      • I am aware that this is not the bugreport forum for cpanratings :) (though I’m not sure what is)

        About [perl.org] says to email ask@perl.org.

        (Maybe I should post to Perlmonks as well.)

        Speaking of places where bugs are reported which never get back to the author.

        Having ratings mailed to authors is a good suggestion, but I’m not sure how it relates to my points; was it supposed to, did I miss something?

        I was just sort of on a roll at that point. If nothing else it means the authors might actually r

        • Speaking of places where bugs are reported which never get back to the author.

          Uh, I did say that the intent was to get some discussion going to exchange ideas maybe, not to report a bug. The issues I brought up are no bugs; see also the title I chose.

          Just about every author has an user@cpan.org address. Its best way to guarantee the author, any author, will get your message.

          I agree, but that isn’t the direction I was talking about. I was wondering how the author (or someone else) could respo

          •     Speaking of places where bugs are reported which never get back to the author.

            Uh, I did say that the intent was to get some discussion going to exchange ideas maybe, not to report a bug. The issues I brought up are no bugs; see also the title I chose.

            Just a general hate flung in the direction of Perlmonks. You just happened to be in the path.

            I agree, but that isn’t the direction I was talking about. I was wondering how the author (or someone else) could respond to a review(e

            • Ah! Of course. Noone but the author really needs to respond to a review. Good thinking. I’ll add one minor point: an author should not be able to post “self-reviews.”

              I think together with the stuff I wrote in reply to Ask that makes a cohesive set of proposed changes to mitigate the problems we currently see. I’ll summarize these in an email to him and see if I can lend a hand with the implementation.

  • 1. Patches welcome. :-) There's a todo list [perl.org].

    2. The "was this helpful?" feature would solve some of these issues, I think.

    3. It was a concious choice not to allow comments on reviews. I didn't and don't want it to turn into a discussion forum. I understand the problem of it going that way anyway sometimes, but I'd rather not encourage it.

      - ask

    --

    -- ask bjoern hansen [askbjoernhansen.com], !try; do();

    • Hey, thanks for dropping by. :) I’ll see about how to be more useful (and where I can squeeze this into my crowded headspace (finding time is not really an issue)). As I said, mostly I wanted to stir up some talk, because I can see this issues but I’m not bright enough to have solutions.

      I suppose “was this helpful?” is already on the todo list as “Helpfulness ratings”? That sounds like a good idea, yes. I thought of a meta/moderation system, but I’m unsure if the

Stories, comments, journals, and other submissions on use Perl; are Copyright 1998-2006, their respective owners.