On rating quests
Posted 02 April 2013 - 01:13 PM
Thoughts? Agree? Disagree? As someone who plays a lot of custom quests, I think the rating system is more useful if people are little more critical and selective.
Posted 02 April 2013 - 01:24 PM
To be honest though, at times I rate quests highly simply so that the author won't get discouraged. I've only put up one quest on this place so far and it got 4/5, which I was happy with. Not everyone takes rating quests seriously.
Posted 02 April 2013 - 01:26 PM
Posted 02 April 2013 - 01:34 PM
I think the problem here is that people try and rate quests too high/too low to move the average vote closer to where they think it should be as opposed to honestly voting what they think about it. I'm pretty sure the new site has a system for combating this, so this should hopefully be a thing of the past.
How would that work?
I know the imdb (internet movie database) uses an algorithm that makes your votes count less towards the total rating if you regularly give extreme ratings. Is it something like this?
Posted 02 April 2013 - 01:40 PM
Posted 02 April 2013 - 01:47 PM
Posted 02 April 2013 - 01:55 PM
I think the problem here is that people try and rate quests too high/too low to move the average vote closer to where they think it should be as opposed to honestly voting what they think about it. I'm pretty sure the new site has a system for combating this, so this should hopefully be a thing of the past. I agree though, rank inflation is an issue if one is trying to compare an older quest to a newer one.
Question: Will some older quests get a default rating to start on, or are all ratings being reset for all quests on the new site?
Posted 02 April 2013 - 01:56 PM
Hmm ... I've been thinking a lot about the rating system lately, in part because, to be honest, I'm a little neurotic about my own. A few things I've noticed ...
1. Comedic quests tend to rate higher on average.
2. Almost nobody rates lower than a 3.
3. Yes, newer quests tend to rate higher.
It could be that with time, more votes come that average out the quests, whereas the newer quests are more likely to suffer from extremes. That is, if a quest "should" be rated 4 stars on average, but one person rates it a five, it's more likely to jump up to 4.7 and round up if there's fewer four star ratings to balance it out.
I also find that when rating, many people comment about the difficulty, the graphics, glitches and item locations, but very few comment on story or atmosphere.
Also, this. A rather lengthy article describing the tendency for five-star rating systems to lean toward the extremes of one star and five star.
Edited by Aevin, 02 April 2013 - 02:00 PM.
Posted 02 April 2013 - 02:01 PM
Question:(...) or are all ratings being reset for all quests on the new site?
I hope not, because that would almost guarantee the kind of vote Evan was talking about (voting extremely to influence how you think the quest should be rated in the database, as opposed to voting honestly).
An algorithm to make people's votes count more or less based on how diverse their ratings are would be the best solution IMO.
Posted 02 April 2013 - 02:04 PM
"A pretty good quest, but it has some major flaws. Four stars." And then, "This quest is virtually perfect in every way and everybody EVER should play it!!! Four stars!"
(Not actual ratings, but I've seen some like them out there.)
Edited by Aevin, 02 April 2013 - 02:05 PM.
Posted 02 April 2013 - 02:18 PM
I'm not very good at rating certain ratings. If I really like it, I usually just give it a 5 :/
Posted 02 April 2013 - 02:23 PM
Posted 02 April 2013 - 02:34 PM
I think rating would be easier if those descriptions were to be added in a short form under the rating because otherwise people assign their own meaning to the star system. Force a meaning and people will be more likely to think about fitting it to the meaning.
So i other words put clear description under the stars that tell what each star means. 1 star is god awful. 5 is very enjoyable i want more. Etc.
Edited by aaa2, 02 April 2013 - 02:35 PM.
Posted 02 April 2013 - 02:44 PM
To the topic at hand, from a general standpoint, indeed, people seem to rate too high or too low. I chalk that up to there being very few members of PureZC who are actually professional reviewers.
To elaborate, I feel like maybe the majority of raters don't know how to properly review, and that's fine, it won't hurt them in life. Unless they make a bad recommendation to a friend and that friend beats them up for their bad taste. Er, well, I do feel like people tend to rate quests too highly just because they like them, and while that does annoy me, that's fine, that's none of my business. I try to rate fairly when I do rate, and not based on the rest of the votes.
It was noted that joke quests tend to score higher, and honestly I have mixed opinions about that. Usually people tend to make joke ratings in response that praise it, or even joke reviews that are negative. But I think the feeling is that few people want to be seen as a buzzkill and so don't rate honestly. It's kind of annoying to see joke quests listed highly in the top-rated list, unless they're actually pretty solid gameplay as well, like The Darknut Within. There are others that are just awful, and while I enjoy the spirit of the mood with the quests, seeing endless praise of it does get kind of annoying. What other solution would I have? Make a joke comment that says you're giving it five stars but actually rating it a 3, or something like that, I dunno.
At the end of the day, though, it's honestly none of our business. Would it be nice if everyone rated things thoughtfully rather than "I like this, five stars"? Yes. But again, few (if any) of us are professional reviewers, and few of that number will think it worth the time to improve their feedback. We can try reaching out, like in this thread, in hopes that people will think about their review styles and changing them, but that's about all we can do. We can't force it.
While I like a good story, and it can definitely enhance an already great game, it's honestly the very least of my concerns when it comes to rating games/quests. Maybe it's because I grew up with the more simplistic games of the NES and SNES, but story has never been an important factor in judging the quality of a game for me. If the story is awful but the gameplay is amazing, I forgive the bad story because I want to keep playing anyway. Atmosphere I guess is similar, but possibly more important to me than story, since that can affect how you feel playing the game, and if it's a poor atmosphere, I'm hesitant to continue.
There's something else few people seem to rate on: Use of music. But again, no one here is a professional so they probably just think "Eh, the music was okay/great/subpar, but whatever," and leave that out of their ratings. I guess if the music choice is okay or good, people will just be like "That's how it goes" and talk about other aspects of the quest. If it's great or awful, I suppose that's where the exception lies, because people will want to praise your music choice or berate you for choosing awful music for places. Or outright having horrible quality MIDIs.
Blah blah I'm rambling here so done now.
Posted 02 April 2013 - 02:51 PM
While I like a good story, and it can definitely enhance an already great game, it's honestly the very least of my concerns when it comes to rating games/quests. Maybe it's because I grew up with the more simplistic games of the NES and SNES, but story has never been an important factor in judging the quality of a game for me.
I think a lot of people think like that. And I don't find anything wrong with it. Given that many of the people here are programmers and veteran game designers/rom hackers, it's not surprising that the interests of the majority are in the technical stuff behind game making. I, on the other hand, consider myself a storyteller who just happens to make games, so I do sometimes wish people valued story more. I should also clarify that I don't believe a good story needs to be very complex.
Edited by Aevin, 02 April 2013 - 02:57 PM.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users