On rating quests
#31
Posted 02 April 2013 - 05:06 PM
#32
Posted 02 April 2013 - 05:12 PM
You could also make a rating system such as the "likes" on facebook (I don't even use facebook, though I know how it works), with no possibility to give a dislike. This still works because quests that nobody likes don't get a "like" rating.
The amount of "likes" generated by a quest would indicate how many people felt it was a worthy quest, yet it would not get bogged down by ratings solely destined to lower a quest's ranking.
On the other hand I can imagine joke quests getting a lot of "likes" simply because they amused people.
Well, it's just a random thought, this can certainly be planned out better, but it's an idea.
I feel like this completely defeats the purpose of a rating system and just turns into a popularity contest. If someone doesn't like something, they should be able to provide constructive feedback. Exactly the same reason why the Youtube style doesn't work either. It's just a mindless up/down system without requiring people to put any thought into their ratings
Also, relevant video for this entire thread.
http://www.youtube.c...h?v=fuJxrTQh5Zs
#33
Posted 02 April 2013 - 05:13 PM
I feel like this completely defeats the purpose of a rating system and just turns into a popularity contest. If someone doesn't like something, they should be able to provide constructive feedback. Exactly the same reason why the Youtube style doesn't work either. It's just a mindless up/down system without requiring people to put any thought into their ratings
Also, relevant video for this entire thread.
http://www.youtube.c...h?v=fuJxrTQh5Zs
Yeah, I can understand that... well like I said it was a random thought.
#34
Posted 02 April 2013 - 05:23 PM
I've written for a gaming mag, and part of my job was reviewing games. We worked with a 1-10 format, where 1 is unacceptable and 10 is amazing. I got a lot of flak from both readers and the guys running the magazine for being way too harsh with my ratings - and interestingly enough, I've gotten similar reactions here on this site. I got flak from readers because their favorite franchises didn't get 10/10's from me, and I got flak from the owners of the magazine because they're adspace sellers, meaning that I was giving low scores to games from the companies who were literally paying their paycheck and keeping both the site and the magazine running.
There's no single cause to this problem. In the case of game critics, I think a lot of it stems from poor management (though those with integrity will rate games based on what they are, not what they wish they were), and on the readers side it probably has a lot to do with fanboyism.
Here however, I think the problem stems from the fact that we are a community filled with both creators and players alike. Almost all of us have dabbled in questmaking at one point or another, and an equal amount of people, if not more, have played quests. No one likes being told that their work isn't great - that's completely normal, but at the same time I also think it's really healthy to be told when you're doing something that just isn't good enough. Cause if no one tells you, you'd never know what you're doing wrong, and in turn you wouldn't be able to improve.
That said, I will freely admit that I have been way too harsh in the past. No wait, let me rephrase that: the way I've explained my harsh opinions have been tactless in the past. There's no such thing as too harsh, but there is such a thing as coming off as mean, and it's when that happens that conflicts arise. In the end of the day, I'd prefer ditching the rating system altogether, and instead focusing on text-only reviews.
Why? Because the past has shown us that we all have different ideas of what a 4/5 is, or how good that's supposed to be. With text-only, all that's left is the opinion itself, and if put properly, that's way more informative than three or four stars could ever hope to be, both to the prospective player and the quest developer looking for ways to improve.
#35
Posted 02 April 2013 - 05:50 PM
Darknut within....yeah, I'm guilty of giving a joke quest a joke rating, which is probably a bad thing. Rating while sleep deprived and easily amused is a bad idea.
I've also noticed that there's a weird stigma with joke quests that if you don't like them or rate them highly, then you "didn't get the joke" or don't have a sense of humor. While it's all fun to laugh at a joke quest, they shouldn't be placed on a higher pedestal because they're not taking themselves seriously.
10 star rating would require you to first answer design questions about the quests and maybe write a review that must be passed by a certain staff member as a suitable review for the rating giving, to ensure the rating and the review match up.
-After further talk with others in skype, this may not be a good idea. lol
Edited by Franky, 02 April 2013 - 06:03 PM.
#36
Posted 02 April 2013 - 06:37 PM
It exists, knowing it exists helps us clarify if the rating is justified, most often with new quests, they are not. My quest was in number 1 position in Zeldaclassic.com for over a year.
A year and a half later, gone, because it received an honest rating based on its flaws. The fact that it was up there for an entire year blew my mind, but still misleading.
I'd say most quests that are rated 5/5 and its only a few months old would indicate novelty score.
#37
Posted 02 April 2013 - 09:54 PM
#38
Posted 02 April 2013 - 10:17 PM
But yeah, it's probably best to wait until the update before we get too excited about changing anything.
#39
Posted 02 April 2013 - 11:16 PM
Yes, most quests would be 2-star or 3-star, but that, combined with screenshots and also comments explaining what people liked and didn't like, should give potential quest players a good idea as to whether it's something that interests them. I just see too many quests with 4 point-something averages these days that are inferior to quests from five years ago with 3 point-something averages. People rate too generously these days. I've been guilty of it too. Perhaps the ability to edit a rating or comments might be useful as well. For example, if the author takes the time to fix bugs, I think the player should be able to re-rate in light of the fixes. It's all about making the rating system as helpful as possible. When there are hundreds of custom quests from which to choose, helping custom quest players separate the wheat from the chaff is important.
Edited by FerentzRocks, 03 April 2013 - 12:48 AM.
#40
Posted 03 April 2013 - 09:01 AM
#41
Posted 03 April 2013 - 09:38 AM
But even then there's the problem that some might 'advertise' their project more than others, which could result in the numbers showing simply the most prominent quests. That's not good, because that does in no way guarantee that it's a good game.
In the end of the day I really don't have a solution to the problem. I think the best rating system (if I had to pick one) would be a 1-100 kind of thing, but that'd only work if people actually used the entire scale - and that's already a problem to begin with.
#42
Posted 03 April 2013 - 09:49 AM
I'm on the "like or dislike" bandwagon. The current 5 star system just doesn't work. The list of "Top Rated Quests" is quite hilarious at the moment in my opinion.
Really? I can only see a couple quests there that I feel don't belong where they are and ultimately it's all a matter of opinion.
A like system would essentially reduce ratings to only 2 possible results, a 0 or a 5. Oh gee, that's how a lot of people rate quests already. I'd hardly call that fixing the problem, it only cuts out those individuals that actually put thought into their ratings.
I'm rubbish at maths, but the median idea Evan put forward sounds pretty solid. What if we took the median score and the current rating and averaged the two?
Edited by Moosh, 03 April 2013 - 09:50 AM.
#43
Posted 03 April 2013 - 10:11 AM
So, let's say a new quest comes in and has 2 ratings of 5. Instead of being rated 5, it would be rated like this:
(5+5+3)/3 = 4.33
Likewise, if someone just comes and gives a horrible first review, it helps solve that too.
(0+3)/2 = 1.5
I think this is the best system since it just drifts everything slightly toward the center. The less reviews, the more it pushes the rating toward an basic 3. The problem is, no system is perfect. Personally, I'm not sure a like/dislike system would work well on PureZC. That really seems to shine when tons of people post ratings.
#44
Posted 03 April 2013 - 10:19 AM
Well, I'll go ahead and jump in to say that the new site has a more advanced algorithm to come up with ratings. It's called a Bayesian Average. It basically helps weed out outliers by adding in an extra rating of a 3.
So, let's say a new quest comes in and has 2 ratings of 5. Instead of being rated 5, it would be rated like this:
(5+5+3)/3 = 4.33
Likewise, if someone just comes and gives a horrible first review, it helps solve that too.
(0+3)/2 = 1.5
I think this is the best system since it just drifts everything slightly toward the center. The less reviews, the more it pushes the rating toward an basic 3. The problem is, no system is perfect. Personally, I'm not sure a like/dislike system would work well on PureZC. That really seems to shine when tons of people post ratings.
Wow, I agree. Drifting everything slightly toward the center sounds like the best thing to do. So if someone sees a decent quest and thinks "aww I don't want to hurt his feelings, here, take 5 stars" the quest does't go at the top of therankings just like that.
Nice system, LTM!
Edited by Peteo, 03 April 2013 - 10:21 AM.
#45
Posted 03 April 2013 - 10:21 AM
What if ratings were weighted slightly more based on length? That way a well thought out review could carry more weight than "Another great quest by *quest creator*."
Edited by Moosh, 03 April 2013 - 10:23 AM.
1 user(s) are reading this topic
0 members, 1 guests, 0 anonymous users