134 replies to this topic

[Hoff123]

I don't like likes and dislikes... But I can see why it would be easier.

[Evan20000]

You could also make a rating system such as the "likes" on facebook (I don't even use facebook, though I know how it works), with no possibility to give a dislike. This still works because quests that nobody likes don't get a "like" rating.

The amount of "likes" generated by a quest would indicate how many people felt it was a worthy quest, yet it would not get bogged down by ratings solely destined to lower a quest's ranking.

On the other hand I can imagine joke quests getting a lot of "likes" simply because they amused people.

Well, it's just a random thought, this can certainly be planned out better, but it's an idea.

I feel like this completely defeats the purpose of a rating system and just turns into a popularity contest. If someone doesn't like something, they should be able to provide constructive feedback. Exactly the same reason why the Youtube style doesn't work either. It's just a mindless up/down system without requiring people to put any thought into their ratings

Also, relevant video for this entire thread.
http://www.youtube.c...h?v=fuJxrTQh5Zs

[Jamian]

I feel like this completely defeats the purpose of a rating system and just turns into a popularity contest. If someone doesn't like something, they should be able to provide constructive feedback. Exactly the same reason why the Youtube style doesn't work either. It's just a mindless up/down system without requiring people to put any thought into their ratings

Also, relevant video for this entire thread.
http://www.youtube.c...h?v=fuJxrTQh5Zs

Yeah, I can understand that... well like I said it was a random thought.

[Eddard McHorn Van-Schnuder]

I have not read through the thread, but I totally agree with the OP. But it's not just quest-ratings, it's ratings when it comes to games in general.

I've written for a gaming mag, and part of my job was reviewing games. We worked with a 1-10 format, where 1 is unacceptable and 10 is amazing. I got a lot of flak from both readers and the guys running the magazine for being way too harsh with my ratings - and interestingly enough, I've gotten similar reactions here on this site. I got flak from readers because their favorite franchises didn't get 10/10's from me, and I got flak from the owners of the magazine because they're adspace sellers, meaning that I was giving low scores to games from the companies who were literally paying their paycheck and keeping both the site and the magazine running.

There's no single cause to this problem. In the case of game critics, I think a lot of it stems from poor management (though those with integrity will rate games based on what they are, not what they wish they were), and on the readers side it probably has a lot to do with fanboyism.

Here however, I think the problem stems from the fact that we are a community filled with both creators and players alike. Almost all of us have dabbled in questmaking at one point or another, and an equal amount of people, if not more, have played quests. No one likes being told that their work isn't great - that's completely normal, but at the same time I also think it's really healthy to be told when you're doing something that just isn't good enough. Cause if no one tells you, you'd never know what you're doing wrong, and in turn you wouldn't be able to improve.

That said, I will freely admit that I have been way too harsh in the past. No wait, let me rephrase that: the way I've explained my harsh opinions have been tactless in the past. There's no such thing as too harsh, but there is such a thing as coming off as mean, and it's when that happens that conflicts arise. In the end of the day, I'd prefer ditching the rating system altogether, and instead focusing on text-only reviews.

Why? Because the past has shown us that we all have different ideas of what a 4/5 is, or how good that's supposed to be. With text-only, all that's left is the opinion itself, and if put properly, that's way more informative than three or four stars could ever hope to be, both to the prospective player and the quest developer looking for ways to improve.

[Omega]

Darknut within....yeah, I'm guilty of giving a joke quest a joke rating, which is probably a bad thing. Rating while sleep deprived and easily amused is a bad idea.

I've also noticed that there's a weird stigma with joke quests that if you don't like them or rate them highly, then you "didn't get the joke" or don't have a sense of humor. While it's all fun to laugh at a joke quest, they shouldn't be placed on a higher pedestal because they're not taking themselves seriously.

Joke quests could still be well designed and deserve 5. Though I'm not saying they'd deserve the same 5 as say HoD or the DoR tileset. For this reason we should move to 10/10 stars instead of five. The ratings will be edited as so. So you can still choose great or awesome for a quest without it being 10 stars. maybe 8 is awesome, and 9 is amazing. You know, something like that but the names can be voted on. Maybe require a few questions for higher ratings. Example:

10 star rating would require you to first answer design questions about the quests and maybe write a review that must be passed by a certain staff member as a suitable review for the rating giving, to ensure the rating and the review match up.

-After further talk with others in skype, this may not be a good idea. lol

Edited by Franky, 02 April 2013 - 06:03 PM.

[TheLegend_njf]

Novelty and impulse rating.

It exists, knowing it exists helps us clarify if the rating is justified, most often with new quests, they are not. My quest was in number 1 position in Zeldaclassic.com for over a year.

A year and a half later, gone, because it received an honest rating based on its flaws. The fact that it was up there for an entire year blew my mind, but still misleading.

I'd say most quests that are rated 5/5 and its only a few months old would indicate novelty score.

[Mero]

I think we should ditch the current stars out of 5 system with something more practical, and jump on the "like" bandwagon with every other site on the internet.

[Aevin]

I don't see how likes would do anything ... It's just a sign of how many people played it, so older quests would obviously have more, and it would discourage thoughtful reviews while not stratifying the quests in any meaningful way.

But yeah, it's probably best to wait until the update before we get too excited about changing anything.

[SofaKing]

As of right now there are six possible ratings, anywhere from 0-5. I might go down to, say, four, and better define them (as opposed to just bad, poor, good, great, etc.) Those are subjective terms that mean different things to different people. As an example, a 1-star rating might mean it suffers from game-breaking bugs, just terrible game play, or other major, major flaws. Few completed quests that have even a little effort put into them would fall into this category. Two stars means it's playable, and maybe even fun and interesting in places, but needs some work to be a game that most would want to play from beginning to end and get truly excited about. Three stars means it's quite enjoyable and fun to play but lacks the length, epic feeeling, level design quality, etc., to be considered a masterwork. Four stars would be reserved for quests that almost feel like they could be commercially produced. These should be rare. It's like a bell curve, most quests are somewhere in the middle, which is how the quest DB in fact works.

Yes, most quests would be 2-star or 3-star, but that, combined with screenshots and also comments explaining what people liked and didn't like, should give potential quest players a good idea as to whether it's something that interests them. I just see too many quests with 4 point-something averages these days that are inferior to quests from five years ago with 3 point-something averages. People rate too generously these days. I've been guilty of it too. Perhaps the ability to edit a rating or comments might be useful as well. For example, if the author takes the time to fix bugs, I think the player should be able to re-rate in light of the fixes. It's all about making the rating system as helpful as possible. When there are hundreds of custom quests from which to choose, helping custom quest players separate the wheat from the chaff is important.

Edited by FerentzRocks, 03 April 2013 - 12:48 AM.

[Peteo]

I'm on the "like or dislike" bandwagon. The current 5 star system just doesn't work. The list of "Top Rated Quests" is quite hilarious at the moment in my opinion.

[Eddard McHorn Van-Schnuder]

Yeah, it's not a perfect system, but neither is likes or dislikes. As I said earlier, I'm personally against the very idea of a rating, as I don't think it's enough to properly convey the thoughts of the comment author. Another way would be to let the numbers speak for themselves, and have the 'top rated', or perhaps 'most popular' category be made out of the quests who has the most downloads.

But even then there's the problem that some might 'advertise' their project more than others, which could result in the numbers showing simply the most prominent quests. That's not good, because that does in no way guarantee that it's a good game.

In the end of the day I really don't have a solution to the problem. I think the best rating system (if I had to pick one) would be a 1-100 kind of thing, but that'd only work if people actually used the entire scale - and that's already a problem to begin with.

[Moosh]

I'm on the "like or dislike" bandwagon. The current 5 star system just doesn't work. The list of "Top Rated Quests" is quite hilarious at the moment in my opinion.

Really? I can only see a couple quests there that I feel don't belong where they are and ultimately it's all a matter of opinion.

A like system would essentially reduce ratings to only 2 possible results, a 0 or a 5. Oh gee, that's how a lot of people rate quests already. I'd hardly call that fixing the problem, it only cuts out those individuals that actually put thought into their ratings.

I'm rubbish at maths, but the median idea Evan put forward sounds pretty solid. What if we took the median score and the current rating and averaged the two?

Edited by Moosh, 03 April 2013 - 09:50 AM.

[LinktheMaster]

Well, I'll go ahead and jump in to say that the new site has a more advanced algorithm to come up with ratings. It's called a Bayesian Average. It basically helps weed out outliers by adding in an extra rating of a 3.

So, let's say a new quest comes in and has 2 ratings of 5. Instead of being rated 5, it would be rated like this:

(5+5+3)/3 = 4.33

Likewise, if someone just comes and gives a horrible first review, it helps solve that too.

(0+3)/2 = 1.5

I think this is the best system since it just drifts everything slightly toward the center. The less reviews, the more it pushes the rating toward an basic 3. The problem is, no system is perfect. Personally, I'm not sure a like/dislike system would work well on PureZC. That really seems to shine when tons of people post ratings.

[Peteo]

Well, I'll go ahead and jump in to say that the new site has a more advanced algorithm to come up with ratings. It's called a Bayesian Average. It basically helps weed out outliers by adding in an extra rating of a 3.

So, let's say a new quest comes in and has 2 ratings of 5. Instead of being rated 5, it would be rated like this:

(5+5+3)/3 = 4.33

Likewise, if someone just comes and gives a horrible first review, it helps solve that too.

(0+3)/2 = 1.5

I think this is the best system since it just drifts everything slightly toward the center. The less reviews, the more it pushes the rating toward an basic 3. The problem is, no system is perfect. Personally, I'm not sure a like/dislike system would work well on PureZC. That really seems to shine when tons of people post ratings.

Wow, I agree. Drifting everything slightly toward the center sounds like the best thing to do. So if someone sees a decent quest and thinks "aww I don't want to hurt his feelings, here, take 5 stars" the quest does't go at the top of therankings just like that.

Nice system, LTM!

Edited by Peteo, 03 April 2013 - 10:21 AM.

[Moosh]

That sounds kinda silly since the only real effect it has is that it pulls down good quests that don't have a lot of ratings. I mean when you have like 20 newgrounds users rating 5 stars will that 3 really make any difference? In that scenario averaging in the median wouldn't really do much either I guess.

What if ratings were weighted slightly more based on length? That way a well thought out review could carry more weight than "Another great quest by *quest creator*."

Edited by Moosh, 03 April 2013 - 10:23 AM.