Jump to content

Photo

Reviews and Ratings Rule Addition


  • This topic is locked This topic is locked
97 replies to this topic

#16 nicklegends

nicklegends

    Trofessional Pransposer

  • Contributors
  • Real Name:Ed
  • Gender:Male
  • Location:California

Posted 26 October 2016 - 07:27 PM

Frankly, I think this rule is an unequivocally terrible idea.  We already have standards for harassment that could easily be met by unexplained strings of low ratings.  For other ratings, we have no right to say that some people's honest opinions are more valuable than others.

 

I've said this before in the staff forums to no effect, so I'll share it with a larger audience now: we should do what YouTube and other sites figured out a long time ago and switch to a like/dislike system for all our database content.  As Saffith said, a five-star scale means different things to different people, but in general, people are self-consistent when it comes to what they like or don't like.  The added benefit is that people with extremely low or extremely high standards don't appreciably throw off the average by giving reviews that top-out or bottom-out the scale—because everybody has the same power.

 

A like/dislike system removes the need for this rule and all its dangerous implications.


  • Chris Miller, Rambly, Migokalle and 2 others like this

#17 NewJourneysFire

NewJourneysFire

    Deified

  • Contributors
  • Real Name:Grant
  • Gender:Male

Posted 26 October 2016 - 07:30 PM

More often than not, a dislike button is often abused and thus makes it a useless irritating thing. This is why Facebook doesn't have it. :P

 

I've seen videos on youtube get disliked the moment they are uploaded sometimes. Many people are just prone to dislike something made from a person they dislike.


Edited by NewJourneysFire, 26 October 2016 - 07:31 PM.


#18 thepsynergist

thepsynergist

    Composer/Remixer

  • Members
  • Real Name:Jeff
  • Gender:Male
  • Location:California, USA

Posted 26 October 2016 - 07:47 PM

More often than not, a dislike button is often abused and thus makes it a useless irritating thing. This is why Facebook doesn't have it. :P

 

I've seen videos on youtube get disliked the moment they are uploaded sometimes. Many people are just prone to dislike something made from a person they dislike.

 

Be that as it may, the average rating would offset the one person that decides they want to troll (that's not even what we have here, by the way, just expressing what's common on youtube), so it shouldn't even be an issue, unlike ratings, which feel more weighted than the just "I like it" vs "I don't like it".


Edited by thepsynergist, 26 October 2016 - 07:47 PM.


#19 LinktheMaster

LinktheMaster

    Hey Listen, Kid

  • Members
  • Real Name:Matt
  • Gender:Male
  • Location:United States

Posted 26 October 2016 - 07:53 PM

I don't think a like / dislike system is that helpful. I know it's easy to compare stuff to YouTube, but YouTube has millions of people who rate things. Even your average run of mill video with a few thousand views is apt to have 50 people or more rate it. With that, a ratio is pretty clear on the quality of something.

 

Contrast that with PureZC, where it's not uncommon for quests to not even have 5 ratings. I don't think 2 likes and 1 dislike is that helpful for judging how good something is. Does like mean that something is okay, or is it only reserved for stuff you'd recommend to other people? On the flip side, would dislikes be for stuff that's meh, or would it just be reserved for stuff that is really bad? Add that to the fact that saying you dislike something seems a little bit more personal than rating something a 3, I think stuff would start having an even higher rate of likes than you may consider, which compromises the whole accuracy of everything.

 

Keep in mind that many sites such as Amazon actually do still use the five star rating, so a like/dislike rating system is not unanimously decided to be the best rating system ever. In the end, the biggest restriction is our size as the impact of one person's review is far too great with such few ratings. Nothing short of dropping off high and low reviews will fix that, but still.. what are you supposed to do with only 4 reviews? At that point you'd be ignoring 50% of the ratings, so that's not perfect either. This would impact a large number of database entries, so it's not like it's a relatively small number of quests that would fall under this category.

 

In the end, I fall back on what I typically say with this sort of thing: there's no easy answer for PureZC. :shrug:


  • Nathaniel, Shane, Aevin and 1 other like this

#20 Air Luigi

Air Luigi

    Tio guachi

  • Members
  • Gender:Male
  • Location:Spain

Posted 26 October 2016 - 08:02 PM

You could get around this entire thing if you told the system to exclude extremes when calculating averages. That's what a proper statistial analysis would do, too. And that way you could keep the ratings in the database, they just wouldn't affect the overall rating if they are too outlandish to be considered representative. You wouldn't even have to monitor it at all after you implemented the algorithm to take care of it.

 

This. I found weird always how much impact has every single negative review on the averages. Basically if someone wants to destroy the average rating, it's quite easy to do it.... while it takes tons and tons of 5/5 for a quest to climb on the rankings. You can have 30 reviews with mostly 5/5 and some 4/5, one single person rate the quest with 0/5, and boom, the quest is "dismissed". It's ridiculous at times.


Edited by Air Luigi, 26 October 2016 - 08:05 PM.

  • Sheik likes this

#21 Dimentio

Dimentio

    Yeah, I don't buy that.

  • Members
  • Real Name:Joshua
  • Gender:Male
  • Location:America's Hat.

Posted 26 October 2016 - 08:11 PM

Except Youtube/Facebook don't tie Likes/Dislikes to ratings. The rating system itself discourages spam reviews, and makes it rather obvious to spot which ones are spam reviews. In all actuality, there should be a rule that encourages people to give constructive criticism if they choose to dislike a quest. That way, the quest author has actual information to work on to make the quest better, at the same time preventing somebody and their alternate accounts/bots from coming on and spamming a quest down into oblivion, since there would be an obvious rule break once it is found that they did not give constructive criticism and simply said "It was trash".

Yes, a change to the Like/Dislike system may ruin the ratings we have now, but... aren't most of them outdated? This site came about in 2002, and most of those reviews came from the 2006-2008 era, before the final release of 2.5. People should be rating things by today's standards. If people still think a quest is a timeless masterpiece, nothing would stop them from rating it as such again. If people once thought something was great, but it turned out to be eh, then they can go and rate it on their new standards. A Like/Dislike system might not be as accurate as a 5 stars system, but it certainly helps when dealing with drama like this, where instead of people getting butthurt at a user rating their favorite quest ever 4 stars, there wouldn't be any fighting, because that person, instead of rating it 4 stars, rated it "I like it". And that's all that needs to be said. We aren't a community of professional critics. Most of us are amateur game developers, and most of us are amateur critics. Why do we need 5 stars? That means we have 6 rating choices: "I hate it." "I don't like it." "Eh." "It's okay" "It's great" "It's fantastic!". Why? A Like/Dislike system would be simpler. A user likes the quest? No more spending 5 minutes choosing whether the quest was "Good" or "Great". You just pick "I like it". No more getting worried at people raging at you for rating something "Terrible". You just pick "I didn't like it". Nobody can blame you for not liking something. It makes it easier for the staff, and it makes it easier for the reviewers. 

"But what about the old reviews? We'll have a bunch of unrated quests with nobody playing them!", you say. Well, then you're underestimating PureZC. Unrated quests popup at the last page of the "sort by ratings" quest database, so they're easy to find. People will play unrated quests they've never played before. And if they played it,and liked it, they might give it a second shot. You could also have a "sort by votes" command, where you can sort the quests by the number of reviews and see how many people rated the quest. That way, they can check to see how valid they think the average rating is. Also, maybe you can save the "old" ratings as comments so that people can see legacy reviews or that people who played more recently (like within the past couple months) can just repost their old review, just with the new vote system. 

I went on a ramble there, but I really believe a Like/Dislike system would work out quite nicely.



#22 nicklegends

nicklegends

    Trofessional Pransposer

  • Contributors
  • Real Name:Ed
  • Gender:Male
  • Location:California

Posted 26 October 2016 - 08:37 PM

Keep in mind that many sites such as Amazon actually do still use the five star rating, so a like/dislike rating system is not unanimously decided to be the best rating system ever.

True, but Amazon also doesn't use a simple average to calculate their scores. They de-value extreme low and extreme high scores.

#23 Nimono

Nimono

    Ultra Miyoa Extraordinaire!

  • Database Staff
  • Real Name:Matthew
  • Gender:Male
  • Location:Static Void Kingdom

Posted 26 October 2016 - 08:38 PM



Except Youtube/Facebook don't tie Likes/Dislikes to ratings. The rating system itself discourages spam reviews, and makes it rather obvious to spot which ones are spam reviews. In all actuality, there should be a rule that encourages people to give constructive criticism if they choose to dislike a quest. That way, the quest author has actual information to work on to make the quest better, at the same time preventing somebody and their alternate accounts/bots from coming on and spamming a quest down into oblivion, since there would be an obvious rule break once it is found that they did not give constructive criticism and simply said "It was trash".

I'd like to state as a fellow user that I do not agree with this idea- specifically, the part about it ONLY being on dislikes. It should be a given on ANY rating, like or dislike. :P After all, "GREAT QUEST 5/5" is just as meaningless as "this was garbage 0/5" in helping people decide whether or not something is good, right?


  • ShadowTiger and Naru like this

#24 Dimentio

Dimentio

    Yeah, I don't buy that.

  • Members
  • Real Name:Joshua
  • Gender:Male
  • Location:America's Hat.

Posted 26 October 2016 - 08:47 PM

I'd like to state as a fellow user that I do not agree with this idea- specifically, the part about it ONLY being on dislikes. It should be a given on ANY rating, like or dislike. :P After all, "GREAT QUEST 5/5" is just as meaningless as "this was garbage 0/5" in helping people decide whether or not something is good, right?

My problem with that is, what if there's no constructive criticism to give? I mean, it's unlikely, but what if a quest comes around where you just can't find flaws?

 

...Then again, you can just say "I can't think of any flaws", in which case never mind, constructive criticism should be enforced for both (but mostly the negative ratings, since if there is something in the quest you don't like, then there's more value in constructive criticism there than a positive rating).



#25 Nimono

Nimono

    Ultra Miyoa Extraordinaire!

  • Database Staff
  • Real Name:Matthew
  • Gender:Male
  • Location:Static Void Kingdom

Posted 26 October 2016 - 08:50 PM

My problem with that is, what if there's no constructive criticism to give? I mean, it's unlikely, but what if a quest comes around where you just can't find flaws?

 

...Then again, you can just say "I can't think of any flaws", in which case never mind, constructive criticism should be enforced for both (but mostly the negative ratings, since if there is something in the quest you don't like, then there's more value in constructive criticism there than a positive rating).

There IS constructive criticism to give. It's called "point out what you DID like". :P Criticism isn't JUST the negative, after all! :D That's why it's important that you point out both positives AND negatives when offering criticism. Show what someone did wrong so they know what to fix, but simultaneously point out what they did right so they can feel proud of it and perhaps use that success as inspiration to fix their mistake. And also provide ideas. But if there's nothing to really complain about, make sure to point out the good things, the parts you REALLY liked, and why- that'll show them what they could focus on next time.


  • Dimentio likes this

#26 NewJourneysFire

NewJourneysFire

    Deified

  • Contributors
  • Real Name:Grant
  • Gender:Male

Posted 26 October 2016 - 08:55 PM

Some people are just negative. It's just who they are.
  • Jared, Dimentio and Naru like this

#27 Dimentio

Dimentio

    Yeah, I don't buy that.

  • Members
  • Real Name:Joshua
  • Gender:Male
  • Location:America's Hat.

Posted 26 October 2016 - 08:57 PM

There IS constructive criticism to give. It's called "point out what you DID like". :P Criticism isn't JUST the negative, after all! :D That's why it's important that you point out both positives AND negatives when offering criticism. Show what someone did wrong so they know what to fix, but simultaneously point out what they did right so they can feel proud of it and perhaps use that success as inspiration to fix their mistake. And also provide ideas. But if there's nothing to really complain about, make sure to point out the good things, the parts you REALLY liked, and why- that'll show them what they could focus on next time.


Or that. That works.



#28 Nathaniel

Nathaniel

    Deified

  • Forum Moderators
  • Gender:Male
  • Location:Anywhere, USA

Posted 26 October 2016 - 09:16 PM

I don't think it really matters what system we have in place.  I can guarantee you guys if we changed to any other system, be it like, like/dislike, or something else, we are bound to hear the same sorts of criticisms all over again, and we would get a very similar conversation to this one all over again sometime in the future.  It's easy to have a short memory of this conversation later in the future, plus add all the new members coming into such a repeated conversation.  We would be back to square one, and all such efforts would likely be for nought.  You can change systems, but you rarely can change people.

 

All systems are flawed, because people are flawed.  And as LTM pointed out, just because a particular system works somewhere else, doesn't necessarily mean it's for PureZC.  To change a system in place, there should be enough conclusive points to overturn it, and I just don't think there is.  You have to account for the size of our member base, and yes, the history of this site, which does in fact matter.  To toss away old ratings from members who no longer visit this site, many of whom likely won't return again, automatically invalidates them, which is in itself a form of censorship, even if the reviews themselves are not being removed.  Standards were lower in the past because ZC didn't offer as much, so the perspective factored in on what was available at the time.  I don't think it's accurate to look at a quest from 2002 with 2016 glasses on, especially version 2.50 glasses in 2016.  Imagine if someone said "Ocarina of Power should have used scripts."  Or even "Ocarina of Power would have been better with scripts."  Most would probably find that absurd, because you simply couldn't do those things back then.  Not just scripts, but fewer tilesets were available, fewer features in ZQuest, and you name it.  So for that reason I think it's important to keep those very old reviews and ratings around.


  • Air Luigi, Orithan and David like this

#29 Air Luigi

Air Luigi

    Tio guachi

  • Members
  • Gender:Male
  • Location:Spain

Posted 26 October 2016 - 10:07 PM

A bit of offtopic:

 

Standards were lower in the past because ZC didn't offer as much, so the perspective factored in on what was available at the time.  I don't think it's accurate to look at a quest from 2002 with 2016 glasses on, especially version 2.50 glasses in 2016.  Imagine if someone said "Ocarina of Power should have used scripts."  Or even "Ocarina of Power would have been better with scripts."  Most would probably find that absurd, because you simply couldn't do those things back then.  Not just scripts, but fewer tilesets were available, fewer features in ZQuest, and you name it.  So for that reason I think it's important to keep those very old reviews and ratings around.

 

In reality, I think many of the old quests are still interesting compared to 2.5 quests (and in some cases better than the new quests). What made a quest good is the world building and secrets, and you can do that really good even in 1.90. You can do so much and have so many features in 2.5 today... that many people is missing the point of what made ZC good in the past (tip: the exploration and discovery of a interesting world). Features help with creativity and variety, but in my book, if you have a boring world, you have a lackluster quest, despite all the features and flashy stuff. There are quests that were made in 2.10 in the last two years that received a hot reception anyway (TotheTop for example). So, there is hope ;)

 

[/end of offtopic]


  • Einsiety likes this

#30 Jamian

Jamian

    ZC enthusiast

  • Members
  • Gender:Unspecified

Posted 27 October 2016 - 03:01 AM

If you've been playing Mario Maker, you'll probably have seen limits of a "like" system in action. Many of the top rated levels are regularly music stages, automatic stages, or very gimmicky stages that are easy to "like" because they're quite fun and there's really nothing to dislike about them, but they're usually not the overall best levels.

 

I can understand why Nintendo is doing this instead of a real rating system, they'll want to avoid any drama and keep the system really simple so nobody will take it too seriously. There are also so many levels being made and played every day that policing an actual rating system would be an absolutely dauting task. It's on a much, much bigger scale than ZC, and it makes sense in that context. But back to the point, does the Mario Maker top rated levels necessarily represent the community's all time favourites? I don't think many would say their one favourite level is a music one or an automatic one, yet those often receive tons of stars, because even if they're not the best of the best, they are easy to "like".

 

So, there is no perfect solution, but if you want the rating system to better reflect how the community feels about a quest, I think being able to rate just how much you liked it is a much better option.


Edited by Jamian, 27 October 2016 - 03:16 AM.

  • Air Luigi and Shane like this


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users