Jump to content

Photo

Reviews and Ratings Rule Addition


  • This topic is locked This topic is locked
97 replies to this topic

#1 Aevin

Aevin

  • Members
  • Pronouns:He / Him
  • Location:Oregon

Posted 26 October 2016 - 03:31 PM

 I'd like to inform you all of an addition to the PureZC Site and Forum Rules, under rule #12. Let me start by saying this is a very sensitive subject. We believe that everyone should be allowed to express their opinions in all areas of the site in ways that don't harm other members. We don't like to come across as engaging in censorship or policing of opinions. However, we've recently encountered some situations where we feel we need the flexibility to take action when certain ratings and reviews styles are counter to the welfare of the site. We really wanted to have something on the books so no one would be taken by surprise if we take this kind of action in the future.


Essentially, we believe there's a sort of "community consensus" or "baseline" for star rating scores, that's demonstrated through the average range of scores reviewers tend to give. What makes this challenging is that this isn't a solid number, but more of a general view pieced together from many reviews. This is not at all to say that you can't rate something zero stars if you think it's warranted, even if most people think it deserves five stars. This is targeted more at larger patterns, such as someone repeatedly rating many quests much lower than everyone else. You can have "high standards," of course, but past a certain point it starts to mess with the way the database ratings system works. For example, in an extreme case, imagine a reviewer deciding that by "professional standards," none of the quests here could hope to go over a 3-star rating, because without a high budget and dedicated team, they'll always be "amateurish." This would pull the ratings of every quest they played down, give an inaccurate impression of the quests' quality according to most members' expectations, and could hit the quest makers hard.

 

In cases like this, the staff may ask the reviewer to modify their scores, or if other options fall through, remove their ability to rate content altogether.

 

I know this kind of thing may sound alarming, and some may view it as a "slippery slope" that allows the staff to censor opinions they disagree with. However, I want to stress that most members will never have to worry about this rule at all. This is meant to be exercised only in extreme cases, and multiple staff members will always be involved in discussions on this to serve as a check and ensure we're being fair.

 

We also want to remind everyone that, as already stated in the rules, we expect criticism to be constructive and respectful. Again, harsh opinions are fine, but please keep the creators' feelings in mind and avoid reviews that could be taken as disparaging or cruel.


  • ShadowTiger, Shane, Yapollo and 5 others like this

#2 Deedee

Deedee

    Bug Frog Dragon Girl

  • Moderators
  • Real Name:Deedee
  • Pronouns:She / Her, They / Them
  • Location:Canada

Posted 26 October 2016 - 04:33 PM

...This rule sounds like an addition just to target one singular particular person. This rule just feels shoehorned in just to force one person to just their ratings. While I agree that having high standards can ruin the database, making it so that a person has to change their review style because they've never rated anything 5 stars before just sounds outright childish. This rule just sounds too specific for it to have any meaningful contribution to the site, unless the one person who this rule is targeting is ruining the site that much for others.

While it sounds like it has potential, I'd really want the rule to be expanded so that it's actually useful beyond handcuffing one person. I'm sorry, but that's my opinion.


  • Chris Miller, Rambly, thepsynergist and 5 others like this

#3 Sheik

Sheik

    Deified

  • Members

Posted 26 October 2016 - 04:34 PM

You could get around this entire thing if you told the system to exclude extremes when calculating averages. That's what a proper statistial analysis would do, too. And that way you could keep the ratings in the database, they just wouldn't affect the overall rating if they are too outlandish to be considered representative. You wouldn't even have to monitor it at all after you implemented the algorithm to take care of it.


  • Rambly, Mitsukara and Air Luigi like this

#4 Deedee

Deedee

    Bug Frog Dragon Girl

  • Moderators
  • Real Name:Deedee
  • Pronouns:She / Her, They / Them
  • Location:Canada

Posted 26 October 2016 - 04:37 PM

You could get around this entire thing if you told the system to exclude extremes when calculating averages. That's what a proper statistial analysis would do, too. And that way you could keep the ratings in the database, they just wouldn't affect the overall rating if they are too outlandish to be considered representative. You wouldn't even have to monitor it at all after you implemented the algorithm to take care of it.

Hmm, that sounds pretty reasonable. Ignore the occasional 1 or 2 5 star ratings in a quest that's consistently getting 0 stars, or ignore the 1 or 2 low ratings in a quest consistently getting 5 stars when calculating averages. I like this.



#5 Mani Kanina

Mani Kanina

    Rabbits!

  • Members

Posted 26 October 2016 - 04:50 PM

Alright, I just got done bumping up all my reviews Aevin, as requested. (wait, why do we not have a thumbs up smiley?? :S)



#6 Jamian

Jamian

    ZC enthusiast

  • Members

Posted 26 October 2016 - 04:52 PM

...This rule sounds like an addition just to target one singular particular person. This rule just feels shoehorned in just to force one person to just their ratings. While I agree that having high standards can ruin the database, making it so that a person has to change their review style because they've never rated anything 5 stars before just sounds outright childish. This rule just sounds too specific for it to have any meaningful contribution to the site, unless the one person who this rule is targeting is ruining the site that much for others.

While it sounds like it has potential, I'd really want the rule to be expanded so that it's actually useful beyond handcuffing one person. I'm sorry, but that's my opinion.

 

This decision may have been triggered by one person's recent reviews (I haven't been following this, so I don't know the "story" here), but this is an issue that has been debated many times before, and is likely to come back up now and then.


  • Shane likes this

#7 Aevin

Aevin

  • Members
  • Pronouns:He / Him
  • Location:Oregon

Posted 26 October 2016 - 04:57 PM

The staff isn't typically in the business of publicly explaining our interactions with members as we don't like to publicly shame people. But I doubt anyone doesn't understand Lunaria was affected by this, and she's welcome to explain the situation to her heart's content elsewhere. I will, however, say that the discussions about this rule have already happened. This shouldn't be taken to mean you all aren't welcome to bring up your concerns, but the rule itself isn't up for debate.



#8 TheLegend_njf

TheLegend_njf

    Deified

  • Members
  • Real Name:Grant

Posted 26 October 2016 - 05:07 PM

Well, in my opinion, it does feel like trying to control expression of opinion. People with these high standards are now in a position where they are breaking rules for having high taste. To me, that does feel like a form of censorship. But all the same, I find some people's standards to be crazy high and borderline impossible to please sometimes.

 

There's more than likely also quest designers who will probably rate other quests poorly to make their own quests better. This probably does exist and thus their behavior is a form of aggression in nature. I can understand questioning people's motives sometimes.

 

Still though, I'm just against this and I'll be honest about that. I won't make a big deal of it not only does it not really apply to me, but also because that's your choice how you want to manage this, but I just personally don't agree with this decision.

 

Sorry for being devils advocate here, but it's just a point I felt like adding to the conversation. 


  • Rambly and Eddard McHorn Van-Schnuder like this

#9 MarinaraSauce

MarinaraSauce

    Magus

  • Members
  • Real Name:Grant
  • Location:New York

Posted 26 October 2016 - 05:12 PM

As you said, this does seem like a "slippery slope" on paper, but I do have a lot of trust and faith in the staff of this site, so I don't anticipate any abuse of this rule barring any drastic changes to who's in charge. This will probably a case of "just don't be a dick and you won't get the hammer".


  • Shane, Evan20000, Aevin and 1 other like this

#10 Mani Kanina

Mani Kanina

    Rabbits!

  • Members

Posted 26 October 2016 - 05:18 PM

But I doubt anyone doesn't understand Lunaria was affected by this, and she's welcome to explain the situation to her heart's content elsewhere.

Eh? I thought we agreed that I could speak on this matter publicly, plus, this thread is the thread for this matter.


But yes, for those with any sort of misconceptions about this, I was actually the one who suggested this rule to be added in the first place. Given that I was contacted about this matter it was rather obvious

  • Eddard McHorn Van-Schnuder and Naru like this

#11 Aevin

Aevin

  • Members
  • Pronouns:He / Him
  • Location:Oregon

Posted 26 October 2016 - 05:21 PM

I just meant that I'd like to keep this thread focused on the rule change itself. If you wanna discuss your situation in detail, by all means make your own thread for it.


  • Mani Kanina and Naru like this

#12 Rambly

Rambly

    Hero of Time

  • Members

Posted 26 October 2016 - 05:52 PM

As you said, this does seem like a "slippery slope" on paper, but I do have a lot of trust and faith in the staff of this site, so I don't anticipate any abuse of this rule barring any drastic changes to who's in charge. This will probably a case of "just don't be a dick and you won't get the hammer".

"just don't be a dick and you won't get the hammer" make this rule seem redundant to me at best tho considering we already have an elastic clause
 

This shouldn't be taken to mean you all aren't welcome to bring up your concerns, but the rule itself isn't up for debate.

what's the point of allowing people to take up their concerns if their concern is, say, that the rule itself is bad?

 

i do not think one person having relatively low ratings on average does anything to harm the community, but whatever, this isn't a democracy, i suppose...


  • nicklegends, Eddard McHorn Van-Schnuder, Erdrick and 2 others like this

#13 Saffith

Saffith

    IPv7 user

  • ZC Developers

Posted 26 October 2016 - 07:36 PM

It seems to me it's a matter of maintaining the integrity of the system rather than suppressing opinions or attitudes. If your standards are such that nothing is worthy of a five-star rating, then you're essentially rating on a different scale than everyone else. In a database where many quests never even get ten ratings, that's a significant disruption.
  • ShadowTiger, Nathaniel, Air Luigi and 8 others like this

#14 thepsynergist

thepsynergist

    thepsynergist

  • Members
  • Real Name:Jeff Lee
  • Location:California, USA

Posted 26 October 2016 - 07:48 PM

Why not just likes and dislikes, like on Facebook?


  • nicklegends likes this

#15 Eddard McHorn Van-Schnuder

Eddard McHorn Van-Schnuder

    smash the bye button

  • Members
  • Real Name:Ronny Wiltersen

Posted 26 October 2016 - 08:12 PM

It seems to me it's a matter of maintaining the integrity of the system rather than suppressing opinions or attitudes. If your standards are such that nothing is worthy of a five-star rating, then you're essentially rating on a different scale than everyone else. In a database where many quests never even get ten ratings, that's a significant disruption.

@ Saffith: You're assuming 'everyone else' is using the same standards in their reviews, which is naturally impossible seeing as standards, as well as opinions about quests, are entirely subjective and as such they will vary from person to person.

 

@ Thread: However the most problematic thing going on here is that you have a group of people, elected by themselves to reside over things such as what another person's 'standards' are, as if they can read their mind and make such decisions on their own without any actual input from the person in question. Let me put it very simply: the staff has no business 'deciding' what Lunaria's standards are. I don't care what you think Lunaria's motivations may be, as long as they're not actively attempting to ruin somebody's day I don't think anyone has any business telling them what to do.

 

Staff is obviously free to do whatever they damn well please, that is not what I'm questioning here. And I don't doubt for a second that they genuinely believe that this is a good change, because it will absolutely relieve them of the extra headache of having to deal with problems when they come up - such as people complaining about somebody's reviews. Staff's goal here is probably to try to reduce the kind of involvement they have in those cases, by making sure they can swiftly make a change and thus move on. I get that, I've been a moderator here as well so I have seen these kinds of discussions go on first hand. And I think it's commendable that they're trying to make things better for everyone, but when it comes to this one it's based on twisted logic and I think the echo-chamber nature of the staff forums has ensured that no matter what we as users have to say about this, they will have made up their minds on their own. Which is their right as staff, but that does not make it right.

 

I don't think censorship is the biggest problem here. That would be that this change was made because staff has received complaints about Lunaria's reviews. They wanted to have a reason to be able to say that she had to change, and since she wasn't actually breaking any rules, they ensured that there was a new rule for her to break. As such I find it odd that they did not elect to remove Lunaria's reviews entirely, and instead settled on asking her to change all the ratings instead. Especially considering I have had staff members personally tell me that Lunaria's reviews were 'hostile', and with the logs between Lunaria and Aevin up there it certainly seems that the issue here was more than just the stars tied to the reviews. And let's not forget that the rating itself is also subjective, it is an entirely personal choice, and now staff has in a way taken that subjectivity away from Lunaria's reviews, effectively censoring an opinion - despite not having changed any of the actual wording. 

 

Censorship is of course part of the issue, and it doesn't make it look good when you have staff members making promises about this not being used outside of extreme cases, when we now have actual, hard evidence that proves that they have already used this rule to intervene in a situation that in no way can be described as extreme - I've read Lunaria's reviews and there is nothing that implies there are any ulterior motives on her end for posting those reviews, other than to simply share her own opinion. Claiming otherwise is a lie, a series of low ratings does not suddenly become evidence of a trend that implies the reviewer is trying to cause some sort of damage. That is nuts.

 

Staff, I know we haven't had the best of times lately, but like I've asked of you in the past, please try to look at this from a perspective other than that of a staff member. I don't want to sit here and point fingers and call bad guy, I know that isn't true, and like I've said I am fairly certain that you're being genuine about trying to make things better. However it is misguided, and I think you've become increasingly blind to issues like this. If you're not going to reconsider the change, at least consider what I'm suggesting regarding the way you go about these things. It isn't good for you, and it isn't good for us.


Edited by Robin, 26 October 2016 - 08:13 PM.

  • Chris Miller, Rambly, Sheik and 1 other like this


1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users