97 replies to this topic

[Jared]

I don't have too much to say, but I don't see an issue. I don't see this as censorship. The rule is basically saying we should be using tactful ways to explain what we mean in reviews. That's all.

 

You get kicked out of a store for being rude and disrespectful. Why should PZC be any different? You can learn to get your point across without hurting people.

 

Why do you think ZC activity on Pure is getting lower in the first place? People are getting too harsh. In reality, no one likes their feelings hurt. I don't see why this is such a hard concept to grasp.

Edited by Jared, 27 October 2016 - 05:12 PM.

[Eddard McHorn Van-Schnuder]

Oh come on now ... My point is that there's options for people to leave feedback without leaving a score if they so wish. This is a convenience for those that don't want to leave reviews, not any sort of demand toward a particular person. And frankly, I think you all know that. Stop twisting everything I say to reflect as badly on the staff as possible. I can understand everyone's concerns here, but some of you are being rather silly.
 
I'd also like to point out that the rule addition doesn't exactly represent a change in policy. In fact, I think the existing rules allow for situations like these. The rule is intended to increase clarity and better indicate some situations in which we may take action. It's not just for the staff, it's for you members, because we consider communication of our intentions to be important. Some of you are asking for increased transparency from the staff, but do nothing but criticize when we attempt it with the most honest and well-meaning of intentions. Honestly ...
 
In any case, I've said my piece. You all are welcome to respond to this post, tear it apart, whatever. Go to town, you guys, if that makes you feel good. I won't be replying again, and I'd advise my fellow staff members to do the same.

Aevin, no one is twisting your words, or anybody else's. We are reacting to not only the actual events that have taken place here, but also the language you guys are using. That one is on you, don't try to shift the blame over on those of us who doesn't agree with your view of this situation. In fact, I think it's pretty problematic that a staff member is reacting the way you're doing here, while at the same time you're trying to suggest that what Lunaria have been doing is somehow 'hostile' and disruptive to the database. The logic doesn't work.
 
I have said several times that I trust that you guys really believe that this is a good thing, I am not questioning your honesty (though I do, and think we all should take it for granted. I do appreciate it of course.), but I am still criticizing the change you have decided to make. I don't think it's fair of you to just shut down any criticism just because it gets a little bit uncomfortable, especially when you've already opened a thread about the issue and allowed a discussion to take place. Which again I think is great that you did that much, where as in the past these kinds of announcements have sometimes been posted as a locked thread - or been locked when things got a little heated. But saying you're not going to take part in the community that you're supposed to be residing over just because the criticism is getting uncomfortable is in a way the same as saying that what the user's think doesn't matter to staff as a whole. Not saying you actually said that, but that is the impression you're giving me at least.
 

I don't have too much to say, but I don't see an issue. I don't see this as censorship. The rule is basically saying we should be using tactful ways to explain what we mean in reviews. That's all.
 
You get kicked out of a store for being rude and disrespectful. Why should PZC be any different? You can learn to get your point across without hurting people.
 
Why do you think ZC activity on Pure is getting lower in the first place? People are getting too harsh. In reality, no one likes their feelings hurt. I don't see why this is such a hard concept to grasp.

Are you seriously suggesting that 'ZC activity' in general is actually getting low because people are harsh? Jared, that doesn't hold up. If Lunaria is the only person that made this an issue, are you then saying she is the cause?! I won't pretend to know why there are less quests being made these days, or if there even is, however suggesting that it is because people don't like being told that not everyone in the freaking world loves the thing their made... that doesn't seem right.

Edited by Robin, 27 October 2016 - 05:27 PM.

[Mani Kanina]

Oh come on now ... My point is that there's options for people to leave feedback without leaving a score if they so wish. This is a convenience for those that don't want to leave reviews, not any sort of demand toward a particular person. And frankly, I think you all know that. Stop twisting everything I say to reflect as badly on the staff as possible. I can understand everyone's concerns here, but some of you are being rather silly.

Though, the point was that I wanted to leave a review? And how is it not any sort of demand towards a particular person? I mean, I was specifically instructed to change how I write any future reviews.... So it basically is a demand against a specific user: change your ways or go elsewhere with them.

I don't think anyone in this thread is sitting and cackling "wow, time to fling some shit at staff, lol". The only thing I see here that reflects badly on staff is their own actions, maybe you don't agree with peoples complaints and views? Fine, I can respect that. But coming across as that you're somehow the victim because people happened to not agree with your views? That's silly.
 

I'd also like to point out that the rule addition doesn't exactly represent a change in policy. In fact, I think the existing rules allow for situations like these. The rule is intended to increase clarity and better indicate some situations in which we may take action. It's not just for the staff, it's for you members, because we consider communication of our intentions to be important. Some of you are asking for increased transparency from the staff, but do nothing but criticize when we attempt it with the most honest and well-meaning of intentions. Honestly ...

Indeed, this is not at all a change in policy, if anything this rule just makes it public what was already a thing happening. And you're right, this increase in transparency is great and I honestly think it's beneficial for all parties. But really, I think the wrong conclusion to draw is that transparency = bad, based on this. If anything, if making matters like this public makes things get a bit heated maybe it points toward that users actually don't like these practices as a whole? Food for thought if anything.

I also find the statement that staff cares about communication your intentions to be laughable. It took an outside source to suggest this being made into to a rule rather than a backroom deal between staff and individual users. Furthermore, the matter was not once publicly brought up before the rule hit. Like I said in my PM to you when you sent your draft to me, showing something to one user is not transparency.
 

In any case, I've said my piece. You all are welcome to respond to this post, tear it apart, whatever. Go to town, you guys, if that makes you feel good. I won't be replying again, and I'd advise my fellow staff members to do the same.

I mean, you did reply to my post, made a hill out of a rather pedantic remark, and then ignored the rest. Which did come across as doing just that yourself.

Furthermore, it's fine to criticise users but staff is above getting criticised for their actions and choices? :s

[Jared]

I'm not just talking about the reviews. I'm talking about in general. But I know how you are, Robin. Once you're fixated on a certain opinion you wont' get off of it and always fight for it. So, I won't try to push.

[Eddard McHorn Van-Schnuder]

I'm not just talking about the reviews. I'm talking about in general. But I know how you are, Robin. Once you're fixated on a certain opinion you wont' get off of it and always fight for it. So, I won't try to push.

It hurts that you believe that.

[Rambly]

Oh come on now ... My point is that there's options for people to leave feedback without leaving a score if they so wish. This is a convenience for those that don't want to leave reviews, not any sort of demand toward a particular person. And frankly, I think you all know that. Stop twisting everything I say to reflect as badly on the staff as possible. I can understand everyone's concerns here, but some of you are being rather silly.

 

I'd also like to point out that the rule addition doesn't exactly represent a change in policy. In fact, I think the existing rules allow for situations like these. The rule is intended to increase clarity and better indicate some situations in which we may take action. It's not just for the staff, it's for you members, because we consider communication of our intentions to be important. Some of you are asking for increased transparency from the staff, but do nothing but criticize when we attempt it with the most honest and well-meaning of intentions. Honestly ...

 

In any case, I've said my piece. You all are welcome to respond to this post, tear it apart, whatever. Go to town, you guys, if that makes you feel good. I won't be replying again, and I'd advise my fellow staff members to do the same.

I can't speak for everyone, but I'm not accusing the staff of ill intentions.  A lack of foresight?  Sure; I don't think the rule's implementation or the announcement had a lot of foresight behind either of them, and I don't think all of the consequences were fully anticipated.  I know the rule wasn't created to single out a certain user, but I still think it likely won't get used again, because the original situation was such a weirdly specific one...

 

I appreciate the increased transparency, but probably the main reason people want transparency is so people can offer critique and alternative points of view.  It's going to be a side effect of increased transparency, no matter what.  I'm personally going to criticize things if I think they're harmful to the community, whether or not they were intended to be harmful.

[Deedee]

I won't pretend to know why there are less quests being made these days

A bit off topic, but I believe the reason why there are less quests being made is because people actually have to put significant time and effort into projects now. You can't just make a reskin of the first quest using the BS Tileset and the same bland dungeon design 1st.qst used. Well, at least without getting 3 stars or so. When a person picks up a quest, they want it to be worth their time. They want it to be something special. If it's not something special, and it was a waste of their time, then that quest is going to get poor-average reviews. Likewise, if it turns out to be something special, or at least fun enough to justify the time put into it, then it will likely get high marks. With the addition of scripting, the enemy editor, 4 warps, and the like, the standards of a "good quest" have risen. Why should I play something like BS Zelda 1 Remake #42 when I can play something new, creative, and/or fun like Engage to Zeldawock, or Freedom in Chains, or Lost Isle? If people want to have a quest that's liked and won't be quickly forgotten, then they have to make it stand out, which means more effort, which means more time spent on it, which means less quest releases per year. It has nothing to do with reviewing standards, it's just the quest standards themselves which have changed.

Also, why the bickering? Even I try not to let my grudges with others affect my interactions with others on the forum, and I'm not even a good role model! Can we please at least try to keep this friendly so this topic doesn't get locked? D:

[Magi_Hero]

Where is the increased transparency that was spoke of? I would love to see this and give my honest opinion as a member of this community for the past almost 14 years.

 

I believe we're adding too many ingredients to the pot and it's spoiling the soup. I am seeing so much major disdain for the staff's decisions in recent times. I have seen favoritism. I have seen so much people putting words in each other's mouths. I have seen blatant disregard for feelings. Ganging up. The whole lot!

 

How about my feelings?

 

I'm disgusted we're at this spot that we're so petty to each other. You're all guilty of this. Fix this now or there won't be a future for PureZC. Don't kill one mans dream of a place he made to openly express himself and for others to do the same.

 

This is shameful at most.

[LinktheMaster]

So, after some consideration, I've come up with a compromise that I am hoping to put into place in the coming weeks. This issue and related problems such as people using reviews as a method of revenge have come up multiple times in the past. It's an issue that has divided the community for a long, long time, and I absolutely hate seeing something fracture everyone like this.

 

Despite everyone's disagreements, we all need to remember that we have the common bond that we're all here because of Zelda Classic, and the reason we're all passionate about this is because we enjoy making and playing quests. The reason we're all here and discussing this is because we all honestly want an environment that is, to the best of our abilities, best for everyone. While there are obviously differences in the thoughts on what that is, I think the string that ties us all together fundamentally brings us closer than the tears which things like this create. There's really no reason for any vile behavior in regards to all this. We can all discuss this calmly without accusations of bad intent, putting words in each others mouths, or acting like the other party is stupid for their beliefs. None of that stuff helps either make a decision or bring the community together.

 

Everyone lashing out and piling on just one or two people isn't the right way to request change, either. I know the rating system is dear to everyone's hearts, but I don't think we need multiple people grouping up to pick apart a single person's statements. Friendly discussion and disagreements about things like this are fine (and somewhat expected since I think it's foolish to expect everyone to like something), but I really don't think the way this topic has run its course was the proper way to do this. I think it's much better for all of us to come together civilly to come up with a solution that most can agree on.

 

Which comes back to my proposal of a compromise.

 

1. Reviews and ratings will be split

Basically, people will be able to rate quests without writing a reviews, and people will be able to write a review without assigning a rating. My hope is with this change, it will promote a higher number of ratings as they'll be easier to make while also raising the quality of the average review as people who will be writing them won't be writing them just so that they can make a rating.

 

2. Ratings will stay as a 5 star system

I still think this is the best middle ground. We've had some people in the past ask for a base 10 system, or even a base 100 system. So, I don't think everyone necessarily disagrees with the base 5 system in the same way. Any conversion of the old rankings would be wonky, and with it not being a clear near unanimous decision it being better, I don't think it's worth the sacrifice of correctness of the old reviews.

 

3. Average ratings won't be affected by outliers as much

To compensate for us using the 5 star system, I'll try to come up with a new algorithm that will minimize the impact of outliers. For quests with large number of reviews, this means that some ratings will not be factored into the average rating. For quests with fewer reviews, it'll probably mean that outliers will be less impactful on the rating.

 

4. A new sorting category will be added (at least for quests)

To either supplement or replace the database sorting on ratings, I'm going to come up with a system that will instead sort entries on popularity. This will include several different data points, not just ratings. Not that ratings will be ignored, but they won't be the sole deciding factor in the ranking. It will be similar to the trending quests, but factor in all historical data rather than simply current numbers.

 

5. The new rule will stay in place

While I do understand the concern regarding the rule, we have had people in the past use ratings as a means to an agenda. Be it as revenge for a completely unrelated comment, as an statement against the system in general, or because they want to have control over the list of top rated quests. We've also seen people who act vile and bash quests with little to absolutely no constructive criticism. You may think it's absurd that this would happen, but we have unfortunately seen these sort of things multiple times. These things simply hurt the community as a whole. they remove any value of ratings and they can dismay quest makers who see bad reviews but don't know how to improve themselves. I don't think it's unreasonable to have a rules in place to prevent any sort of abuse of ratings, even if such a thing almost never happens.

 

I know that this won't please everyone. Heck, I'm not sure if it'll please anyone completely. But, I do think it's a bit of middle ground with not necessarily scrapping the entire current system, but addressing some of the biggest concerns with it.

 

I'm hoping to have the changes completed in a week or two.

[Rambly]

I'm pretty pleased with these proposed changes, actually.  I'm especially happy that there's been a sincere attempt made to address the concerns that were brought up, and I think everything'll work out pretty well in the end.

[Sheik]

I agree with Rambly.

As for the changes: I would also give people a hint what it is that they are rating. Rating something with "stars" is not a thing if it isn't at least somewhat clear what giving a "star" (or zero or five) actually denotes (if it denotes "excellent" or "horrible" it is still unclear what quality is being rated as being either excellent or horrible). There seems to be some vague idea of what that is (the gut feeling, maybe even the consensus) but it would help to add a short explanation. (This would warrant a whole other discussion of what it is that is being rated in a Zelda Classic quest which would in itself make an interesting topic of discussion).

It would also help if it was clear whether the rating is relative to some absolute gold standard (and which one would that be? "whatever I like best" - whatever that would be; Zelda 1? the most favorably rated quest alltogether?) or against some kind of average (and again: what is the average quest?). ...well, food for thought, maybe.

[Avaro]

You could get around this entire thing if you told the system to exclude extremes when calculating averages. That's what a proper statistial analysis would do, too. And that way you could keep the ratings in the database, they just wouldn't affect the overall rating if they are too outlandish to be considered representative. You wouldn't even have to monitor it at all after you implemented the algorithm to take care of it.

 

 

3. Average ratings won't be affected by outliers as much

 

I'm not sold on this. I don't quite understand why the impact of outliers should be lowered. Isn't this like saying "You think different so your opinion doesn't matter"?

[LinktheMaster]

I'm not sold on this. I don't quite understand why the impact of outliers should be lowered. Isn't this like saying "You think different so your opinion doesn't matter"?

It's more of a balancing act. If something has a hundred ratings of 4-5, and someone rates it as a 1, should that cause the ranking on site to go down? I would argue that something with that many ratings should still be ranked above something that has much fewer ratings. Also, thinking on it, I don't believe I will remove outliers when calculating averages, just make them impact the average ranking less.

 

There is still the rating distribution where people can individually see how many ratings of each category something got, and I do not plan on getting rid of that (though I may make ratings anonymous). In fact, with something like this, I would plan on making it more accessible, either shown on the database page itself or on a fly up when hovering over the rating.

[Eddard McHorn Van-Schnuder]

Will Lunaria be allowed to set her ratings back to what they were, and if possible remove them from her reviews should she want to, assuming that's how this would work?

 

Good to see that I was wrong about what I said earlier regarding staff input in this thread, and I'm glad to hear you're taking the community's response to heart. Props!

[Mani Kanina]

Firstly I'd like to say that I agree with practically all you're saying LtM, including your suggested solution. There are just some tidbits that I'd like some clarifications on:
 

Everyone lashing out and piling on just one or two people isn't the right way to request change, either. I know the rating system is dear to everyone's hearts, but I don't think we need multiple people grouping up to pick apart a single person's statements. Friendly discussion and disagreements about things like this are fine (and somewhat expected since I think it's foolish to expect everyone to like something), but I really don't think the way this topic has run its course was the proper way to do this. I think it's much better for all of us to come together civilly to come up with a solution that most can agree on.

While I certainly agree that this wasn't a good way to irk for a change, I would like to point out that it's not like there were any other options. A discussion thread for this rule proposal was not given in advance and the rule itself was brought forward and presented with a large sense of finality.
 

5. The new rule will stay in place
While I do understand the concern regarding the rule, we have had people in the past use ratings as a means to an agenda. Be it as revenge for a completely unrelated comment, as an statement against the system in general, or because they want to have control over the list of top rated quests. We've also seen people who act vile and bash quests with little to absolutely no constructive criticism. You may think it's absurd that this would happen, but we have unfortunately seen these sort of things multiple times. These things simply hurt the community as a whole. they remove any value of ratings and they can dismay quest makers who see bad reviews but don't know how to improve themselves. I don't think it's unreasonable to have a rules in place to prevent any sort of abuse of ratings, even if such a thing almost never happens.

By this extent I assume that the actions taken against me will still stand? Including both the changes I have done to past ratings, any future ones, as well as how I write any new reviews.

By all means, the reasons for this rule that you state are completely reasonable. People who use ratings as a hidden agenda or as revenge for something completely unrelated, or as a statement against something unrelated, are indeed using the rating system incorrectly. It's not like I don't see this happening, people are more clever with it these days, but I have seen people in the past even openly saying that they are rating things lower then they believe in order to lower the average. But I find that neither of these cases to applicable on me specifically. (And I was the one who was intentionally targeted with the rule, that was sorta, part of the point).

I guess there could be those who would argue that I offer no constructive criticism in my reviews, and to that I strongly disagree. I don't sit around bash things with how bad they are (and in those cases where it's debatable that I do so I would argue that the entire review section is already full of vapid reviews that I'd say bash it just as much), rather, I try to point out specific things that I find bad in quests. This means that if the author agrees with my critique (which is not a given), they can go in and change that thing. I try my best to give as much information as possible, but yes, I don't always provide solution examples. I have come to realise that they are seldom wanted. Even if they author agrees on that there is a problem, the solution ideas they want is practically always different.

It's kind of a moot point at this stage anyway, it was already agreed on that my reviews needed to go before I was even taken in into the discussion. That's the fact that I had and did accept when I was first approached about the issue, because let's be honest, I wasn't really given an option. But that does not mean that I don't find it completely unreasonable, nor that I won't criticise it.
 

I'm not sold on this. I don't quite understand why the impact of outliers should be lowered. Isn't this like saying "You think different so your opinion doesn't matter"?

Because outliers are by definition far above or below the average. (And I'm not certain I apply for that in most situations). Since they are few (see 1/20 (at least)) who have such a different view then they are not really relevant to the consensus. It's more relevant to get the proper average out of everyone except the outlier(s) than with the outlier, since that average will better represent the consensus.

The biggest problem is figuring out how far off something has to be for it to be consider an outlier. If a quest only gets 5s, 4s, and 3s, does that mean that a new review with a 2 is an outlier? What about a 1? Given that a 0-5 scale don't have that many choices care will have to be taken I'd say. That being said though, I don't think this will be a change that will do a whole lot. Quests with less than 20 ratings probably can't accurately determine outliers, even more so if the quest has 10 or less. Most quests in the database don't get huge amounts of reviews/ratings though. Maybe that will change when you don't have to write reviews though? Hmmmm.