## uDG08 Voting Results and Methods

Moderator
Posts: 684
Joined: 2002.11
Post: #76
I am against it because it is completely stupid for all the reasons Seth mentioned, and because you keep saying stuff like this:
Quote:Why should one vote outweigh the others?
If you think about that question for more than five seconds, then you might see why I disagree with you.

My web site - Games, music, Python stuff
Moderator
Posts: 625
Joined: 2007.09
Post: #77
diordna made a point, you know.

The average includes all of the votes. None of the votes are thrown out.

Why is this not fair? In your first example:
Code:
```p1 scores: 3,5,5,5,6,6,6,7,7,7,7,7,7,7,7,7,7,8,9=6.47 p2 scores: 5,5,5,5,5,6,6,6,6,6,6,7,7,7,8,8,8,8,9=6.47```

This is fair. All votes are taken into account.

From what I can see, your problem with the average seems to be that it is affected by all the votes; one vote can affect a game's rating. This being the case, it would seem as though your definition of 'fair' would be to eliminate voting at all and having one person rate all of the games.

I prefer the average.

- Lincoln Green
http://www.binkworks.com/
Moderator
Posts: 684
Joined: 2002.11
Post: #78
Lincoln, you continue to amaze me with your simple logic and wit. You stated my argument better than I did.

Also, you stole my awkward sentence for your sig, you scoundrel.

My web site - Games, music, Python stuff
Member
Posts: 65
Joined: 2008.09
Post: #79
Quote:
Quote:Why should one vote outweigh the others?
If you think about that question for more than five seconds, then you might see why I disagree with you.
If you try to argue against that for more than five seconds you'll realize your wrong.

Quote:The average includes all of the votes. None of the votes are thrown out.
Which is the problem if someone nerfs you!

Quote:Why is this not fair? In your first example:
Code:

p1 scores: 3,5,5,5,6,6,6,7,7,7,7,7,7,7,7,7,7,8,9=6.47
p2 scores: 5,5,5,5,5,6,6,6,6,6,6,7,7,7,8,8,8,8,9=6.47

This is fair. All votes are taken into account.
Clearly the majority gave p2 lower scores (5's & 6's), yet there were nerfed 8's and the mean reflects that.
p1 really deserves a 7 because a vast majority thought so. If the person that nerfed him with a 3 were a 10 his score would change to 6.84, that's a .37 difference. That was one vote, and it made a huge impact on his score. (more so than any other vote, do the math)

Seriously, has no one here taken a course in discrete mathematics?

Why do I bother really, try telling the judges of a sporting event to use your method. You're probably the ones doing the nerfing.
Moderator
Posts: 684
Joined: 2002.11
Post: #80
It's not "nerfing" if I just didn't like your game, okay? That's why we are so confused by your arguments about it. You seem to be dead set on excluding people who have different opinions, when those different opinions are the whole point of voting at all.

I gave you the chance to think about my question, and you didn't, so I'll spell it out for you: if someone gets 7, 8, 8, 8, 8, then that person deserves a lower score than the guy who gets 8, 8, 8, 8, 8. It's as simple as that. You seem to think that those two people deserve the same final score, which is totally ridiculous.

Oh, and this ad hominem crap doesn't help anyone, so give it a rest.

My web site - Games, music, Python stuff
Member
Posts: 283
Joined: 2006.05
Post: #81
tcIgnatius Wrote:Once again, why should one vote outweigh the others? the majority felt it deserved an 8, one person felt it deserved less then 8, so it gets less then 8?

Yes. The game didn't please everyone, so it has a lower score than if it had.

I think using the median would be crazy. Inter-quartile mean sounds like a much better idea, but I'm not sure if I prefer it to the current system.

EDIT: You might be more persuasive if you could show some examples of similar competitions that have successfully used the median in the past. I just did a quick search and the only game competition I could remember was the IGF, which according to the rules uses the "average" (which I think means mode, but I didn't completely follow some of the wording).
Moderator
Posts: 3,591
Joined: 2003.06
Post: #82
Heh... I know this discussion is long in the tooth, but I can't help but say your guys' examples are actually a little conservative compared to what happened with one game in the actual contest. The votes looked more like 1,4,4,5,5,5,5,5,6,6,7,7,9 (made up numbers, but illustrative). The one was not only in just one category, but was given 1's by the same judge in all that game's categories. It stuck out like a sore thumb to me when I was auditing the votes. When I said I tested to see if that changed the outcome of the contest, that was one of the games in particular that I investigated. It had very little, if no significant effect at all, if I pulled out that all 1's vote.

Again, I don't know if it was because that judge was mad at that game, or if they mistakenly voted that way, or what, but in the end it counted, and didn't significantly affect the outcome. Do realize that this was not something which occurred much in the contest. Maybe only three games, but they were all fairly mid to low scorers in the first place.

This is why DoG brought up the idea of the truncated mean (pulling off the top and bottom 10 or 20 percent). tcIgnatius is advocating that too, but 25% which is interquartile, (plus he wants to add a median calculation on top). They put truncated means in stuff like skating so that judges, who might strategically sink other teams because they happen to be their coach, get cut out. I think in the example I'm talking about, that is what actually happened, or voter error. The truncated mean would help reduce both issues.

So there is sense to the argument of a truncated mean IMHO.
Moderator
Posts: 1,563
Joined: 2003.10
Post: #83
tcIgnatius Wrote:Why do I bother really, try telling the judges of a sporting event to use your method. You're probably the ones doing the nerfing.

Please keep the discussion civil and refrain from making personal attacks. If this continues, we'll have to start moderating the discussion more heavily, or lock the thread.
Member
Posts: 65
Joined: 2008.09
Post: #84
Quote:It's not "nerfing" if I just didn't like your game, okay? That's why we are so confused by your arguments about it. You seem to be dead set on excluding people who have different opinions, when those different opinions are the whole point of voting at all.

Yes, in an idealistic world there wouldn't be nerfing. With competitors voting for each other it seems like things could get incredibly political. A different opinion is fine, but with the mean, the single opinion counts more then the majority. (this is not difficult math)

Quote:tcIgnatius is advocating that too, but 25% which is interquartile, (plus he wants to add a median calculation on top).
no, if we did just the interquartile mean, i would be fine with that. I was just saying that the special case median I had purposed (which, for gods sake is not just a median) would have been very fair.
Moderator
Posts: 484
Joined: 2008.04
Post: #85

Howling Moon Software - CrayonBall for Mac and iPhone, Contract Game Dev Work
Moderator
Posts: 625
Joined: 2007.09
Post: #86
AndyKorth Wrote:I feel like this thread might benefit from this:

http://en.wikipedia.org/wiki/Median
http://en.wikipedia.org/wiki/Arithmetic_mean

Heh, yeah.

But apparently, tclgnatius is referring to a special case median.

- Lincoln Green
http://www.binkworks.com/
Moderator
Posts: 3,591
Joined: 2003.06
Post: #87
tcIgnatius Wrote:no, if we did just the interquartile mean, i would be fine with that. I was just saying that the special case median I had purposed (which, for gods sake is not just a median) would have been very fair.

I think we might actually be getting somewhere. But I need to back up to where you were describing this. You used this as an example:

Quote:ex g1: 3 3 5 5 5 5 7 7 7 7 9 9 10 = (5+7)/2 = 6
g2: 3 3 5 5 5 5 8 8 8 9 9 10 10 = (4(5) + 3(8) )/7 = 6.28

So as I now gather it, these values are after raw vote truncation (let's say interquartile since that's what you prefer, but it could be any arbitrary percentage off the top and bottom in reality, correct?).

Now, examining this: ex g1: 3 3 5 5 5 5 7 7 7 7 9 9 10 = (5+7)/2 = 6

But 6 is not the median, it is the average of the two most frequently occurring scores (the average of two modes, so to speak). The median would actually be 7. But you don't want the median right?

Okay, so then in the next example: g2: 3 3 5 5 5 5 8 8 8 9 9 10 10 = (4(5) + 3(8) )/7 = 6.28

Now we're giving weight to the two most frequently occurring scores by taking the average of all occurrences of those two most frequently occurring scores. At least that's how I'm reading your example. 5 and 8 are the two most frequently occurring scores and there are four 5's and three 8's.

So am I following at all here?
⌘-R in Chief
Posts: 1,277
Joined: 2002.05
Post: #88
tcIgnatius Wrote:G3. Is WRONG there are three 7's and two 8's and two 9's that means under the special case (the diff in occurrence of the sub-medians < 2) you would say (8+9)/2 = k
(3(7) + 4(k))/7 = 7.85
G2 is RIGHT, and it should be.
G1 is correct, and why did only five people vote for such a high scoring game?

Oh good grief. No special case should be involved

So G3 wins with an 8.5 despite 70% of the voters thinking it deserved a *lower* score? And you somehow think that's fair? I thought you were the one trying to make "every vote weight the same" and yet here you are ignoring the majority's opion which you are supposedly defending. You're contradicting yourself.

Quote:Once again, why should one vote outweigh the others? the majority felt it deserved an 8, one person felt it deserved less then 8, so it gets less then 8?

OF COURSE. Someone didn't like it as much! Why it should it get the same score as a game that more people liked. It's absurd!

If you actually believe this is wrong, then there is no point in arguing with you. Everyone else including contestants believes this is *not* fair.

Goodbye. End of discussion for me. I can't continue trying to convince you why this doesn't make sense if you fundamentally believe something everyone else doesn't.
Member
Posts: 65
Joined: 2008.09
Post: #89
Quote:So as I now gather it, these values are after raw vote truncation (let's say interquartile since that's what you prefer, but it could be any arbitrary percentage off the top and bottom in reality, correct?).
yes, as long as that arbitrary percentage is above or equal to 25%.

Quote:Now, examining this: ex g1: 3 3 5 5 5 5 7 7 7 7 9 9 10 = (5+7)/2 = 6

But 6 is not the median, it is the average of the two most frequently occurring scores (the average of two modes, so to speak). The median would actually be 7. But you don't want the median right?
Right, another example of the special case, where the split between the occurrence of the two largest sub-medians (four 5's and four 7's) is less then two in difference. Here there difference is 0, as they both occur four times.

Quote:Okay, so then in the next example: g2: 3 3 5 5 5 5 8 8 8 9 9 10 10 = (4(5) + 3(8) )/7 = 6.28

Now we're giving weight to the two most frequently occurring scores by taking the average of all occurrences of those two most frequently occurring scores. At least that's how I'm reading your example. 5 and 8 are the two most frequently occurring scores and there are four 5's and three 8's.
Right, now seven votes are the majority. This is fair. If you don't agree maybe we should post the raw data without anonymity .

Here's a situation to help illustrate:
10 people vote for a game in the category most original.
8 give a score of 9
1 gives a score of 7
1 gives a score of 1
Now, imagine those people discussing the vote. The mean would say that game got an 8 in originality. Everybody who voted 9 is a little confused as to why this is. One of the voters who voted 9 goes around the room and asks others how they voted. He asks the seven other votes for 9 and still wonders how this game got an 8. He talks to the person who gave it a 7. The person who gave it a 7 claims that its not all that original, the seven who voted 9 agree this is an original concept and the one who voted 7 admits to being a bit more critical on the topic of originality. The 9 voters then say "well, ok, so it sounds like the mean should have been 8.7", although the majority thought it was better than that. Now they look at the person who voted 1. "Why did you give this game a 1? This isn't a remake or anything close to that." The 1 person explains any of the following:
it was an error
I didn't think it was original
I didn't play it.
I tried to strategically sink it.

You see it doesn't really matter. The game didn't deserve an 8, it deserved a 9. A review should reflect the majority opinion. When we have so few votes, it's really not fair that one person can change the score. (and have more weight the further it deviates from the median value)

If you do not see my point now. You are still stuck on thinking that the mean is more fair, only because it does intuitively seem like it would be. Please assume that it is not fair, then come up with a contradiction. This is how you prove something. It's called proof by "modus tollens" for any logic geek. If you try to prove that the mean is fair through an example, you should realize that there is no proof in that. Proof by example is also called "no proof at all".

Quote:So am I following at all here?
You appear to be the only one here who is.

Quote:So G3 wins with an 8.5 despite 70% of the voters thinking it deserved a *lower* score? And you somehow think that's fair? I thought you were the one trying to make "every vote weight the same" and yet here you are ignoring the majority's opion which you are supposedly defending. You're contradicting yourself.
G3 didn't get an 8.5, how did you get that?

Quote:OF COURSE. Someone didn't like it as much! Why it should it get the same score as a game that more people liked. It's absurd!
Read the other possibilities in the situation I created above.
Moderator
Posts: 3,591
Joined: 2003.06
Post: #90
Took long enough! I *finally* see what you've been trying to get at...

I had to read verrrrrry carefully over all the posts in this thread again (like what, 70 posts?) and try to extrapolate some of your ideas over many posts to put it together though. I have to say you didn't explain it anywhere near as clearly as you thought you did. And of course, tiny mistakes along the way here and there did not help at all, but those are often unavoidable so they come with the story. However, in retrospect, I think you could have taken a more patient and explanatory tone, rather than a defensive and somewhat accusatory one. I don't know about others, but I *do* listen, even if I don't agree on face-value. When you're in the minority viewpoint and trying to explain things to a small mob, you have to take extra caution in your communication. In this case you got through, but not without all this jibber-jabber we just went on about. I'm glad we got here though.

Alright, so now that I can see where you've been trying to point the whole time I can see how it might be a better system. I'm not saying I'm convinced that it *is* the better system, but it *might* be the better system.

I suppose I could help your argument a little by saying that indeed, if a person were to truncate any scores whatsoever, you're already dealing with median scores which are no longer technically an average. So once you're dealing with median scores, one could argue: "Why pretend they're a representative average anymore? They're now medians, so you might as well refine them as medians."