## uDG08 Voting Results and Methods

⌘-R in Chief
Posts: 1,276
Joined: 2002.05
Post: #31
Quote:I don't personally feel that median votes are somehow more fair or representative of what a game deserves. But that's just my opinion.

It's not an opion, it's a fact. If game A has ten scores as 8, and ten scores as 7, and game B has twenty scores as 8, obviously they are not tied.

And no, using the mean of other categories to determine tie breaking in a specific category is not a good idea. You do not choose the winner of the audio category based on how good someone's story was. That's just silly.

The median is not an acceptable means of determining rank, but it is useful for determining outliers in a distribution. Instead of arbitrarily picking a top and bottom percentage to ignore, you could ignore only scores that are "too far" from the median - the outliers. "Too far" is a subjective, but for example, you could ignore results more than 3 away from the median. If the median is 7, ignore anything but scores of 4, 5, 6, 7, 8, 9, and 10.

--

Also, we could increase the range of values. Instead of just 1 to 10 (which is ok with a small number of entries), in order to let voters have more flexibility with ranking the games they played we should probably double the range. IOW, there were two games I liked almost exactly
as much as each other, but I thought one was a bit better. Therefore I want to give it a higher score, but a whole point is a pretty big advantage when there are such few votes.
Member
Posts: 65
Joined: 2008.09
Post: #32
Quote:It's not an opion, it's a fact. If game A has ten scores as 8, and ten scores as 7, and game B has twenty scores as 8, obviously they are not tied.
That was a special case and I had already stated that Game A would have a 7.5 if this were the case.

Quote:And no, using the mean of other categories to determine tie breaking in a specific category is not a good idea. You do not choose the winner of the audio category based on how good someone's story was. That's just silly.
I don't think it's silly. If two games are scored the same in graphics then the better game should get the vote. I'm not singling out specific categories either, like in your example.

Quote:The median is not an acceptable means of determining rank, but it is useful for determining outliers in a distribution. Instead of arbitrarily picking a top and bottom percentage to ignore, you could ignore only scores that are "too far" from the median - the outliers. "Too far" is a subjective, but for example, you could ignore results more than 3 away from the median. If the median is 7, ignore anything but scores of 4, 5, 6, 7, 8, 9, and 10.
I still don't see why we don't just use the median.

Quote:Also, we could increase the range of values. Instead of just 1 to 10 (which is ok with a small number of entries), in order to let voters have more flexibility with ranking the games they played we should probably double the range. IOW, there were two games I liked almost exactly
as much as each other, but I thought one was a bit better. Therefore I want to give it a higher score, but a whole point is a pretty big advantage when there are such few votes.
I agree with this, maybe we could adjust the voting interface to allow the user to score it by typing. With standard radio buttons 1-10 and then an "other" radio button that would allow the voter to type in a 1.0-10.0 entry.
Member
Posts: 26
Joined: 2008.04
Post: #33
Quote:That was a special case and I had already stated that Game A would have a 7.5 if this were the case.

Then assume that Game A had five 7's and six 8's, and that Game B had eleven 8's. The median would give both of these titles an 8, would it not?
⌘-R in Chief
Posts: 1,276
Joined: 2002.05
Post: #34
If the scoring range was 1 to 1000 there wouldn't even be median.

*Every* vote should count*. Not just the ones where you're part of a plurality.

Quote:If two games are scored the same in graphics then the better game should get the vote.

The whole point of the categories is to rank games on how good they are in that category alone. It should have absolutely nothing to do with any other category. Ever.

Add on top of that the fact that nearly every game would be tied with at least one other, and that just makes the ranking in individual categories absolutely useless.

* (except the ones believed to likely be biased and therefore are being rejected by some scheme)
Member
Posts: 65
Joined: 2008.09
Post: #35
Quote:Then assume that Game A had five 7's and six 8's, and that Game B had eleven 8's. The median would give both of these titles an 8, would it not?

Yes, but I think if the median range is between two numbers, then it should be (7+8)/2. That goes along with the special case. Suppose if these two titles tied, the eleven 8's should win.

if the median range is greater than 2, then the votes were more than likely nerfed. I can't see 19 judges, half of them giving the same entry a 4, and the other half a 7. If this were the case though, then that entrant should receive the mean. (it would be nerfed, but at least to no extreme advantage or disadvantage.)
Member
Posts: 65
Joined: 2008.09
Post: #36
Quote:If the scoring range was 1 to 1000 there wouldn't even be median.
It's not between 1 and 1000, and even if it were, there would still be a median. Because there would still be a middle value, or if there are an even number of votes, two middle values that could be added and divided by two.

Quote:The whole point of the categories is to rank games on how good they are in that category alone. It should have absolutely nothing to do with any other category. Ever.
I agree, but there should also be tie breakers.

Quote:Add on top of that the fact that nearly every game would be tied with at least one other, and that just makes the ranking in individual categories absolutely useless.
How would nearly every game tie? if two games receive a 9 for audio, its obvious that the sound quality was really good for both. I say the better overall game gets the win though (with the other 2nd to it), because a greater effort was put into making it. I really don't see how this makes individual categories useless.
⌘-R in Chief
Posts: 1,276
Joined: 2002.05
Post: #37
Game 1: 17, 17, 17, 17, 17, 17, 18, 18, 19, 20, 20, 20, 20, 20 = 18.3
Game 2: 14, 14, 15, 16, 16, 17, 17, 17, 18, 18, 18, 18, 18, 18 = 16.7

Far and away Game 1 wins. more people liked it than liked Game 2, yet according to the median Game 1 gets a 17, and Game 2 gets an 18?

How is that fair?

Median is NOT a relevant way to rank anything.
Member
Posts: 37
Joined: 2009.02
Post: #38
DavidGuy Wrote:Just a quick note about the prizes column:

My entry (Maneuv'It!) is not eligible for 3 prizes but only 2, because my two 3rd-place-finishes are in the same "prize-round" and the rules state that more than one choice per round is not allowed. So I get to pick once in the 3rd round, and once in the 4th round.

As another example Laserface is not eligible for 6 prizes, but rather 3. One pick in each the 1st (for best overall), 2nd (for its four 1st place finishes) and 4th (for its one 3rd place finish) rounds.

This comment by David got me wondering as I had not paid much attention to this part of the rules before. I went back and looked at the rules and was surprised to find that the rules state something even more extreme! To quote:
***
Each round consists of the following steps:

1. First place in the â€œBest Overall Gameâ€ category will select one prize from the pool.
2. Second place in the â€œBest Overall Gameâ€ category will select one prize from the pool.
3. Third place in the â€œBest Overall Gameâ€ category will select one prize from the pool.
4. The first place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.
5. The second place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.
6. The third place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.

No winner may select a prize from the pool more than once per â€œround.â€

***
So by the definition in the rules, the "round" is the entire array of winners, from first to third place in all categories. All of the steps are part of the round. Which means that Laserface is not eligible for 6 or even 3 prizes but only one prize each time the loop is gone through - the same number as every other winning entry.
That comes as a big surprise but mainly because I didn't bother to read the rules thoroughly before. So is this stated incorrectly or is this the actually system? Carlos?
Member
Posts: 65
Joined: 2008.09
Post: #39
Quote:Game 1: 17, 17, 17, 17, 17, 17, 18, 18, 19, 20, 20, 20, 20, 20 = 18.3
Game 2: 14, 14, 15, 16, 16, 17, 17, 17, 18, 18, 18, 18, 18, 18 = 16.7

Far and away Game 1 wins. more people liked it than liked Game 2, yet according to the median Game 1 gets a 17, and Game 2 gets an 18?

How is that fair?

Median is NOT a relevant way to rank anything.

That's not so, the median of Game 1 is 18, the median of Game 2 is 17. I think you think I mean the mode, when I mean the median and not the mean. Although now I'm willing to settle with the Interquartile mean, I just thought the median would be easier to do.
⌘-R in Chief
Posts: 1,276
Joined: 2002.05
Post: #40
I just realized I was thinking mode myself. But reading everything else mode made sense where everyone was saying median. I'll have to rethink this...........

Edit: Ok:

Game 1: 17, 17, 17, 17, 17, 17, 17, 18, 20, 20, 20, 20, 20, 20 = 18.3
Game 2: 14, 14, 14, 15, 16, 17, 18, 18, 18, 18, 18, 18, 18, 18 = 16.7

Median (not mode ) says that Game 2 wins with 18 versus Game 1's 17. Clearly the mean is better here.
Member
Posts: 65
Joined: 2008.09
Post: #41
Quote:Game 1: 17, 17, 17, 17, 17, 17, 17, 18, 20, 20, 20, 20, 20, 20 = 18.3
Game 2: 14, 14, 14, 15, 16, 17, 18, 18, 18, 18, 18, 18, 18, 18 = 16.7

Median (not mode ) says that Game 2 wins with 18 versus Game 1's 17. Clearly the mean is better here.

ok, so we will call this an example of the new system. (seeing as how everything is doubled)
Game 1 would've gotten a 17.5
Game 2 would've gotten an 18 you are right on that, but is the mean really better here?
Didn't game 2 get nerfed by those 14's, how is it that three devs felt it deserved a 14 and eight thought it got an 18, yet it scored a 16.7

Maybe the interquartile mean would work best, 25% off the top and bottom, then averaged. Like someone mentioned earlier that's how it's done in sport competitions.
Member
Posts: 749
Joined: 2003.01
Post: #42
Obviously the mean is better sometimes and the median other.

Â©hâ‚¬ck Ã¸ut Âµy stuÆ’Æ’ Ã¥t ragdollsoft.com
⌘-R in Chief
Posts: 1,276
Joined: 2002.05
Post: #43
tcIgnatius Wrote:Didn't game 2 get nerfed by those 14's, how is it that three devs felt it deserved a 14 and eight thought it got an 18, yet it scored a 16.7

What gives you the right to say someone got nerfed for giving their opinion? If it's 1 when the mean is 17, that's a different story. Just because someone didn't like it exactly as much as everyone else doesn't mean their vote shouldn't count.

Some people like gore. Some don't. The ones that do will like a game with gore more than the people that don't like gore.

You're going to ignore the votes from people who don't like gore just because there aren't as many of them? That's illogical.

The ideal is to ignore obviously biased votes, not silence people who simply don't agree with the majority.
Member
Posts: 65
Joined: 2008.09
Post: #44
Quote:What gives you the right to say someone got nerfed for giving their opinion? If it's 1 when the mean is 17, that's a different story. Just because someone didn't like it exactly as much as everyone else doesn't mean their vote shouldn't count.
I agree, their opinion should count, it's just that mathematically, having a mean makes the minority opinion weigh more than the majority.

Quote:The ideal is to ignore obviously biased votes, not silence people who simply don't agree with the majority.
Yes, the median does not "silence" those who disagree with the majority, it actually makes the weight of their choice equal to each individual voting.
Apprentice
Posts: 10
Joined: 2009.02
Post: #45
pinguoren Wrote:This comment by David got me wondering as I had not paid much attention to this part of the rules before. I went back and looked at the rules and was surprised to find that the rules state something even more extreme! To quote:
***
Each round consists of the following steps:

1. First place in the â€œBest Overall Gameâ€ category will select one prize from the pool.
2. Second place in the â€œBest Overall Gameâ€ category will select one prize from the pool.
3. Third place in the â€œBest Overall Gameâ€ category will select one prize from the pool.
4. The first place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.
5. The second place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.
6. The third place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.

No winner may select a prize from the pool more than once per â€œround.â€

***
So by the definition in the rules, the "round" is the entire array of winners, from first to third place in all categories. All of the steps are part of the round. Which means that Laserface is not eligible for 6 or even 3 prizes but only one prize each time the loop is gone through - the same number as every other winning entry.
That comes as a big surprise but mainly because I didn't bother to read the rules thoroughly before. So is this stated incorrectly or is this the actually system? Carlos?