iDevGames Forums
same deal as last year - Printable Version

+- iDevGames Forums (
+-- Forum: Community Zone (/forum-4.html)
+--- Forum: Contests (/forum-19.html)
+--- Thread: same deal as last year (/thread-6591.html)

Pages: 1 2

same deal as last year - skyhawk - Dec 11, 2003 01:58 PM

all stats hosted on my site

if anyone else would like to donate more stats or list, I would be glad to host them

same deal as last year - NCarter - Dec 11, 2003 02:10 PM

Thanks, skyhawk. I was going to do the same myself, but you've saved me a lot of trouble! Smile

same deal as last year - Jake - Dec 11, 2003 02:28 PM


same deal as last year - DaFalcon - Dec 11, 2003 02:30 PM

Yes, thanks Skyhawk :-)

In case anyone can do some statistical analysis in Excel (correlation between the various categories and overall score, for example) I've dropped it all into a spreadsheet at work:

What is cool to see here is the graph showing all of the games' scores broken down by category. You can see just how tight the competition to get into the top 5 is, and how every little bit of polish, every great sound, every wonderful graphic could be the thing that pushes you up in the ranks!

same deal as last year - KittyMac - Dec 11, 2003 03:13 PM

Thanks Skyhawk!

I was wondering what place PI came in the Sound & Music category. My musician will be happy to hear that we came in 6th.


same deal as last year - Carlos Camacho - Dec 11, 2003 06:12 PM

Thanks for helping everyone. You still have 2002? I thought the link might be broken.

In related news. I plan to set up an independent website for the contest, so that it will have a permanent home, and thus you will all be able to look back on the past year's contests.

And yes, I am interested in getting some thought-provoking analysis of what this all means like we got in 2002.


same deal as last year - Carlos Camacho - Dec 11, 2003 06:17 PM

Ah, I see you have /2002/ up as well.


Argo. scored higher than Kiki from last year in overall. I'll like to know if this year's entries had a higher polish than last year. Smile

Gamep play score fro 2002 and 2003 were the same Smile


same deal as last year - aarku - Dec 11, 2003 06:52 PM

Has anyone averaged all the games categorically and compare those numbers to last year and the year before? It'd be interesting to see how those numbers are changing.


same deal as last year - Holmes - Dec 11, 2003 11:10 PM

Quote:Originally posted by Camacho
Ah, I see you have /2002/ up as well.


Argo. scored higher than Kiki from last year in overall. I'll like to know if this year's entries had a higher polish than last year. Smile

Gamep play score fro 2002 and 2003 were the same Smile


In fact several games from this year scored higher than Kiki overall including Chopper, primate plunge, and Tower of Tears. Lightning's Shadow and Yoink we also close. Congratulations to these devs. I was shocked to see this! So many high quality games! (or voters stuffed full of sugar and morphine?)

same deal as last year - skyhawk - Dec 11, 2003 11:16 PM

maybe that means this year people weren't as stern and hardcore about their votes. maybe they were more willing to throw all 5s around

same deal as last year - DaFalcon - Dec 12, 2003 12:10 AM

[Mandatory Disclaimer from someone who took too many stats classes in college for business: Please note that this makes the assumption that scores across years can be compared. This is not necessarily true because different votes may have voted (with different preferences towards higher or lower scores) and the same voters may have changed their voting habits. Even so, it is interesting and does tell a good story when taken with a grain of salt]

Well, comparing 2003 to 2002, there has been improvement in every category, most notably in sound. A look at the numbers:

Gameplay  Graphics   Orig.    Polish    Sound
Avg. 2002 Score: 2.89      2.94      3.00     2.85      2.53
Avg. 2003 Score: 2.91      3.03      3.01     2.88      2.73

% Improvement:     1%        3%        0%       1%        8%

This makes sense because sound was the category that most obviously needed improvement, so many developers found people to make music for them, or they at least put some thought into the sound effects used.

Overall, scores improved 3% from an average of 14.2 to 14.56.

Now, this is all looking at averages. We can also see that the bar was raised in 2003 for the best overall games:

Kiki the Nanobot 19.59
The Belt 19.5
Galder 19.46
MAFFia 19.31
Black Shades 19

Argonaut - 2149 20.1
Chopper 19.91
Primate's Plunge 19.8
Tower of Tears 19.77
Lightning's Shadow 19.4

These top 5 games show a 2%-3% increase year-to-year.

So is this 3% "inflation" a product of voters' changing voting patterns? I say no, it is a reflection of the overall increased quality of games this year over last, slight as it is. So good job everyone, not only this year, but last year too.

same deal as last year - geezusfreeek - Dec 12, 2003 12:32 AM

[Edit: DaFalcon has made a much better analysis farther down the page. Don't even bother with mine. Rasp ]

I decided I would see how closely each category was was associated with the overall score. That is, how much does each category deviate from the overall score? Here are the steps I took:

A) Instead of using the sum of the categories for the overall score, I used the average.

B) I calculated the sums of the scores in each category:
Overall: 125.18
Gameplay: 125.13
Graphics: 130.09
Sound: 117.26
Polish: 123.90
Originality: 129.54

C) I got the absolute value of the difference of each category and the overall score.
Gameplay: 0.05
Graphics: 4.91
Sound: 7.92
Polish: 1.28
Originality: 4.36

These numbers are rather small and close together. What does this mean? It means that high ratings in any one category don't necessarily mean high ratings overall. There were theories that the polish category was just the sum of the other categories, thus it probably reflected the overall score the most. This appears to be untrue.

Please correct me if my logic is off track.

same deal as last year - DaFalcon - Dec 12, 2003 01:02 AM

There are actual formulas for finding how closely two variables are related, but alas that knowledge slipped right out of my ears the minute my final final in stats ended.... It is probably something simple, too. Even then, you must remember that correlation doesn't necessary imply causation.

Maybe I should go to sleep so I can wake up and speak normal human english again :-)

same deal as last year - Carlos Camacho - Dec 12, 2003 01:33 AM

You guys amaze me.

I'm curious to know if devs from 2002 did better or not.

same deal as last year - DaFalcon - Dec 12, 2003 09:43 AM

Okay I calculated the correlation coefficient of each of these variables (okay, so Excel did it for me). [If you want to read about cc, read this: ]

That is the relationship between variables... If a higher polish always happened with a lower sound score, that would be a negative relationship. Here, however, are only positive relationships. This is as expected. The scale here is -1 to +1, with +1 meaning that every time one goes up a certain amount, the other goes up by a certain (different) amount [the values could be graphed on a straight line and every point would fall on the line].

NOTE!!! "A correlation between two variables does not necessarily mean that one variable causes another to change. It just means that as one variable changes the other changes also."

So even though the relationship between graphics and polish is 0.91, it could just mean that teams that are good at creating graphics are also good at polishing the game. BUT it COULD mean that voters are saying, "oh, this game has nice graphics, it is really polished". We may never know for sure :-)

Here are the relationships:
Gameplay  Graphics  Original  Polish
Gameplay  x            
Graphics  0.773     x        
Original  0.517     0.389     x    
Polish    0.892     0.917     0.451     x
Sound     0.829     0.809     0.501     0.895
As you can see, the strongest relationship is between graphics and polish. Sound also seems to be related fairly strongly to everything except originality, which is not strongly related to anything (relatively speaking). Polish was also strongly related to gameplay. Graphics and gameplay also were related.

Here are the relationships between each category and the overall score for the game:
Gameplay  0.923
Graphics  0.902
Original  0.637
Polish    0.960
Sound     0.928
The strongest relationship here is easily between polish and the overall score. In other words, increase your polish and you may end up with a higher overall score. (Or, it could be that polish means "good in every category" to voters, and that you can't do specific things to increase your polish score) This makes sense because polish was closely related to almost every category.

Interestingly, originality does not really influence the overall score nearly as much as any other category. So you can create a clone but do it REALLY well and still get a high overall score. I checked and for the top 5 games, originality made up only 15-19% of the overall score (it was 15% of argonaut). In "lower ranked" games, originality made up a much higher % of the score, as much as 30% of it (Pantheon).

So there you have some statistical analysis of 2003 from Alan. Have fun :-)