iDevGames Forums
Test-First/Test-Driven Development - Printable Version

+- iDevGames Forums (
+-- Forum: Development Zone (/forum-3.html)
+--- Forum: Game Design (/forum-5.html)
+--- Thread: Test-First/Test-Driven Development (/thread-3796.html)

Pages: 1 2

Test-First/Test-Driven Development - skyhawk - Oct 20, 2006 03:50 PM

Before I get off on a rant here about the retardedness of Test-First driven development, does anyone here practice unit testing via frameworks such as JUnit, SenTestingKit, NUnit, or CppUnit?

So what drive a person to such insanity to test if their code works before writing their code?

Test-First/Test-Driven Development - OneSadCookie - Oct 20, 2006 04:16 PM


Producing high-quality code right off the bat, that you know works, and can show that it continues to work as you refactor it, is insane?

Some things are easier to test than others, to be sure, and at some point the effort of testing does seem to begin to outweigh the benefits, but most of the core "business logic" stuff is eminently testable, and benefits greatly from being developed in a test-driven way.

If you're knocking it, you clearly either haven't tried it, or did it wrong Smile

Test-First/Test-Driven Development - unknown - Oct 20, 2006 04:38 PM

LOL Testing code is for noobs.

Test-First/Test-Driven Development - hokan - Oct 20, 2006 04:40 PM

I can imagine a number of reason why test driven development may be useful, I haven't actually (as of yet) tried it myself, so this is somewhat hypothetical:

* Starting with the test cases forces the programmer to think about the usage of the API[1] function rather than their implementation - the tests will act as a kind of specification describing how the code should behave.

* Starting with the test cases, will add a greater focus on making the API functions easily testable - or putting it another way, functions that require a lot of test code to be set up, to be able to test them, will likely not be tested properly.

* Focusing on the purpose of the API (it's expected behavior) may be helpful when trying to figure out what properties/limitations/features need to be applied to the actual code. Writing tests will highlight some of the issues with using the code in other parts of the system.

I would also imagine that writing code this way will require rewriting/updating test cases (as well as code) as one gets to better understand the problem one is trying to solve.

Test cases are usually simpler than the code they test - one could therefore presume, that rewriting tests is cheaper than redoing code, that ended up doing the wrong thing.

Test driven development will certainly also be unsuitable in some cases e.g. when the API changes rapidly due to the developer experimenting with the best way to organize it.

[1]: I use the term pretty generally here, the API could be a class, some kind of software subsystem or some other well defined piece of software.

Test-First/Test-Driven Development - BeyondCloister - Oct 21, 2006 04:26 AM

I've been involved with projects that used unit testing and they were a great success. A success highlighted when compared to the build and see what happens projects within the same company.

Ensuring the individual classes worked without the clutter of an application was a great help.

A proper design process was followed (activity and sequence diagrams for the classes) that led to the required interfaces for the classes being identified.

Writing the unit tests picked up some design flaws which were then fed back into the design process.

Because all the classes had passed the testing the application just worked as they were dropped into place without all the head scratching of what was broken.

Due to the project having several developers it was possible to have a different developer write the unit tests from the developer writing the class. This ensured everything was tested and not subconsciously (or even intentionally) ignored as can happen when testing your own code.

When changes were required to any of the classes it was possible to check that other classes dependant on them were not affected. This proved very useful as the core library was used in several products.

The testing also helped to focus the project. Checking the over night build and test results in the morning gave an instant indicator of progress. Seeing pages of red instead of green first thing on a Monday morning when just one class has been changed is a bit demoralising but a boost when it all turns green again.

Then there is the final and most important advantage - less end user support.
Because the classes have been tested to destruction there are none of those obscure errors caused by unexpected values or conditions happening turning up. The kind of errors which then take days to reproduce or even only occur on the user's computer. An extra month of development may have occurred because of all the unit testing but it saves the months of chasing the small stuff the client keeps finding.

Test-First/Test-Driven Development - hokan - Oct 21, 2006 08:31 PM

Another advantage of automated testing systems (like unit testing systems), is that it gets a lot easier to redesign systems, without reintroducing errors or requiering lots of manual retesting.

This especially important in systems that gain new features or expands old features over time - think Office, Photoshop, 3D game engines or any other application that is suppoused to live for many years.

Test-First/Test-Driven Development - AndyKorth - Dec 8, 2006 10:19 PM

I highly recommend using unit tests for most games.

I develop games in Java, and use JUnit software. Although my intentions are good, I don't always get around to writing my tests first. However, I do test the large features in my game, and it goes a long way.

The biggest advantage of unit tests is that they stick around. You write feature A.. you have a test for feature A. Look, feature A works.

Several months later, you're working on feature K, and it somehow manages to break feature A. Well, when you run your tests, you notice that immediately. You don't need to spend hours testing every single feature by hand.. a task that would quickly become pretty dull. Maybe you'd say that these features should be independent such that your future code changes will never break an old feature that used to work... Well, anyone with more than a bit of programming experience will know that's not the case- things will change and your old code will break.

Hokan also had a really good point. Once your test for feature A works, you can improve that system or refactor that code and you'll have immediate feedback of whether those changes you did work or not; you still have that old test, and it still applies.

Test-First/Test-Driven Development - AnotherJake - Dec 9, 2006 12:37 AM

AndyKorth Wrote:You write feature A.. you have a test for feature A. Look, feature A works.
I guess I didn't realize it, but I've been programming using unit testing since I started. I write feature A. I test feature A to see if it does what I meant it to do... And look, feature A works! The special unit testing harness I was using was the program I was writing it for in the first place!

Look, I'm usually pretty open-minded about things, but unit testing does indeed strike me as being a tad on the... Never mind, I'll be cool about it and not really say what I think... Nah, scratch that, you test-first/test-driven guys are mad! MAD!!!

Test-First/Test-Driven Development - OneSadCookie - Dec 9, 2006 01:56 AM

Your program does test the code you've written, but only when you manually use it in the appropriate manner.

Your unit tests test the code you've written every single time you build the application.

There's a big difference. The end boss for a game is a good example -- just how often during testing are you actually going to see that guy? If you make some minor but fundamental change, do you make sure you play through the entire game to reach him immediately, or do you realize two weeks later that he's broken some time in the last month?

Test-First/Test-Driven Development - SethWillits - Dec 10, 2006 01:46 AM

If only unit testing a GUI app was easy....

Test-First/Test-Driven Development - OneSadCookie - Dec 10, 2006 03:18 AM

If your GUI app is well-written, all your logic is completely unit-testable, and all your bindings to your GUI are simple and generic. If you're a Cocoa GUI app it's even easier, because Cocoa gives you all the GUI bindings.

If your GUI app is not well written, software like Eggplant (if you have a budget to speak of) or even just AppleScript with full assistive access enabled will let you poke and prod the GUI to your heart's content.

Test-First/Test-Driven Development - ThemsAllTook - Dec 10, 2006 10:41 AM

OneSadCookie Wrote:If your GUI app is well-written, all your logic is completely unit-testable, and all your bindings to your GUI are simple and generic.
This mentality is what bothers me about test-driven development, and various other programming practices. Many of them only work by making the assertion that if you do something that doesn't conform to their way of doing things, you're doing it all wrong. This means you have to invest a lot of effort up front and change your own workflow purely based on faith that the prescribed approach will work for you. It's quite similar to religion.

I have enough worldly experience to have seen such approaches fail disastrously, and by now I'm quite wary of anything that makes that kind of assertion. There's certainly a time and place for unit testing, but it's by no means the only "right" way to write software. There are plenty of other approaches that are perfectly valid, and a blanket statement of "TDD is better" cannot legitimately be made.

That said, I do use unit tests myself for projects of large enough scope. They've proven useful in some cases, but usually only when they're initially written. Nearly every time I've seen a unit test break after passing initially is when the desired behavior of the implementation has changed and the test itself needs to be updated, rather than the implementation breaking. It often seemed more of a hassle than it was worth to be updating the tests all the time. I've also encountered inflexibility when working with other programming teams, in which they didn't want to change their behavior for something we needed, simply for the reason that it would make their tests break!

Test-First/Test-Driven Development - OneSadCookie - Dec 10, 2006 01:58 PM

Concepts like separation of responsibility, and design patterns like MVC, are not generally considered to be "negotiable". Yes, it's a religion, but it's a religion born out of a need to have a maintainable program.

If you can find a better way to design your program, congratulations. Write a paper, become world-famous, go on tour giving lectures in every country, and make millions as a consultant.

If not, take the advice of the people who do know how to design a program well -- and the result will be unit-testable.

To the people who want to avoid changing the design of their program because their unit tests would break, that's like holding a knife by the blade and trying to cut with the handle -- they've missed the entire point. The unit tests are *so* you can redesign the program and still know that it works when you've finished -- and yes, that involves refactoring or rewriting the tests alongside the refactoring of the code.

It's also important to note that whilst this might be a religion, nothing is dogma -- it's constantly being reevaluated, incrementally improved, and discussed. In fact, the idea of "unit tests" is now considered old-fashioned; last I heard "programmer tests" were the state of the art, being a similar concept but somewhat broader in scope.

If you care enough, you too can become involved in the discussion. The people there can give far more convincing arguments as to why this process is great than I can Smile

Test-First/Test-Driven Development - SethWillits - Dec 10, 2006 02:52 PM

OneSadCookie Wrote:If your GUI app is well-written, all your logic is completely unit-testable, and all your bindings to your GUI are simple and generic. If you're a Cocoa GUI app it's even easier, because Cocoa gives you all the GUI bindings.

If your GUI app is not well written, software like Eggplant (if you have a budget to speak of) or even just AppleScript with full assistive access enabled will let you poke and prod the GUI to your heart's content.

Well take Araelium Edit (a text editor for those of you who don't know) for example. (Since this is what I'm talking about, after all Rolleyes ) The logic part of the program is what, exactly? What is there to test except for the GUI? Sure, text mutations like sorting lines, regular expression search, and stuff like that, but most of the program is in keeping the GUI updated with the results of very little logic. I'm completely lost when it comes to figuring out what to test and how to test it.

I'm not on the agile and test-driven development boat, but I really want to be. I've read a few books on it and had long discussions with my father who is now working with Thought Works (big big agile company) and I've heard first his retellings of first hand experiences. I'm sold, but I just don't how to apply it to this kind of project.

Test-First/Test-Driven Development - AndyKorth - Dec 10, 2006 03:20 PM

In general, I find it a bit hard to test the GUI portions of my application. I do tend to skip that part and mostly just test the underlying logic of my programs. However, here's one example of code I wrote last week to test a table I had made. I noticed I was having strange things happen when I tried to delete a row from the table. (It was a list of items for my item editor) So I wrote a unit test:

public void testRemoveItem() {
        int initialRowCount = tm.getRowCount();
        int initialDBSize = DB.items.size();

        //create new item.
        Item newItem = Item.createNewItem();
        newItem.setName("Test item");
        DB.items.add(newItem.getItemID(), newItem);

        //inform the table model that a change to the underlying data has occured.

        assertTrue("DB did not grow", initialDBSize + 1 == DB.items.size());
        assertTrue("Table model did not grow", initialRowCount + 1 ==  tm.getRowCount());

        Item i = tm.get(0);
        assertTrue("DB did not shrink", initialDBSize == DB.items.size());
        assertTrue("Table model did not shrink", initialRowCount == tm.getRowCount());

        assertTrue("We got the same thing we removed!", tm.get(0) != i);

Turns out the bug was that I forgot to have the remove method fire the table changed listeners. A reasonable mistake.. the silly thing we've all done a million times.

The advantage is that this test will stick around. Even when I made big changes to the table model (variable name "tm").. change how it selects or how we add new items to it, etc.... I have some simple assertion that there's nothing grossly wrong with remove. Note that I didn't go through a lot of work making the test.. it could be a lot better... .maybe I shouldn't post it.. heh.