E-mail    Print    Discuss    Blog

A Dozen Ways to Get the Testing Bug in the New Year

Contents 1. Let Your Computer Do the Boring Stuff 2. Stop Debugger Testing 3. Assert Your Expectations 4. Think of It as Design 5. Build Safety Nets 6. Learn by Checked Example 7. Corner Bugs 8. Expand Your Toolbox 9. Make It Part of Your Build Process 10. Buddy Up 11. Travel With a Guide 12. Practice, Practice, Practice Summary Resources

Test-driven development received a lot of attention in 2003, and the interest will grow in 2004. For good reason: everyone agrees testing is important, but now many respected programmers are claiming that by writing tests first, they see better designs emerge. These same programmers quickly point out that test-driven development makes them feel more productive and less stressed. At the end of a shorter programming day they've built a suite of passing tests and code with better designs. Sound too good to be true? Well, there's nothing to lose in giving it a whirl. In fact, there's much to be gained.

This article gives you 12 practical ways to start writing tests, and keep writing tests, regardless of your development process. The first two techniques play off of things you're probably already doing, so you don't have to move too far out of your comfort zone. The next two challenge you to wade deeper into the pool to realize the benefits of test-driven development. The remaining techniques round out the regimen to keep you testing effectively all year. You'll be well on your way to fulfilling your new year's resolutions. Caution: contents have been known to be infectious!

1. Let Your Computer Do the Boring Stuff

The easiest way to start writing tests is to identify situations where you visually inspect results, then replace that human checking process with automated checking. Color me lazy, but I want to put this thing called a computer to work for me. It's much more reliable than I am at keeping my expectations in check. Some results are difficult to check without a human in the loop; don't start there. Instead, go after low-hanging fruit to get a few small victories under your belt. I've found the pervasive main() test driver to be easy pickings for automating. I'm not referring to the entry point that bootstraps your application, but rather the main() method that acts like a test driver by printing results to the console.

For example, imagine that you're writing a Java class that performs the functions of a simple spreadsheet. A spreadsheet cell -- indexed by a column and row combination such as "A1" -- can store a number or a formula that may include numbers and cell references. Here's an example main() test driver for the Spreadsheet class:

public static void main(String args[]) {
        
    Spreadsheet sheet = new Spreadsheet();
        
    System.out.println("Cell reference:");
    sheet.put("A1", "5");
    sheet.put("A2", "=A1");
    System.out.println("A2 = " + sheet.get("A2"));
        
    System.out.println("\nCell change propagates:");
    sheet.put("A1", "10");
    System.out.println("A2 = " + sheet.get("A2"));
        
    System.out.println("\nFormula calculation:");
    sheet.put("A1", "5");
    sheet.put("A2", "2");
    sheet.put("B1", "=A1*(A1-A2)+A2/3");
    System.out.println("B1 = " + sheet.get("B1"));
}

You may recognize this testing style or may have even written similar test drivers yourself, if only to give you some confidence that the code produced the results you expected. The main() method was my testing harness of choice for many years, and I still get the urge to use it from time to time because it's easy. But just as I'm about to take the bait, I remember how a main() test driver sucks the life out of me. See, every time I change code that affects the Spreadsheet class, I want to run the test to see if my change broke anything. I'm confident in my changes if I run the test afterward and see the following console output:

Cell reference:
A2 = 5

Cell change propagates:
A2 = 10

Formula calculation:
B1 = 15

This testing approach has at least one problem: it requires that I visually inspect the output every time I run the test. Worse yet, as the number of results output by the test driver increases, my workload also increases. I'll quickly grow weary of doing work best suited for a computer and stop running the test altogether. Inspecting the output also implies that between test runs I have to remember how the expected output should look. Is the correct result of the formula calculation 10 or 15? Hmm, I can't remember. And if I can't remember, there's little hope of sharing the test with other folks.

JUnit is a computer's taskmaster when it comes to checking expectations. If you've never used JUnit, the JUnit FAQ will get you up and running in less time than it takes to type a main() method signature. Using JUnit, a main() test driver requiring human checking can be easily replaced by automated tests that check their own results. Here's the equivalent Spreadsheet test, expressed in a JUnit test:

import junit.framework.TestCase;

public class SpreadsheetTest extends TestCase {
    
    public void testCellReference() {
        Spreadsheet sheet = new Spreadsheet();
        sheet.put("A1", "5");
        sheet.put("A2", "=A1");
        assertEquals("5", sheet.get("A2"));
    }

    public void testCellChangePropagates() {
        Spreadsheet sheet = new Spreadsheet();
        sheet.put("A1", "5");
        sheet.put("A2", "=A1");
        sheet.put("A1", "10");
        assertEquals("10", sheet.get("A2"));
    }

    public void testFormulaCalculation() {
        Spreadsheet sheet = new Spreadsheet();
        sheet.put("A1", "5");
        sheet.put("A2", "2");
        sheet.put("B1", "=A1*(A1-A2)+A2/3");
        assertEquals("15", sheet.get("B1"));
    }
}

Notice that the result checking is now codified in the use of assertEquals() methods that automatically check whether the expected value (the first parameter) matches the actual value (the second parameter). There's no need for you to remember what the correct results should be.

JUnit is distributed with two test runners -- textual and graphical -- that both produce simple and unambiguous output. Using the textual runner, an "OK" on the console signifies that all of your expectations were met:

> java junit.textui.TestRunner SpreadsheetTest

...
Time: 0.04

OK (3 tests)

Using the graphical runner (junit.swingui.TestRunner), you're looking for a comforting green bar. Most Java IDEs have an integrated graphical runner just waiting to stroke your ego, such as this runner in Eclipse:

Figure 1. Green is good

If your expectations aren't met, JUnit is quick to let you know. Depending on the test runner used, if a test fails, you'll either see an eye-popping failure message on the console or a flaming red bar, along with details of the failed test.

Automation isn't necessarily testing, you say? I couldn't agree more. So now that you have an automated test harness that takes the pain out of manual testing, feel free to write more comprehensive tests. The book Pragmatic Unit Testing will help you strengthen your testing skills and write better tests. As you write more tests, it doesn't cost you anything to keep them running. Indeed, automated tests increase in value over time as ongoing regression tests, so that you have more time to write good tests.

2. Stop Debugger Testing

There was a time when I followed the conventional wisdom of running my code through a debugger to check that the code worked as I expected. That approach worked well, if only I never had to change the code again. But then when the code needed to be changed, I resisted the change because I dreaded firing up the debugger and walking through the code. In other words, using the debugger as a regression testing tool had the same drawbacks as using the main() method as a test driver. It just didn't scale. Consequently, I tended not to touch working code for fear of unknowingly breaking something. The result was code rot.

If you're like me, you use a debugger to validate mental assertions as follows:

1. Set a breakpoint right before a section of code you have questions about.
2. Set watches on interesting variables within that code section.
3. Run the program up to the breakpoint.
4. Single-step through each line, examining the variables along the way.
5. Optionally, manipulate variables on the fly to force certain code paths to be exercised.

The entire time I'm doing this, in my head I have expectations about the values of variables and the results of method calls. The debugger merely gives me answers that I then match against my expectations. That is, when I use a debugger, I'm really settling for human checking that's prone to error and boredom.

Look for opportunities to replace the human checking you do via debugger with an automated test that checks its own results. It's not always easy to initialize the environment required by a section of code you'd like to test. After all, it's running the program up to the breakpoint that builds that context around the code, though usually at a significant start-up cost. But if the code is likely to undergo change that might break it, then it's worth finding a way to write an automated test. In doing so, you might just discover that the code being tested can be decoupled from some of its dependencies.

3. Assert Your Expectations

A lot has been written about test-driven development, summed up in this simple recipe:

1. Write new code only after an automated test has failed.
2. Refactor to keep the code clean.

The notion of writing a failing test before writing the code that makes it pass may seem awkward, but think about how you write code. Before you write a method, you generally have some idea what the method should do. If you don't, one might argue you're not yet ready to write the method. Writing the test first is typically easier and quicker than writing the corresponding code. The test then keeps you from wandering down rabbit holes and lets you know when you're done coding. So, before writing that next method, stop to consider what expectations you already have in mind. Assume you've already written the method, then simply write an automated test that asserts that the method works correctly.

Say, for example, you're staring at a blank editor screen, ready to begin writing a ShoppingCart class. The first thing you want it to do is manage a collection of items. If the same item is added twice, each time with a specified quantity, the shopping cart should contain the sum of both quantities. Now turn that mental assertion -- the success criteria for the code you wish you had -- into an automated test. The following is an example JUnit test:

import junit.framework.TestCase;

public class ShoppingCartTest extends TestCase {

    public void testAddItems() {
        ShoppingCart cart = new ShoppingCart();
        cart.addItems("Snowboard", 1);
        cart.addItems("Lift Ticket", 2);
        cart.addItems("Snowboard", 1);
        assertEquals(4, cart.itemCount());
    }
}

Serious question: how much more effort would it take to write that test after you'd already expended brain cycles deciding what the shopping cart should do? Think of test-driven development as a way of structuring and refining that thought process.

Now that you have a test, write just enough code to make it pass. No more, no less. Just let the test guide you rather than speculating about what you might need in the future or worrying about the next test. When the test passes, refactor the code as necessary to keep it clean and as simple as possible. Then re-run the test to make sure refactoring didn't break anything. Repeat by asserting your expectations for what the code should do next. Before long you'll fall into your own test-code-refactor rhythm. Stick with it; it will serve you well.

View all java.net Articles.