Running Individual Test Cases from Ant

JUnit and Ant are indispensable tools for Java development. This article discusses how to use them together more effectively, allowing you more control over which test cases get run. We'll start by showing how to run a specific Test class from Ant, and then move on to selectively running individual test case methods inside a Test.

But first, why would you want to do this? Test-first design and Extreme Programming suggest that we should run all of the tests, all of the time. However, not everyone has the luxury of using the test-first approach. The techniques discussed in this article were developed to deal with writing test cases for an existing, completely untested system. In such a system, tests are often just used as an easier way to make sure code doesn't blow up, rather than testing it with the UI.

The main reason I have found for wanting to run only selected test cases is in the case of bug fixes. I'll write a test method (or three) to exercise the bug, and then run those test cases repeatedly as I work on the bug. In an untested system, tests rarely are completely self-verifying, so I'll need to look at the log output. Being able to look at the output from only one test case is helpful for debugging. Another reason for wanting to run only a few tests is that tests often take a long time to run. Running test cases in isolation from each other is also warranted because, as often as not, the problem causing a test to fail is in the test itself.

I certainly recommend running all of the tests on a regular basis. But if you are already modifying your Ant build or editing your source code in order to selectively run your tests, you may find that the techniques discussed in this article will help shorten your code-compile-test cycle, resulting in better productivity for you.

For the purposes of this article, I assume you've got Ant installed and configured correctly so you can use the optional <junit> task. To do this, you need to make sure junit.jar and optional.jar are in the ANT_HOME/lib directory of your Ant install. If you need more help, check out the Ant documentation (in particular, the section on library dependencies).

Running Tests from a Single Class

It's easy to run all tests from Ant (see Cooking with Java XP), but what if you only want to run a specified Test? To do this, create a target called runtest that uses an Ant property set on the command line. This target assumes you have a src property for your source code, a compile target to compile the code before running the tests, and a classpath reference set to your classpath.

<target name="ensure-test-name" unless="test">
    <fail message="You must run this target with -Dtest=TestName"/>
</target>

<target name="runtest" description="Runs the test you specify on the command 
    line with -Dtest=" depends="compile, ensure-test-name">
    <junit printsummary="withOutAndErr" fork="yes"> 
        <classpath refid="classpath" />
        <formatter type="plain" usefile="false"/>
        <batchtest>
            <fileset dir="${src}">
                <include name="**/${test}.java"/>
            </fileset>
        </batchtest>
    </junit>
</target>

The ensure-test-name target verifies that the test property is set. If test is not set, ensure-test-name fails and warns the user to set the property.

The runtest target does the work. It depends on compile to make sure it's run with your latest code. The <junit> task runs the Test you specify with the test property. The task is configured to print all standard out and error output on the console. The formatter is plain with usefile="false" to print all messages to the console in plain text format, and printsummary="withOutAndErr" will include all messages printed to standard out and error (if you use Log4J, set the appender to ConsoleAppender to see your log messages on standard out). I also fork the JUnit JVM to avoid "Jar Hell;" this is optional. For more configuration of the <junit> task, see the Ant documentation for it.

The runtest target uses <batchtest> instead of <test>, so we don't have to specify the fully qualified class name of the Test. If no class matches the <include> criteria, no test cases will be run and the build will be successful (the summary report will show no test cases were run). If a class that's not a Test matches, the build will fail when JUnit tries to run it.

To run only the Test MyTest, you use the command:

ant runtest -Dtest=MyTest

Here, -D is the flag to pass a property to Ant. In the runtest target, you are passing in the name of the test class to run as the value of the test property.

Because runtest uses <batchtest>, you can use wildcard matching with the test property value. For example, to run all the classes ending in "Test", execute this command:

ant runtest -Dtest=*Test

Running a Test from a Single Method

After implementing running a test from a single TestCase class, I found myself wishing I could run specific test case methods inside of a TestCase class. Unfortunately, there's no way to do this in JUnit without changing the TestCase's suite method and recompiling. But by modifying the runtest target to take another parameter, and with a little bit of Java cleverness, we can dynamically create a TestSuite that only contains the test case methods we want.

First, let's modify the runtest target to take an optional argument that describes which test cases to run, as a comma-separated list of test case names. This Ant property will be used to create a JVM property.

<target name="runtest" description="Runs the test you specify on the command 
    line with -Dtest=" depends="compile, ensure-test-name">
    <junit printsummary="withOutAndErr" fork="yes"> 
        <sysproperty key="tests" value="${tests}"/>
        <classpath refid="classpath" />
        <formatter type="plain" usefile="false"/>
        <batchtest>
            <fileset dir="${src}">
                <include name="**/${test}.java"/>
            </fileset>
        </batchtest>
    </junit>
</target>

As you can see, the only modification to this version of runtest is to set an Ant property named tests as a JVM property using the <sysproperty> element. The tests property is optional, so there is no checking for it as in the case of the ensure-test-name target. Since there is no way to conditionally change a variable in Ant, if tests is not set on the command line, the JVM property will have the literal value "${tests}". The Java code that reads the JVM property will need to be aware of this.

To run all the test cases in a Test, we'll use the old command line:

ant runtest -Dtest=MyTest

To run only a few test cases, we'll use a command like this:

ant runtest -Dtest=MyTest -Dtests=testFoo,testBar

As you can see, the tests property is optional.

Supplementing JUnit

Now we need some code that can be used by Tests to dynamically create a TestSuite from the tests JVM property. This can be implemented as a subclass of TestCase that your tests then extend instead of TestCase, or it can be implemented with a few static methods in a utility class. I favor the latter approach because it makes it easier for developers to integrate the code into existing Tests.

We need two public methods: one to determine if the tests property is set and one to construct the TestSuite from the tests property. Both of these methods need to be static so that they can be called from the static suite method of a TestCase. The code uses reflection to dynamically construct instances of the TestCase subclass with each test case name from the tests property. Reflection must be used because each TestCase object needs to be constructed at runtime with the name of a test case from the tests property.

public final class TestUtils {
    private static final String TEST_CASES = "tests";
    private static final String ANT_PROPERTY = "${tests}";
    private static final String DELIMITER = ",";

    /**
     * Check to see if the test cases property is set. Ignores Ant's
     * default setting for the property (or null to be on the safe side).
     **/
    public static boolean hasTestCases() {

        return
            System.getProperty( TEST_CASES ) == null ||
            System.getProperty( TEST_CASES ).equals( ANT_PROPERTY ) ?
            false :
            true;
    }

    /**
     * Create a TestSuite using the TestCase subclass and the list
     * of test cases to run specified using the TEST_CASES JVM property.
     *
     * @param testClass the TestCase subclass to instantiate as tests in
     * the suite.
     *
     * @return a TestSuite with new instances of testClass for each
     * test case specified in the JVM property.
     *
     * @throws IllegalArgumentException if testClass is not a subclass or
     * implementation of junit.framework.TestCase.
     *
     * @throws RuntimeException if testClass is written incorrectly and does
     * not have the approriate constructor (It must take one String
     * argument).
     **/
    public static TestSuite getSuite( Class testClass ) {

        if ( ! TestCase.class.isAssignableFrom( testClass ) ) {
            throw new IllegalArgumentException
	        ( "Must pass in a subclass of TestCase" );
        }

        TestSuite suite = new TestSuite();

        try {

            Constructor constructor =
                testClass.getConstructor( new Class[] { String.class } );

            List testCaseNames = getTestCaseNames();

            for ( Iterator testCases = testCaseNames.iterator();
                  testCases.hasNext(); ) {

                String testCaseName = (String) testCases.next();

                suite.addTest( (TestCase) constructor.
                               newInstance( new Object[] { testCaseName } ) );
            }

        } catch ( Exception e ) {
            throw new RuntimeException
                ( testClass.getName() +
                  " doesn't have the proper constructor" );
        }

        return suite;
    }

    /**
     * Create a List of String names of test cases specified in the
     * JVM property in comma-separated format.
     *
     * @return a List of String test case names
     *
     * @throws NullPointerException if the TEST_CASES property
     * isn't set
     **/
    private static List getTestCaseNames() {

        if ( System.getProperty( TEST_CASES ) == null ) {
            throw new NullPointerException( "Test case property is not set" );
        }

        List testCaseNames = new ArrayList();
        String testCases = System.getProperty( TEST_CASES );

        StringTokenizer tokenizer = new StringTokenizer( testCases, DELIMITER );
        while ( tokenizer.hasMoreTokens() ) {
            testCaseNames.add( tokenizer.nextToken() );
        }

        return testCaseNames;
    }
}

As you can see, the hasTestCases method ignores the tests property if it is set to the default of "${tests}", and the private helper method getTestCaseNames uses a StringTokenizer to parse the comma-separated list of test case names, adding each one to a List. The getSuite method uses this List to reflectively construct new TestCase subclass instances and add them to the TestSuite that it returns. If a test case name is bogus or empty, nothing bad will happen. JUnit will not be able to find the bogus test case and it will be skipped.

Running the Tests You Want

Now, all the pieces are in place for you to choose which test cases to execute at run time without recompiling.

public static TestSuite suite() {

    if ( TestUtils.hasTestCases() ) {
        return TestUtils.getSuite( MyTest.class );
    }

    TestSuite suite = new TestSuite( MyTest.class );

    return suite;
}

When writing a TestCase, use TestUtils.hasTestCases to check for the tests property, and use TestUtils.getSuite with the Class object for your TestCase subclass to return the dynamically constructed TestSuite. Otherwise, construct and return the TestSuite as usual.

Luke Francl is a Minneapolis-based software engineer and democracy geek.

Related Topics >> Extreme Programming   |   Testing   |   

Article Links >>

• Login or register to post comments
• Printer-friendly version
• ShareThis
• 10092 reads