Skip to main content

UISpec4J: Java GUI Testing Made Simple

May 17, 2007

{cs.r.title}



UISpec4J is an
open source functional and/or unit-testing Java library for
Swing-based Java applications that is focused on simplicity.
UISpec4J's APIs are designed to hide, as much as possible, the
complexity of Swing, resulting in easy-to-write and easy-to-read
test scripts.

GUI Testing

With the advent of "http://agilemanifesto.org/">Agile development processes such
as Extreme Programming ( "http://extremeprogramming.org/">XP), automated testing is
being adopted by an increasingly larger number of development
teams. There is, however, one area of software that has always had a
reputation for being difficult to test: graphical user
interfaces.

In this article, we explain how we faced the same problem on an
XP project and came up with a solution by building our own GUI
testing toolkit, which is now available as a free open source
product. But before delving into this, let's start from the
beginning: how can we test GUIs?

Using Event-Driven Robots

When it comes to setting up automated tests for graphical user
interfaces, the conventional approach is to use event-driven robots
that record human interactions with the interface and can replay
these interactions at will. The interesting part of this approach
is that it is very simple to create new tests--you don't need a
developer or someone with scripting skills to do this.

The trouble is, this simplicity comes with a number of
down sides:

  • You need to have the interface to record the
    tests
    . You thus cannot use the tests as a guide for
    driving the development, and you make your development cycles
    longer by preventing developers and testers from working at the
    same time on a given feature.

  • The test suite becomes a burden, preventing changes in
    the GUI
    . Writing the first twenty tests is easy, but what
    do you do when you have more than a thousand tests? When you want
    to change a given screen in your application, will you be willing
    to chase down the hundreds of impacted tests and record them all
    again?

In other words, using generated scripts is simply not always
compatible with an incremental, agile approach.

Using Test-Specific APIs

A better solution would be to manually code these tests, so that
they can be written before the interface is available, and use a
rich programming language allowing the incremental refactoring of a
test suite that will remain easy to modify. In the Java/Swing
world, there are several existing libraries for doing this, such as
Abbot, "http://jfcunit.sourceforge.net/">JFCUnit, "http://www.marathontesting.com/">Marathon, and "http://jemmy.netbeans.org/">Jemmy.

When we were working on a large Java/Swing application several
years ago, we had a look at these libraries, but in the end we didn't use them because their APIs were too close to Swing and resulted
in tests that looked too "technical."

The UISpec4J Approach

We wanted a library that would nicely fit into our XP
process:

  • Test-first programming: For both panel-level
    unit testing and application-level acceptance testing, meaning that
    we would be able to write tests reading like user interactions
    before the GUI was available.

  • Refactoring: We would use the Java language
    (rather than, say, Python or Ruby) to benefit from the powerful
    refactoring tools available for that language, thus limiting
    duplication in the test suites.

We then resolved to create our own testing library--and
UISpec4J was born.

Let's now have a look at some UISpec4J code, to see whether
these goals have been successfully met.

The Address Book Sample Application

We obviously can't claim to have the most original sample
application on the planet, but be that as it may, the application
we will be using here is a simple address book that manages a
collection of contacts sorted by categories. Figure 1 shows the GUI
of this application.

The address book GUI is split into three main areas of
interest:

  1. On the left-hand side, a table displays a list of contacts with
    basic details. The user can add a new entry in the table by
    clicking on the button labeled ... "New contact." How
    intuitive!
  2. On the right-hand side, a tree shows available categories for
    filtering contacts. A "New category" button is placed on top of the
    tree for creating categories.
  3. At the bottom of the frame, a form details the currently
    selected contact. The form is composed of labels and editable text
    fields.

AddressBook screenshot "498" height="354" />
Figure 1. Address Book screen

A list of possible interactions with the address book could
be:

  • Creating a new contact.
  • Switching between contacts.
  • Editing an existing contact.
  • Filtering contacts by category.

But before playing with the components, we need to set up a test
class and catch our application.

Creating a Test Class

A typical way to create a UISpec4J test is to create a class
extending UISpecTestCase, which is itself a subclass
of JUnit's
TestCase.

[prettify]public class AddressBookTest extends UISpecTestCase {
  ...
}
[/prettify]

We then tell this class that it needs to run the address book
application using the main() found in the
AddressBook class, and that it can run this
application with no arguments. To do this, we set up an "adapter"
(UISpecAdapter), whose role is to implement the
adaptation between the tests suite and the application. In most
cases, we just use the MainClassAdapter provided with
the library:

[prettify]public class AddressBookTest extends UISpecTestCase {
  protected void setUp() throws Exception {
    setAdapter(new MainClassAdapter(Main.class, new String[0]));
  }
  ...
}
[/prettify]

We are now ready to create our first test.

First Test: Creating a Contact

Let's write a test that creates a new contact using the "New
contact" button and then checks that a new row appears in the
contacts table.

The UISpecTestCase class proposes a
getMainWindow() method that uses the
UISpecAdapter introduced above to return an UISpec4J
Window object representing the window displayed by the
application. This Window object can be used to fetch
individual UI components such as the "New contact" button, the
contacts table, and the various text fields used for entering
contact information.

Here is the corresponding test:

[prettify]public void testCreatingAContact() throws Exception {
  // 1. Retrieve the components
  Window window = getMainWindow();
  Table table = window.getTable();
  Button newContactButton = window.getButton("New contact");

  // 2. Check that the contacts table is empty and displays
  // the proper column names
  assertTrue(table.getHeader().contentEquals(new String[]{
    "First name", "Last name", "E-mail", "Phone", "Mobile"
  }));
  assertTrue(table.isEmpty());

  // 3. Click on the "New contact" button and check that an
  // empty row is displayed in the contacts table
  newContactButton.click();
  assertTrue(table.contentEquals(new String[][]{
    {"", "", "", "", ""}
  }));
  assertTrue(table.rowIsSelected(0));

  // 4. Change the fields of the created empty contact and check
  // that the contacts table is updated accordingly
  window.getTextBox("first").setText("Homer");
  window.getTextBox("last").setText("Simpson");
  window.getTextBox("email").setText("homer@simpson.com");
  window.getTextBox("phone").setText("01.02.03.04.05");
  window.getTextBox("mobile").setText("06.07.08.09.10");
  assertTrue(table.contentEquals(new String[][]{
    {"Home", "Simpson", "homer@simpson.com", "012345", "242424"}
  }));
}
[/prettify]

Finding Components

As you can see in the first part of the test, we use the window
object to retrieve the components using specific methods such as
getTree(), getButton(), etc. For each
component type, there are various strategies for fetching
individual components.

If you know that there is only one button in the panel, then
just ask for it!

[prettify]Button button = panel.getButton();
[/prettify]

If there are several buttons in the panel, you specify which
button you want using its displayed label:

[prettify]Button button = panel.getButton("New contact");
[/prettify]

If there are several buttons with the same displayed label, you
can distinguish among them using an "inner name" provided in the
production code by the application developers with the
JComponent.setName() method. This is, for instance, what
we do for the text boxes displayed at the bottom of the address
book window--they all look the same, so we need to rely on
internal names that we set in the production code: first, last,
email, etc. For instance:

[prettify]TextBox emailBox = panel.getTextBox("email");
[/prettify]

If none of these methods can do the job, you can still provide
your own piece of code for matching components:

[prettify]Button button = panel.getButton(new ComponentMatcher(){
  boolean matches(Component component) {
    return component.isEnabled();
  }
});
[/prettify]

Note: for I18N purposes, we usually force the default locale to
Locale.EN in our tests and use English names for
retrieving components.

Using a Table Component

The Table class provides lots of methods for
checking and manipulating the underlying JTable
component. Most of these methods work with two-dimensional arrays
representing what the end user is expected to see. For
instance:

[prettify]assertTrue(table.contentEquals(new String[][]{
  {"Homer", "Simpson", "homer@simpson.com", "012345", "2424242"},
  {"Bart", "Simpson", "bart@simpson.com", "123456", "34343434"}
}));
[/prettify]

Note: the Table.contentEquals() method, as many
other UISpec4J component methods, return Assertion
objects instead of Booleans. These Assertion objects
are used by UISpec4J to implement retry strategies, mostly for the
case of multithreaded GUIs where there might be slight delays
between actions performed in the tests and the corresponding
changes in the UI. This mechanism is completely transparent in the
tests, provided that you use UISpecTestCase's
assertXxx methods, or those of the
UISpecAssert class.

Second Test: Dealing with the Categories Tree

Let's now move to a more complicated test: "a contact must
belong to the category in which it was created." Additionally, selecting
a category in the tree should trigger the filtering of the contact
list.

Here is the code for this test:

[prettify]public void testContactsBelongToTheirOriginatedCategories()
    throws Exception {
  // 1. Create the categories structure and check the display
  createCategory("", "friends");
  createCategory("", "work");
  createCategory("work", "design-up");
  assertTrue(window.getTree().contentEquals("All\n" +
    " friends\n" +
    " work\n" +
    " design-up"));

  // 2. Create some entries in the "friends" category
  window.getTree().select("friends");
  window.getButton("New contact").click();
  window.getTextBox("first").setText("Homer");
  window.getTextBox("last").setText("Simpson");
  window.getButton("New contact").click();
  window.getTextBox("first").setText("Marge");
  window.getTextBox("last").setText("Simpson");

  // 3. Create some entries in the "work/design-up" category
  window.getTree().select("work/design-up");
  window.getButton("New contact").click();
  window.getTextBox("first").setText("Regis");
  window.getTextBox("last").setText("Medina");
  window.getButton("New contact").click();
  window.getTextBox("first").setText("Pascal");
  window.getTextBox("last").setText("Pratmarty");

  // 4. Check the contents of the root category (category "All")
  window.getTree().selectRoot();
  assertTrue(window.getTable().contentEquals(new String[][]{
    {"Homer", "Simpson", "", "", ""},
    {"Marge", "Simpson", "", "", ""},
    {"Regis", "Medina", "", "", ""},
    {"Pascal", "Pratmarty", "", "", ""},
  }));

  // 5. Check the contents of the "friends" category
  window.getTree().select("friends");
  assertTrue(window.getTable().contentEquals(new String[][]{
    {"Homer", "Simpson", "", "", ""},
    {"Marge", "Simpson", "", "", ""},
  }));

  // 6. Check the contents of the "work" category
  window.getTree().select("work");
  assertTrue(window.getTable().contentEquals(new String[][]{
    {"Regis", "Medina", "", "", ""},
    {"Pascal", "Pratmarty", "", "", ""},
  }));
}
[/prettify]

Steps 1 to 3 set up the initial environment and steps 4 to 6
check the contacts displayed for each category. For the setup part,
we use a createCategory() method that is described in
the next section. On our own projects, we usually end up creating a
lot of utility methods like this one, and extract a kind of
functional language for manipulating the GUI.

Working with Dialogs

The code of the createCategory() method is shown
below. We first click on the "New category" button, and then
intercept a modal dialog that pops up for querying the name of the
category to create.

[prettify]protected void createCategory(String parentCategoryPath, String categoryName) {
  window.getTree().select(parentCategoryPath);
  WindowInterceptor.init(window.getButton("New Category").triggerClick())
    .process(new WindowHandler() {
      Trigger process(Window dialog) {
        assertTrue(dialog.titleEquals("Category name:"));
        dialog.getInputTextBox().setText(categoryName);
        return dialog.getButton("OK").triggerClick();
    }
  })
  .run();
} 
[/prettify]

This method uses the WindowInterceptor utility
class to catch the popped-up modal dialog. This is the main class
of UISpec4J's window interception mechanism, and provides a means
for working with windows without requiring the user interface to be
showing on the screen and without requiring any change in the
production code.

Here is how this works:

  1. init(): We first initialize the
    interceptor with what we call a "trigger." A Trigger
    is a piece of code that causes a dialog to be shown by the
    application--for instance, an Open button that displays a file
    chooser, or a Delete button that displays a confirmation window.
    Many UISpec4J components offer ready-to-use components through
    triggerXXX() methods, but you can also provide your
    own implementation of the Trigger interface.

  2. process(): The window displayed by
    the trigger is caught and given to a WindowHandler
    interface. We provide a WindowHandler implementation
    that first checks that the displayed window's title is "Category
    name", then enters the value of the categoryName
    variable into the displayed text field, and then closes the window
    using its OK button.

    Implementing interceptions like this can look cumbersome, and
    one would like to be able to test the displayed window without
    having to create inner classes. But in the case of modal dialogs
    there is no choice: the application is waiting for the dialog to be
    closed, which means that the main thread of the application is
    blocked, so you need to process the dialog within a new thread. One
    of the strengths of the interception mechanisms is that they hide
    the multithreading issues associated with the management of modal
    dialogs.

    Should you need to intercept non-modal dialogs, however, things
    are much simpler. You would retrieve the window from within
    the test with a single statement; for instance:

    [prettify]Window window = WindowInterceptor.run(window.getButton("Show").triggerClick());<br />
        window.getTable() ...<br />
    window.getButton() ...
    [/prettify]
  3. run: This is when the whole
    interception gets really executed. The trigger is run, and the
    displayed window is processed by the handler. The
    run() method will throw an exception if no window is
    shown or if the displayed modal dialog is never closed by the
    handler.

It is important to note here that the UISpec4J toolkit is
ultimately responsible for handling the display of every window: if
the application tries to show a dialog box or a window from outside a
WindowInterceptor call, the toolkit will immediately
raise an exception and make the current test fail.

Using Tree Components

The tree content is being checked before filling the "friends"
and "work/design-up" categories with some contacts. As for tables,
we also use a "graphical" representation: an indented concatenation
of the string representations of the tree nodes separated by the
newline (\n) character. In our case:

[prettify]assertTrue(window.getTree().contentEquals(
    "All\n" +
    " friends\n" +
    " work\n" +
    " design-up"));
[/prettify]

You will also have noticed that tree paths are expressed with
simple strings, usually the displayed tree node names separated
with slashes--for instance, work/design-up. This policy can be
overriden quite easily when using UISpec4J's Tree
component.

Other UISpec4J Features

UISpec4J provides other interesting features that we will only
present briefly here.

Using Displayed Values Instead of Internal Ones

UISpec4J uses the values displayed by the components.
For complex components such as JList,
JTable, or JTree, this means relying on
renderers, not just the internal model of the component. For most
cases, these renderers use JLabel components, so
UISpec4J will simply retrieve these labels and check their
displayed text. In more advanced cases, you can easily customize
how String values are to be interpreted for a specific
complex component.

Extending UISpec4J to Handle New Kinds of Components

Even though the UISpec4J API is rich enough to handle most of
the situations we have had to deal with, there will always be new
situations to take into account. For instance, many projects will
want to use UISpec4J with custom-made UI components for which there
is no UISpec4J wrapper--for instance, a Table with advanced
sorting and filtering capabilities, or a Calendar
component, or any custom component coming along with third-party
libraries (e.g. JavaHelp).

UISpec4J provides an extension mechanism that allows you to
implement your own UISpec4J component wrappers, and plug them into
the library by enhancing the Panel class so that it can be used to
find them in containers as for any other UISpec4J component.

Conclusion

This article presents only an overview of UISpe4J's philosophy
and capabilities. This framework aims to ease the automated testing
of Swing-based applications:

  • A large toolset allows for writing clear test scripts with little
    effort.
  • Advanced users can extend this toolset for manipulating third-party components and checking rendered content in more detail.

Should you have any problems, questions, or suggestions, please
feel free to either or "http://tech.groups.yahoo.com/group/uispec4j/">join our forum--we look forward to hearing from you!

Resources


width="1" height="1" border="0" alt=" " />
Régis Medina is the founder of Design-up, a group of french freelance consultants specialized in helping development teams get rid of chronic problems and get better software done faster.
Alexander Potochkin is an engineer on the Swing team.
Related Topics >> Extreme Programming   |   GUI   |   Testing   |