Fitnesse Testing for Fast-Paced Agile Web Development

Contents Why Would a Development Team Release Software Updates Every Day? Why Fitnesse? HtmlFixture Asserting Text and Links Are Correct HtmlFixture Asserting Form Values Are Correct Before and After a Form Submission HtmlFixture with Preprocessing to Enable Dynamic Assertions XMLHTTPFixture Asserting XML Output Is Correct Test Writing Tips Conclusion What Next? Resources

One of the goals of an agile web software development team is to
deliver quality software in a timely fashion. For most web
applications the server-side software uses a database and the final
products are HTML and/or XML. HTML is rendered by a browser and XML
is either processed by a browser (AJAX) or exchanged with a partner
web application (business-to-business integration). "http://www.fitnesse.org/">Fitnesse tests can communicate with
the database and can make assertions against HTML and XML. Agile
web teams can use Fitnesse to create comprehensive test suites that
make system-level assertions. Once these test suites exist, the team
can release updates to their production web site as often as every
day, confident that all of the system-level requirements (both old
and new) are being met.

The purpose of this article is to illustrate the use of Fitnesse
by agile web development teams. It first will discuss some of the
reasons that agile web development teams would want to adopt a
rapid release schedule. Next it will suggest that Fitnesse is a
viable testing tool for use with this schedule. Then it will
provide a number of examples to demonstrate the simplicity and
power of Fitnesse. Finally, it will provide tips regarding Fitnesse
test writing.

In this article, the agile term "whole team" refers to the
collective group of software developers, DBAs, technical testers,
managers, and customers. Agile software development is a
discipline. For more information regarding agile software
methodologies, see Ron Jeffries's "http://www.xprogramming.com/xpmag/whatisxp.htm">XP web
site.

Why Would a Development Team Release Software Updates Every
Day?

• Short answer: Good for the business.
• The company's product is an e-product.
  Super-short releases allow the marketing department to obtain rapid
  customer feedback and to respond to that feedback quickly. They can
  test modifications to existing products and release new products
  sooner and more often. The agile concept of rapid feedback works
  well for the marketing team, too!
• It's good for search engine optimization.
  Frequently updated web sites perform better in organic search
  engine results. This enables the business to attract customers to
  its web site without paying for sponsored links.
• To respond to changing customer needs. Businesses
  that support diverse customer niches demand more agility from their
  products.
• For contractual agreements with e-partners.
  Contractual agreements involving business-to-business web site
  integration can dictate when the software is released.

The rapid release schedule accommodates a wide range of sizes
and types of projects, because each day the whole team has the
opportunity to release software updates, regardless of how long the
updates took to develop. Although the whole team could probably get
by without releasing its software quite as often, once a rapid
release schedule is adopted, the team members will wonder how they
ever managed without it.

Simple web sites can be managed effectively by a content
management system and require little developer, DBA, and technical
tester work. However, more complex web sites require software
customizations to provide tracking services for their e-products.
This tracking can include page impressions, clicked links, dynamic
advertising, and links to partner sites, which can then be customized
for each e-product. These types of requirements are dynamic enough
to keep patterns in the software design from forming, thereby
forcing the whole team to release new products more often.

Why Fitnesse?

• It's easy to use. Web developers, DBAs, technical
  testers, managers, and customers can all learn how to use wiki
  markup to make test assertions.
• It's easy to install and lightweight to run. Get
  started with Fitnesse by running it on a developer
  workstation.
• It performs assertions against HTML, XML, Java, and the
  database, all from a single tool. The complexity of your
  requirements will not outgrow Fitnesse's capabilities, because it
  can be extended to test anything testable with Java. Test writing
  is limited only by the whole team's imagination.
• It's open source and well documented.

Agile web developers write Java software to test their Java
objects, web pages, databases, and XML documents. "http://www.junit.org/">JUnit is a popular testing framework
for regression testing Java technologies. Fitnesse is analogous to
JUnit in that it is a testing engine built using Java technologies.
However, Fitnesse is different because its user interface is a web
application with test suites created and managed using wiki
markup. The key difference is that JUnit is primarily used by
developers, whereas Fitnesse's user interface is friendly to
non-developers, too. The developer extends Fitnesse's testing
engine to expose new assertion methods. Then any team member
(technical or not) can use the wiki markup to populate and run
these new assertion methods. In the end, the developers use both
JUnit and Fitnesse; each for different purposes.

Fitnesse describes itself as a "fully integrated standalone
wiki, and acceptance testing framework" that runs test assertions
using wiki markup. The wiki markup is used to create test tables
whose assertions are run inside of the Java web FitServer. Upon
completion of the test table, the FitServer displays a green-colored cell for each correct assertion and a red-colored cell for
each failed assertion.

A "http://www.fitnesse.org/FitNesse.FixtureCode">fixture is a
Java class that parses a wiki test table and then connects the test
table to methods in the Java class. Fixtures support input
parameters and connect a method's output to the wiki's display. For
teams that want to quickly get started, there are HTML and database
fixtures that can
be installed and used without customization. For teams that need
more functionality, the complexity of the assertions is limited
only by the whole team's time and imagination, because Fitnesse's
Java test fixtures are meant to be extended and customized. For
instance, the developer team can write Java software that queries a
database and then uses that data to assert that a web page is
rendering that data correctly in HTML or XML.

HtmlFixture Asserting Text and Links Are Correct

Project requirements:

• Create a home page with title, meta-keywords, and a meta-description.
• Create a "Daily News" page with title, meta-keywords, and a meta-description.
• Create a link from the home page to the "Daily News" page.
• Use the mock-up in Figure 1 to create the display.

"Mock-up of horizontal navigation including the Daily News link" />
Figure 1. Mock-up of horizontal navigation including the "Daily
News" link

The HtmlFixture test table below loads a web page,
asserts that the title and meta tags are correct, asserts that the
"Daily News" display text and URL are correct, and then clicks on
the link to prove that it works. Use Fitnesse "http://fitnesse.org/FitNesse.MarkupVariables">markup variables
to define the HtmlFixture's URL. This will allow you
to quickly toggle the environment (develop, test, production) being
tested. The domain markup variable is defined in the
root suite so that the test runner can control which environment
all of the HTML tests will run against.

[prettify]!define domain {www.jhound.com}

[/prettify]

Also, give each tested HTML element a unique id tag
so that the Element Focus command can be used to find
and focus on the element.

[prettify]<a id="rssMenuLink" href="/rss/index.do">Daily News</a>

[/prettify]

Here is the example wiki source code. The pipe (|)
character starts and ends cells and rows.

[prettify]!|com.jbergin.HtmlFixture|
|http://${domain}/|
|Element Focus|jhoundTitle|title|
|Text|www.JHound.com JavaHound- Welcome to 
 JavaHound|
|Element Focus|Description|meta|
|Attribute|name|Description|
|Attribute|content|Find Java information: a hub 
 for information regarding Java, Extreme 
  Programming, Web Services, and Java 
  certifications.|
|Element Focus|Keywords|meta|
|Attribute|name|Keywords|
|Attribute|content|java j2ee, java junit, java 
 hibernate, java web services, java extreme 
 programming, java testing, java web applications, 
 java struts, java servlets, java jsp, java xml, 
 java certification, java certification quiz, java 
 object/relational mapping, javaone 2004|
|Element Focus|rssMenuLink|a|
|Attribute|href|/rss/index.do|
|Text|Daily News|
|Click|nowOnDailyNewsPage|
|Element Focus|jhoundTitle|title|
|Text|www.JHound.com JavaHound- JavaHound RSS|
|Element Focus|Description|meta|
|Attribute|name|Description|
|Attribute|content|Read daily news articles|
|Element Focus|Keywords|meta|
|Attribute|name|Keywords|
|Attribute|content|java technology news, sports 
 news, world news, us news, political news, java 
 news, extreme programming news, j2ee news|

[/prettify]

Here is the line-by-line description:

1. Defines the test table's fixture type as
   HtmlFixture.
2. HTMLUnit requests a web page and parses its HTML with an XML
   parser.
3. Finds and focuses on a HTML title element with an
   id="jhoundTitle".
4. Asserts that its text is correct.
5. Finds and focuses on a HTML meta element with an
   id="Description".
6. Asserts that its name attribute is Description.
7. Asserts that its content attribute's text is correct.
8. Finds and focuses on a HTML meta element with an
   id="Keywords".
9. Asserts that its name attribute is Keywords.
10. Asserts that its content attribute's text is correct.
11. Finds and focuses on a HTML anchor element with an
    id="rssMenuLink".
12. Asserts that its href attribute is correct.
13. Asserts that its text is correct.
14. Clicks on the link; HTMLUnit then parses the response HTML.
15. (15-22) Proves that clicking the link worked by asserting that
    the "Daily News" page's title and meta tags are correct.

Figure 2 shows what the HtmlFixture test table
looks like after it has finished running with all of its assertions
passing.

Figure 2. HtmlFixture asserting display text and links are correct--click image for full-size screen shot

Figure 3 shows what the HtmlFixture test table
looks like after it has finished running, with one of its assertions
failing.

Figure 3. HtmlFixture asserting display text is incorrect--click image for full-size screen shot

Steps needed to run this example:

1. "http://www.fitnesse.org/FitNesse.DownloadingAndInstallingFitNesse">
   Download and install Fitnesse.
2. Download and
   install the HtmlFixture into Fitnesse.
3. Create a Fitnesse test page with the aforementioned wiki source
   code.

HtmlFixture Asserting Form Values Are Correct Before and After
a Form Submission

Project requirements:

• Create a quiz question with four possible answers.
• The second answer option (or "B") is the correct answer.
• When the user selects the correct answer, it should be
  highlighted in green and the submit button text should change to
  "Correct, Next Question."
• Use the mock-ups in Figures 4 and 5 to create the display.

"Mock-up of a quiz question before the user has made a selection" />

Figure 4. Mock-up of a quiz question before the user has made a
selection

width="437" height="153" alt=
"Mock-up of a quiz question after the user has made a selection" />
Figure 5. Mock-up of a quiz question after the user has made a
selection

The HtmlFixture test table loads a web page,
submits its form, and then asserts that the submission result
displays the "Correct Answer" page. Here is the example wiki source
code:

[prettify]!|com.jbergin.HtmlFixture|
|http://${domain}/java/quiz.do|
|Element Focus|radio_answerB|input|
|Set Value|checked|
|Element Focus|form_quiz|form|
|Submit|CorrectAnswerPage|
|Element Focus|submitButton_correct|input|
|Attribute|value|Correct, Next Question|
|Attribute|type|submit|

[/prettify]

Here is the line-by-line description:

1. Defines the test table's fixture type as
   HtmlFixture.
2. HTMLUnit requests a web page and parses its HTML with an XML
   parser.
3. Finds and focuses on a HTML input element with an
   id="answerB".
4. Sets the second radio button (or answer B) to checked.
5. Finds and focuses on the HTML form element with an
   id="form_quiz".
6. Submits the form; HTMLUnit then parses the response HTML.
7. Finds and focuses on a HTML input element with an
   id="submitButton_correct".
8. Asserts that its value attribute is

   Correct, Next
   Question

   .
9. Asserts that its type attribute is submit.

Figure 6 shows what the HtmlFixture test table
looks like after it has finished running with all of its assertions
passing.

Figure 6. HtmlFixture asserting form values are correct before and after a form submission--click image for full-size screen shot

Steps needed to run this example:

1. "http://www.fitnesse.org/FitNesse.DownloadingAndInstallingFitNesse">
   Download and install Fitnesse.
2. Download and
   install the HtmlFixture into Fitnesse.
3. Create a Fitnesse test page with the aforementioned wiki source
   code.

HtmlFixture with preprocessing to enable dynamic
assertions

Project requirements:

• Update the quiz question text and dollar values from "?page=2#Figure4">Figure 4 to be database-driven.

The domain and database_level "http://fitnesse.org/FitNesse.MarkupVariables">markup variables
are defined so that the test runner can control which environment
the test will run against:

[prettify]!define database_level {prod}
!define domain {www.jhound.com}

[/prettify]

This example is leveraging the Symbol command in
HtmlFixture. The Symbol command is used
within a HtmlFixture test table to create symbol/value
pairs. These symbol/value pairs can be referenced later in the same
test table or, as in this example, subsequent test tables. Instead
of using a static HtmlFixture test table to create the
symbol/value pairs, the first test table is a customized "http://fitnesse.org/FitNesse.FixtureCode">ColumnFixture
that dynamically preloads HtmlFixture's symbol
key/value pairs. Here is the example wiki source code for the first
test table:

[prettify]!|com.jhound.fitnesse.DynamicSymbolSettingFixture|
|domain|database_level|loadDynamicData?|
|${domain}|${database_level}|symbols successfully 
 loaded|

[/prettify]

Here is the line-by-line description:

1. Defines the test table's fixture type as
   DynamicSymbolSettingFixture.
2. Sets DynamicSymbolSettingFixture's publicly
   defined properties domain and
   database_level to the values defined in line 3. Calls
   DynamicSymbolSettingFixture's
   loadDynamicData() method.
3. Displays the values for domain and database_level. Asserts that DynamicSymbolSettingFixture's
   loadDynamicData() method returns

   symbols successfully
   loaded

   .

Here is DynamicSymbolSettingFixture's Java source
code. This fixture code processes the first test table and loads
the symbol values at runtime from a database:

[prettify]package com.jhound.fitnesse;

import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import com.jbergin.HtmlFixture;
import fit.ColumnFixture;

public class DynamicSymbolSettingFixture
      extends ColumnFixture {
  /*
   * domain and database_level need to be public
   * so that the Fitnesse engine can set their 
   * values.
   */
  public String domain; 
  public String database_level;

  public String loadDynamicData() {
    String returnMessage = null;
    String questionLevel = null;
    String questionText = null;
    Connection c = null;
    try {
      c = ConnectionManager.getConnection(
                           this.database_level);
      PreparedStatement pStmt = c
        .prepareStatement(
         "select question_level, question_text "
       + "from quiz_questions "
       + "where question_level = 100 ");
      ResultSet rs = pStmt.executeQuery();
      if (rs.next()) {
        questionLevel = rs
              .getString("question_level");
        questionText = rs
              .getString("question_text");
      } else {
        returnMessage = 
          "Error: did not find any data!";
      }
      rs.close();
      pStmt.close();
    } catch (SQLException e) {
      returnMessage = "Error: " + e;
    } 
    finally {
      if (c != null) {
        try {
          c.close();
        } catch (SQLException e1) {
          // do nothing
        }
      }
    }
    /*
     * Set symbol key/value pairs that will be 
     * accessible from within the HtmlFixture test
     * table.
     */
    HtmlFixture.setSymbol("url", "http://"
          + this.domain + "/java/quiz.do");
    HtmlFixture.setSymbol("questionValue", "$"
          + questionLevel + " Question: ");
    HtmlFixture.setSymbol("questionText",
          questionText);

    return (returnMessage == null) 
          ? "symbols successfully loaded"
          : returnMessage;
  }
}

[/prettify]

The second test table is a HtmlFixture. The static
URLs and static assertion values, as seen in previous examples, are
replaced by the symbols url,
questionValue, and questionText. The
symbols are used to create this HtmlFixture test table
at runtime. Here is the example wiki source code for the second
test table:

[prettify]!|com.jbergin.HtmlFixture|
|symbol url|
|Element Focus|span_scoreValue|span|
|Text|symbol questionValue|
|Element Focus|span_question|span|
|Text|symbol questionText|

[/prettify]

Here is the line-by-line description:

1. Defines the test table's fixture type as
   HtmlFixture.
2. HTMLUnit requests a web page using the url symbol
   and parses its HTML with an XML parser.
3. Finds and focuses on a HTML span element with an
   id="span_scoreValue".
4. Asserts that its text is correct using the
   questionValue symbol.
5. Finds and focuses on a HTML span element with an
   id="span_question".
6. Asserts that its text is correct using the
   questionText symbol.

This test will remain valid because the assertions are connected
to the database. The test remains valid from day to day and in
different environments (develop, test, production) because the test
dynamically accounts for any changes. Figure 7 shows what the
DynamicSymbolSettingFixture and
HtmlFixture test tables look like after they have
finished running with all of their assertions passing.

Figure 7. HtmlFixture with preprocessing to enable dynamic assertions--click image for full-size screen shot

Steps needed to run this example:

1. "http://www.fitnesse.org/FitNesse.DownloadingAndInstallingFitNesse">
   Download and install Fitnesse.
2. Download and
   install the HtmlFixture into Fitnesse.
3. Create a customized "http://fitnesse.org/FitNesse.FixtureCode">ColumnFixture
   that preloads HtmlFixture's symbol key/value
   pairs.
4. Create a Fitnesse test page with the aforementioned wiki source
   code.

XMLHTTPFixture Asserting XML Output Is Correct

Project requirements:

• Allow users of the web site to download the quiz's questions
  and answers in XML.

Figure 8 shows the XML document that the
XMLHTTPFixture test table is going to test.

Figure 8. XML output being tested--click image for full-size screen shot

This fixture depends upon "http://xmlunit.sourceforge.net/">XMLUnit 1.0 and "?page=3#resources">www.JHound.com Fixtures. This test table is
similar to the HtmlFixture test tables in Figures 2, 3, and 6. The
difference is that it loads an XML (not HTML) document over HTTP and
then uses XPath expressions to assert that the document is both
correctly formed and that it contains the correct data. The
XMLHTTPFixture is usable without modification and
currently has three assertion commands:

1. Exists: assert that a XML element or attribute
   exists. Use @attributeName for attributes.
2. Not Exists: assert that a XML element does not
   exist.
3. Value: assert that a XML element or attribute has
   a particular text value. Use @attributeName for attributes.

Here is the example wiki source code:

[prettify]!|com.jhound.fitnesse.XMLHTTPFixture|
|http://${domain}/java/quiz-xml.do|
|Exists|//www.jhound.com|
|Exists|//www.jhound.com/certification-millionaire
 |
|Exists|//www.jhound.com/certification-millionaire
 /questions|
|Exists|//www.jhound.com/certification-millionaire
 /questions/question[1]|
|Exists|//www.jhound.com/certification-millionaire
 /questions/question[1]/@value|
|Value|//www.jhound.com/certification-millionaire
 /questions/question[1]/@value|$100|
|Exists|//www.jhound.com/certification-millionaire
 /questions/question[1]/problem|
|Value|//www.jhound.com/certification-millionaire
 /questions/question[1]/problem|Which of the 
 following correctly declares and initializes an 
 array of Strings?|
|Exists|//www.jhound.com/certification-millionaire
 /questions/question[1]/answer[1]|
|Value|//www.jhound.com/certification-millionaire
 /questions/question[1]/answer[1]|String[] array =
 {"hello"; "bye";"testing"};|
|Exists|//www.jhound.com/certification-millionaire
 /questions/question[1]/answer[2]|
|Value|//www.jhound.com/certification-millionaire
 /questions/question[1]/answer[2]|String[] array =
 {"hello", "bye","testing"};|
|Exists|//www.jhound.com/certification-millionaire
 /questions/question[1]/answer[3]|
|Value|//www.jhound.com/certification-millionaire
 /questions/question[1]/answer[3]|String[3] array 
 = {"hello", "bye","testing"};|
|Exists|//www.jhound.com/certification-millionaire
 /questions/question[1]/answer[4]|
|Value|//www.jhound.com/certification-millionaire
 /questions/question[1]/answer[4]|String[] array =
 new String[3] {"hello", "bye","testing"};|
|Not Exists|//www.jhound.com/certification-
 millionaire/questions/question[1]/answer[5]|

[/prettify]

Here is the line-by-line description:

1. Defines the test table's fixture type as
   XMLHTTPFixture.
2. Opens a java.net.URLConnection and loads the
   requested URL into a org.w3c.dom.Document object.
3. (3-19) XMLUnit's SimpleXpathEngine is used to assert
   that the Document object's expected elements and
   attributes exist and that they contain the correct the data.

Figure 9 shows what the XMLHTTPFixture test table
looks like after it has finished running with all of its assertions
passing.

Figure 9. XMLHTTPFixture asserting XML output is correct--click image for full-size screen shot

Figure 10 shows what the XMLHTTPFixture test table
looks like after it has finished running with a couple of its
assertions failing.

Figure 10. XMLHTTPFixture asserting XML output is incorrect--click image for full-size screen shot

With a little more work to the XMLHTTPFixture, these
test tables could be made dynamic like the test tables in "?page=2#Figure7">Figure 7.

Steps needed to run this example:

1. "http://www.fitnesse.org/FitNesse.DownloadingAndInstallingFitNesse">
   Download and install Fitnesse.
2. Download and
   install XMLUnit 1.0 into Fitnesse.
3. Download and install the
   XMLHTTPFixture into Fitnesse.
4. Create a Fitnesse test page with the aforementioned wiki source
   code.

Test Writing Tips

• Use Fitnesse to create system-level test suites for web
  applications. Test the HTML output with
  HtmlFixture. Start with HtmlFixture
  because it is easy to use, and then graduate to more dynamic
  tests.
• Employ developers to write the Fitnesse tests.
  Developers should apply the "http://www.xprogramming.com/xpmag/whatisxp.htm#test">test-first
  development technique, used in writing unit-level software, to
  their system-level test writing. Test-first development's rapid
  feedback is an important part of agile software development. When
  developers write the Fitnesse tests, they are forced to understand
  the problem at the system level, because they must think about how
  they are going to integrate their unit-level code into the larger
  system and then test that the larger system works. Once a developer
  has passed the system-level test, they know the task is complete.
  Technical testers then have the freedom to expand system-level
  tests.
• Run the tests as part of the whole team's continuous
  integration. Run the test suites before and after every
  promotion. Set up the tests to run on a scheduled basis. As the
  test suite grows it will take longer and longer to run. Therefore,
  running the suite on a scheduled basis allows the whole team to check
  the results without having to wait for them to finish.
• Make the tests dynamic from both user interface and
  data perspectives. Click on links and submit forms to prove
  that the page flow integrates successfully. Also, connect
  assertions to the database so that the tests remain valid from day
  to day and in different environments (development, testing,
  production). If the data being tested can change, then the test
  should dynamically account for those changes.
• Do small releases. It is easier to maintain
  the correctness of Fitnesse tests when, at any given time, there
  are only a small number of tests failing. While a project is
  actively being developed, its Fitnesse tests will fail in the
  production environment. Therefore, longer-running projects will
  cause a lot of the Fitnesse tests to fail against the production
  environment for an extended period of time. This will inhibit the
  whole team's ability to measure the correctness of the system.
• Put all Fitnesse test tables into one Fitnesse test
  page when testing the same requirement on multiple web
  pages. Putting all of the Fitnesse test tables into one
  Fitnesse test page ensures that the tests will only needed to be
  updated in one place when the requirement changes.
• Group the tests into suites by product or
  component. Ideally, all of the tests would be run each time
  any software update is made. However, as the test suite grows and
  takes longer to run, the whole team is more likely to take the easy
  road and not run all of the tests. Therefore, it is important to be
  able to run a subset of tests for a single product or
  component.

Conclusion

This article demonstrated the power of Fitnesse as a system-level testing tool for web applications. Developing comprehensive
system-level tests will provide rapid feedback to the whole team,
thus preventing mistakes in integration and promotion. The tests
track the correctness of the system and will eventually become as
important as the web software they test. Having this test safety
net allows the whole team to maintain a rapid release schedule and
deliver smaller projects simultaneously with larger ones. In
conclusion, Fitnesse is good for the whole team (web developers,
DBAs, technical testers, managers, and customers) because everyone
can get involved in Fitnesse test writing and running.

What Next?

The next advancement in your Fitnesse test writing is to further
integrate the tests with the database. For example, what if a web
application needs to record the number of times a particular link
has been clicked? The answer is to write and run a custom
ColumnFixture to check the current number of clicks,
then run a HtmlFixture test table to click the link,
and then re-run the custom ColumnFixture test table
again to confirm that the number of clicks has increased by
one.

Resources

• Java fixture code for this
  article, including www.JHound.com fixtures
• Wiki markup
  for this article
• Fitnesse
• Fitnesse
  for HTML
• HTMLUnit
• JUnit
• XMLUnit
• "http://www.xprogramming.com/xpmag/whatisxp.htm">XP/Agile
  methodologies as defined by Ron Jeffries

The author would like to thank Gary Brown for teaching him
the agile development methodology. He would also like to thank
Kelli Moran-Miller and Travis Fritts for reviewing this article and
providing invaluable feedback.