XMLHTTPFixture Asserting XML Output Is Correct
Project requirements:
- Allow users of the web site to download the quiz's questions
and answers in XML.
Figure 8 shows the XML document that the
XMLHTTPFixture test table is going to test.
|
Figure 8. XML output being tested--click image for full-size screen shot
|
This fixture depends upon XMLUnit 1.0 and www.JHound.com Fixtures. This test table is
similar to the HtmlFixture test tables in Figures 2, 3, and 6. The
difference is that it loads an XML (not HTML) document over HTTP and
then uses XPath expressions to assert that the document is both
correctly formed and that it contains the correct data. The
XMLHTTPFixture is usable without modification and
currently has three assertion commands:
Exists: assert that a XML element or attribute
exists. Use @attributeName for attributes.Not Exists: assert that a XML element does not
exist.Value: assert that a XML element or attribute has
a particular text value. Use @attributeName for attributes.
Here is the example wiki source code:
!|com.jhound.fitnesse.XMLHTTPFixture|
|http://${domain}/java/quiz-xml.do|
|Exists|//www.jhound.com|
|Exists|//www.jhound.com/certification-millionaire
|
|Exists|//www.jhound.com/certification-millionaire
/questions|
|Exists|//www.jhound.com/certification-millionaire
/questions/question[1]|
|Exists|//www.jhound.com/certification-millionaire
/questions/question[1]/@value|
|Value|//www.jhound.com/certification-millionaire
/questions/question[1]/@value|$100|
|Exists|//www.jhound.com/certification-millionaire
/questions/question[1]/problem|
|Value|//www.jhound.com/certification-millionaire
/questions/question[1]/problem|Which of the
following correctly declares and initializes an
array of Strings?|
|Exists|//www.jhound.com/certification-millionaire
/questions/question[1]/answer[1]|
|Value|//www.jhound.com/certification-millionaire
/questions/question[1]/answer[1]|String[] array =
{"hello"; "bye";"testing"};|
|Exists|//www.jhound.com/certification-millionaire
/questions/question[1]/answer[2]|
|Value|//www.jhound.com/certification-millionaire
/questions/question[1]/answer[2]|String[] array =
{"hello", "bye","testing"};|
|Exists|//www.jhound.com/certification-millionaire
/questions/question[1]/answer[3]|
|Value|//www.jhound.com/certification-millionaire
/questions/question[1]/answer[3]|String[3] array
= {"hello", "bye","testing"};|
|Exists|//www.jhound.com/certification-millionaire
/questions/question[1]/answer[4]|
|Value|//www.jhound.com/certification-millionaire
/questions/question[1]/answer[4]|String[] array =
new String[3] {"hello", "bye","testing"};|
|Not Exists|//www.jhound.com/certification-
millionaire/questions/question[1]/answer[5]|
Here is the line-by-line description:
- Defines the test table's fixture type as
XMLHTTPFixture.
- Opens a
java.net.URLConnection and loads the
requested URL into a org.w3c.dom.Document object.
- (3-19) XMLUnit's
SimpleXpathEngine is used to assert
that the Document object's expected elements and
attributes exist and that they contain the correct the data.
Figure 9 shows what the XMLHTTPFixture test table
looks like after it has finished running with all of its assertions
passing.
|
Figure 9. XMLHTTPFixture asserting XML output is correct--click image for full-size screen shot
|
Figure 10 shows what the XMLHTTPFixture test table
looks like after it has finished running with a couple of its
assertions failing.
|
Figure 10. XMLHTTPFixture asserting XML output is
incorrect--click image for full-size screen shot
|
With a little more work to the XMLHTTPFixture, these
test tables could be made dynamic like the test tables in Figure 7.
Steps needed to run this example:
-
Download and install Fitnesse.
- Download and
install XMLUnit 1.0 into Fitnesse.
- Download and install the
XMLHTTPFixture into Fitnesse.
- Create a Fitnesse test page with the aforementioned wiki source
code.
Test Writing Tips
- Use Fitnesse to create system-level test suites for web
applications. Test the HTML output with
HtmlFixture. Start with HtmlFixture
because it is easy to use, and then graduate to more dynamic
tests.
- Employ developers to write the Fitnesse tests.
Developers should apply the test-first
development technique, used in writing unit-level software, to
their system-level test writing. Test-first development's rapid
feedback is an important part of agile software development. When
developers write the Fitnesse tests, they are forced to understand
the problem at the system level, because they must think about how
they are going to integrate their unit-level code into the larger
system and then test that the larger system works. Once a developer
has passed the system-level test, they know the task is complete.
Technical testers then have the freedom to expand system-level
tests.
- Run the tests as part of the whole team's continuous
integration. Run the test suites before and after every
promotion. Set up the tests to run on a scheduled basis. As the
test suite grows it will take longer and longer to run. Therefore,
running the suite on a scheduled basis allows the whole team to check
the results without having to wait for them to finish.
- Make the tests dynamic from both user interface and
data perspectives. Click on links and submit forms to prove
that the page flow integrates successfully. Also, connect
assertions to the database so that the tests remain valid from day
to day and in different environments (development, testing,
production). If the data being tested can change, then the test
should dynamically account for those changes.
- Do small releases. It is easier to maintain
the correctness of Fitnesse tests when, at any given time, there
are only a small number of tests failing. While a project is
actively being developed, its Fitnesse tests will fail in the
production environment. Therefore, longer-running projects will
cause a lot of the Fitnesse tests to fail against the production
environment for an extended period of time. This will inhibit the
whole team's ability to measure the correctness of the system.
- Put all Fitnesse test tables into one Fitnesse test
page when testing the same requirement on multiple web
pages. Putting all of the Fitnesse test tables into one
Fitnesse test page ensures that the tests will only needed to be
updated in one place when the requirement changes.
- Group the tests into suites by product or
component. Ideally, all of the tests would be run each time
any software update is made. However, as the test suite grows and
takes longer to run, the whole team is more likely to take the easy
road and not run all of the tests. Therefore, it is important to be
able to run a subset of tests for a single product or
component.
Conclusion
This article demonstrated the power of Fitnesse as a system-level testing tool for web applications. Developing comprehensive
system-level tests will provide rapid feedback to the whole team,
thus preventing mistakes in integration and promotion. The tests
track the correctness of the system and will eventually become as
important as the web software they test. Having this test safety
net allows the whole team to maintain a rapid release schedule and
deliver smaller projects simultaneously with larger ones. In
conclusion, Fitnesse is good for the whole team (web developers,
DBAs, technical testers, managers, and customers) because everyone
can get involved in Fitnesse test writing and running.
What Next?
The next advancement in your Fitnesse test writing is to further
integrate the tests with the database. For example, what if a web
application needs to record the number of times a particular link
has been clicked? The answer is to write and run a custom
ColumnFixture to check the current number of clicks,
then run a HtmlFixture test table to click the link,
and then re-run the custom ColumnFixture test table
again to confirm that the number of clicks has increased by
one.
The author would like to thank Gary Brown for teaching him
the agile development methodology. He would also like to thank
Kelli Moran-Miller and Travis Fritts for reviewing this article and
providing invaluable feedback.
