TestDox: create simple documentation from test case names in C++

Simple ideas can be powerful. TestDox is based on such an idea: Test method names should be sentences. Using this convention, TestDox can generate a readable overview of your tests, as in:

CustomerLookup
 – finds customer by id
 – fails for duplicate customers

It was Chris Stevenson's idea to present the camel case method names in JUnit test cases as sentences and in 2003 he created the Java program TestDox. The idea has spread to tools or unit test frameworks for several programming languages such as C# (.net) and PHP and it seems common for frameworks for Behavior Driven Development (BDD). However I haven't come across it in conventional C++ unit test frameworks (check-it).

The original Java TestDox just creates an overview of the test suites and their test cases: it documents the evolving design and code in an agile way. To date unit test frameworks may be able to create the overview as part of the test run and also indicate the test's success or failure as in:

CustomerLookup
 [x] finds customer by id
 [ ] fails for duplicate customers

See for example PHPUnit, Other Uses for Tests: Agile Documentation. Although I don't know of any non-BDD C++ test frameworks that provide TestDox reporting off-the-shelf, many frameworks may support it via customization of their reporting mechanism. Below I intend to present several examples of such customization. In case you cannot create the TestDox report from your test framework, or do not want to do so, the Python TestDox script presented here may be of help.

Python TestDox script

The Python TestDox script was initially created for use with the C++ test frameworks Boost.Test and CppUnit. It is distributed under the Boost open source license. Feel free to grab the source code and change it to your own needs. Please let me know if you made any substantial changes or additions. We then can add those to testdox.py, so that other users also may benefit from them.

Recognized test name patterns

TestDox recognizes test names that start with test, testThat, or with prefix_test[That], such as:

• testSystemExceptionIsCatchedWithCorrectReason
• testThatSystemExceptionIsCatchedWithCorrectReason
• format_testThatStringIsCorrectlyFormatted

In its report, TestDox presents these names as follows:

• System exception is catched with correct reason.
• format: String is correctly formatted.

TestDox help screen

Usage: testdox.py [options] [directory | file...]

Create test documentation from testsuite files based on testdox format.  See
also: http://blog.dannorth.net/introducing-bdd/

Options:
  --version        show program's version number and exit
  -h, --help       show this help message and exit
  -d, --debug      show debug messages
  -v, --verbose    show more messages
  -q, --quiet      show less messages
  -t, --selftest   perform selftest; can also use option -v
  --title=title    specify title for report [none]
  --framework=fw   select test framework: Boost.Test, CppUnit [Boost.Test]
  --format=format  select output format: html, text [text]
  --glob=pattern   filename pattern to use with directories [*.cpp|*.h]
  --depth=n        directory recursion depth, norecurse is 1 [recurse]
  --norecurse      prevent visiting subdirectories [no]
  --htmlbodyonly   only generate contents of body tag fragment [no]

TestDox output examples

• CppUnit example
• Boost.Test example

Framework customization

Boost.Test

• View customization
• Boost.Test at Boost

UnitTest++

• View customization
• UnitTest++ on Google Code, svn repository
• UnitTest++ on SourceForge

Page created 6 September 2010

Martin Moene Electronics Department · Leiden Institute of Physics · Leiden University