Next Previous

Retired Document

Important: This document may not represent best practices for current development. Links to downloads and other resources may no longer be valid.

Testing HeaderDoc

Beginning in version 8.7, HeaderDoc includes a regression test suite that makes it easier to modify the parser code without causing regressions. This chapter describes how to use the test suite.

Obtaining a HeaderDoc Tarball

Beginning in HeaderDoc 8.8, the test suite is installed as part of the HeaderDoc installation. Thus, you can run the test suite without downloading a source tarball. However, the main reason to run it is if you are modifying the source code, in which case it is much easier to work with a standalone copy of HeaderDoc.

You can download the HeaderDoc tarball from www.opensource.apple.com. Click the latest version of OS X on that page, then search for headerdoc on the resulting page and click the download link to the right.

Running the Tests

You can run tests in two ways: all at once or individually.

To run all of the tests at once, type the following commands:
# From an installed copy:
headerdoc2html -T run
# From a standalone copy:
cd /path/to/headerdoc-version
./headerDoc2HTML.pl -T run
HeaderDoc runs the complete battery of tests and produces a summary of the results at the end.

To run a single test or a group of tests, specify their path or paths after the run subcommand. For example:

# From an installed copy

headerdoc2html -T run /path/to/Xcode.app/Contents/Developer/usr/share/headerdoc/testsuite/parser_tests/PHP_function_1.test

# From a standalone copy:

./headerDoc2HTML.pl -T run testsuite/parser_tests/PHP_function_1.test

Handling Test Failures

When using the run subcommand, you get only a summary output telling what test failed. Finding out exactly what went wrong can be more challenging. Here are the basic steps:

Use the update subcommand as though you were trying to update test results (headerDoc2HTML.pl -T update [optional list of tests]). When a test fails, HeaderDoc displays the test result differences.
Type less and press return to get a contextual diff of some of the more verbose sections of the result.
If you see that the change is expected (if you made a change to the code and the new results reflect a desired change), type confirm to update the test results with the new data.
Important: This step will fail if you are running an installed copy of HeaderDoc because the test suite is not installed in a writable location. See Obtaining a HeaderDoc Tarball to learn how to download and extract a HeaderDoc source tarball.
If the test results are not as expected, type skip to go on to the next test without updating the test results.

Creating a Test

Creating a test is somewhat more involved than running one. To create a test, you need to have a problematic HeaderDoc comment block and declaration. After you have these, perform the following steps:

Create a minimal test case header or script that contains only the declaration that causes the problem (and the class surrounding it, if applicable).
Rename all classes, functions, variables, and so on to have a name that is unique across the entire test set.
Important: The HeaderDoc test suite does not reset the symbol name cache between tests so that certain tests can verify the correctness of the API reference (apple_ref) collision handling code. This also means that you must name all functions, variables, and so on in a way that is unique across all of the tests. If you do not, your tests will behave differently when run individually than when run as a group.
If you include the name of your test (with underscores for spaces) as part of every symbol name, your names will almost never collide with those in any other test. Be sure to do this for all names, including constants within enumerations.
Fix the bugs in HeaderDoc so that the code is parsed correctly.
Run the existing tests to make sure you didn’t break anything else.
Type headerDoc2HTML.pl -T create to create a new test.
Choose a unique name for the test. The name for the test must be unique after spaces and special characters are replaced by underscores. For a list of existing tests, look in the subdirectories within the testsuite directory.
In general, the names of tests in languages other than C or C++ should begin with the name of the programming language.
Choose a programming language for the test. (This must match the declaration.)
If you are creating a C language test, choose whether you are creating a C preprocessor test or a parser test.
Parser tests are intended to test both the code parser and comment handling code. Most of the time, you should be writing a parser test. Parser tests are limited to a single declaration per test, however, and thus are unsuitable for determining whether a C preprocessor directive correctly modifies a declaration.
C preprocessor tests are intended to test whether a C preprocessor macro correctly alters a declaration. Unlike a parser test, the C preprocessor test does not test the comment handling code. It takes two blocks of code. The first block is a series of C preprocessor macros that are all parsed. The second is a declaration to operate on. This declaration is parsed and information about that declaration is stored in the test results.

From this point on, the steps differ based on the type of test you choose.

Creating a Parser Test

To create a parser test, after completing the steps in Creating a Test, do the following:

Paste the HeaderDoc comment block, then press Control-D on a new line to end the comment.
Paste the code, then press Control-D on a new line to end the declaration.
Type in an explanatory message to describe how the test case differs from other similar tests, then press Control-D on a new line to end the message.
If you are overwriting an existing test case, HeaderDoc asks you to confirm that you intended to do so. In general, say no unless you made a mistake previously and are overwriting a test you just created.
Wait for HeaderDoc to build the test data.
Submit the test case and patches via bugreport.apple.com.

Creating a C Preprocessor Test

C preprocessor macro tests apply a series of C preprocessor macros to a declaration (which can be anything from a C preprocessor macro to a class).

To create a C preprocessor test, complete the steps in Creating a Test, then do the following:

Paste the block of C preprocessor macros, then press Control-D on a new line to end the list of macros.
Paste the declaration that the C preprocessor macros should modify, then press Control-D on a new line to end the declaration.
Type an explanatory message to describe how the test case differs from other similar tests, then press Control-D on a new line to end the message.
If you are overwriting an existing test case, HeaderDoc asks you to confirm that you intended to do so. In general, say no unless you made a mistake previously and are overwriting a test you just created.
Wait for HeaderDoc to build the test data.
Submit the test case and patches via bugreport.apple.com.

Next Previous