Overview

A simple means of testing a text transformation is to provide "before" and "after" files, run the transformation on the "before" file, and check that the result matches the "after" file. (The "after" file is commonly referred to as the "golden" file.) For our import.xsl and export.xsl stylesheets, we can provide a file in the external format, and an equivalent file in s3xml. Then running import.xsl on the external file should produce the matching s3xml file, and vice versa for export.xsl -- each can serve as the golden file for the other.

To simplify testing xsl stylesheets, and thus encourage people to provide tests, we can supply a test helper that can be called from our regular test process, and to which the user supplies just pointers to the stylesheets and their before and after files.

Specifics

Test file locations

The test helper can run as a Python unit test, under control of eden/modules/unit_tests/suite.py.

The user can specify tuples of (xsl script, input file, golden file, option1 = value1, ...) where the paths are given relative to the eden root directory, and options might be used to tell the test helper such things as whether the order of repeated elements is significant. A list of tuples would constitute the full instructions to the test helper.

Where should the tuples be placed? They may as well go in a single file, in a directory under eden/modules/unit_tests.

In case people would like to add tests for other code under eden/static, we can follow the eden/static directory structure. That is, tests for stylesheets could be placed in eden/modules/unit_tests/static/formats so that, for instance, if people later want to write tests for tools, those could be placed in eden/modules/unit_tests/static/scripts/tools.

Verifying results

The transformed file will need to be compared against the golden file. Options for performing the comparison:

• For XML output, transform the input file, which will yield an etree. Read the golden file into an etree as well. Then compare the etrees.

• For S3XML output, read both using S3XML utilities and compare the S3XML trees.

• For other formats, transform the input, then convert it to a string. Read in the golden file and do a textual comparison.

String comparison is fragile, as alterations in whitespace or change in ordering of elements (where that change is not significant to the meaning) will break the test. However, this is simple, and may be appropriate for getting started, even for formats that produce XML or S3XML.

This can be made less fragile by ignoring whitespace and similar differences where they are not significant -- the user can be allowed to specify what can be ignored.

• Allow the user to provide a comparison function.

This goes against the purpose of making testing easy, so should be avoided. If this becomes necessary, the user can be allowed to specify the function in the options. These comparison functions could be added to the test helper, or placed in other files in the same directory. This goes against the purpose of making testing easy, so should be avoided.