Overview

A simple means of testing a text transformation is to provide "before" and "after" files, run the transformation on the "before" file, and check that the result matches the "after" file. (The "after" file is commonly referred to as the "golden" file.) For our import.xsl and export.xsl stylesheets, we can provide a file in the external format, and an equivalent file in s3xml. Then running import.xsl on the external file should produce the matching s3xml file, and vice versa for export.xsl -- each can serve as the golden file for the other.

To simplify testing xsl stylesheets, and thus encourage people to provide tests, we can supply a test helper that can be called from our regular test process, and to which the user supplies just pointers to the stylesheets and their before and after files.

Specifics

Test file locations

The test helper can run as a Python unit test, under control of eden/modules/unit_tests/suite.py.

The user can specify tuples of (xsl script, input file, golden file, option1 = value1, ...) where the paths are given relative to the eden root directory, and options might be used to tell the test helper such things as whether the order of repeated elements is significant. A list of tuples would constitute the full instructions to the test helper.

Where should the tuples be placed? They may as well go in a single file, in a directory under eden/modules/unit_tests.

In case people would like to add tests for other code under eden/static, we can follow the eden/static directory structure. That is, tests for stylesheets could be placed in eden/modules/unit_tests/static/formats so that, for instance, if people later want to write tests for tools, those could be placed in eden/modules/unit_tests/static/scripts/tools.

Verifying results

The transformed file will need to be compared against the golden file. Options for performing the comparison:

• For XML output, read both the transformed file and the golden file into etrees, and compare those.

• For S3XML output, read both using S3XML utilities and compare the S3XML trees.

• For other formats, compare the files as strings.

String comparison is fragile, as alterations in whitespace or change in ordering of elements (where that change is not significant to the meaning) will break the test. However, this is simple, and may be appropriate for getting started, even for formats that produce XML or S3XML. The test helper can run as a Python unit test, under control of eden/modules/unit_tests/suite.py. In case people would like to add tests for other code under eden/static, we can follow the eden/static directory structure. That is, tests for stylesheets could be placed in eden/modules/unit_tests/static/formats so that, for instance, if people later want to write tests for tools, those could be placed in eden/modules/unit_tests/static/scripts/tools.

• Allow the user to provide a comparison function.

This goes against the purpose of making testing easy, so should be avoided. The test helper can run as a Python unit test, under control of eden/modules/unit_tests/suite.py. In case people would like to add tests for other code under eden/static, we can follow the eden/static directory structure. That is, tests for stylesheets could be placed in eden/modules/unit_tests/static/formats so that, for instance, if people later want to write tests for tools, those could be placed in eden/modules/unit_tests/static/scripts/tools.