I have developed a BlackBox Testing framework / facility where you could add Kettle Transformations and test the results against a Golden file.
1. The Test directory structure must be as follows:
Main dir (Name the directory whatever you would like. For example: Kettle BlackBox Tests)
|- Input (Has to be named "Input")
|- Output (Has to be named "Output")
|- Temp (Has to be named "Temp")
|- Golden (Has to be named "Golden")
2. The Input, Output, Temp, Golden directories must be named strictly as Input, Output, Temp, and Golden respectively. The Input directory must consist of all the input files required to run the transformations. The Transformations directory must consist of all the Transformations for each test case. The Output directory is where the transformation's output must be directed. The Golden directory is where the the golden files for the test cases must exist.
3. The Transformation file names must start with Test_Case_#.ktr [eg: Test_Case_1.ktr, Test_Case_34.ktr]
4. The Output file name must have the following format: Test_Case_#_Out_Actual.[txt|csv] [eg: Test_Case_1_Out_Actual.csv, Test_Case_34_Out_Actual.txt]
5. The sub-directories of the Golden directory must be numeric. [for example: 2.5.1, 3.0.1, 3.0.2]
6. The Golden directory is searched for the file to be compared against starting from the highest version then backwards. For example: If Test_Case_5_Out_Actual.txt exists in both 3.0.2 and 2.5.1, the search feature looks for the file 3.0.2 first and then goes to the directory with lower version numbers. The search stops as soon as the file is found in 3.0.2.
7. The exceptions encountered while executing a particular test case or while comparing Output file to Golden are logged and we would continue with next test case. To verify the results of the execution of the transformations, examine the logs generated.
8. doTests() method executes all the Transformations in the Transformation directory and compares all the files in Golden directory.
**Currently there is no support for testing Jobs.
9. The program looks to execute specific test cases or ALL test cases. To execute specific test cases, add the test case number to the testIDs property in the testing-config.xml file. For example: the configuration below executes test case number 1, 2, 3 and 7.
To run ALL test cases remove any entry in the value node. For example:
10. You can specify the logging level ranging from 1 to 6 in the testing-config.xml. For example:
Where 1 provides minimal logging and 6 provides max logging information.