I have developed a BlackBox Testing framework / facility wherein you could add Kettle Transformations and test the results against a Golden file.
1. The Test directory structure should be as follows:
Main dir (Name it whatever you would like. eg: Kettle BlackBox Tests)
|- Input (Has to be named "Input")
|- Output (Has to be named "Output")
|- Temp (Has to be named "Temp")
|- Golden (Has to be named "Golden")
2. The Input, Output, Temp, Golden dirs should be named strictly as Input, Output, Temp, Golden respectively. The Input directory should consist of all the input files required to run the transformations. The Transformations directory should consist of all the Transformations for each test case. The Output directory is where the transformation's output should be directed. The Golden directory is where the the golden files for the test cases should exist.
3. The Transformation file names should start with Test_Case_#.ktr [eg: Test_Case_1.ktr, Test_Case_34.ktr]
4. The Output file name should have the following format: Test_Case_#_Out_Actual.[txt|csv] [eg: Test_Case_1_Out_Actual.csv, Test_Case_34_Out_Actual.txt]
5. The sub directories of the Golden dir should be numeric. [eg: 2.5.1, 3.0.1, 3.0.2]
6. The Golden directory would be searched for the file to be compared against starting from the highest version and backwards. For ex: If Test_Case_5_Out_Actual.txt exists in both 3.0.2 and 2.5.1, the search would look for the file 3.0.2 first and then go to directory with lower numbers. The search would stop as soon as the file is found in 3.0.2.
7. The exceptions encountered while executing a particular test case or while comparing Output file to Golden would be logged and we would continue with next test case. To verify the results of the execution of the transformations, please look at the logs generated.
8. doTests() method will execute all the Transformations in the transformation dir and compare all the files in Golden dir.
**Currently there is no support for testing Jobs.
9. The program looks to execute specific test cases or ALL test cases. To execute specific test cases, add the test case number to the testIDs property in testing-config.xml file. For ex: the configuration below would execute test case number 1, 2, 3 and 7.
If you want to run ALL test cases then remove any entry in the value node. For ex:
10. You can specify the logging level to be in the range of 1 to 6 in the testing-config.xml. For ex:
Where 1 provides minimal logging and 6 provides max logging info.