Hitachi Vantara Pentaho Community Wiki
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

I have developed a BlackBox Testing framework / facility where you could add Kettle Transformations and test the results against a Golden file.

1. The Test directory structure must be as follows:
         Main dir (Name the directory whatever you would like. For example: Kettle BlackBox Tests)
             |- Input (Has to be named "Input")
             |- Output (Has to be named "Output")
             |- Temp (Has to be named "Temp")
             |- Golden (Has to be named "Golden")
                 |- 2.5.1
                 |- 3.0.1
                 |- 3.0.2

2. The Input, Output, Temp, Golden directories must be named strictly as Input, Output, Temp, and Golden respectively. The Input directory must consist of all the input files required to run the transformations. The Transformations directory must consist of all the Transformations for each test case. The Output directory is where the transformation's output must be directed. The Golden directory is where the the golden files for the test cases must exist.

3. The Transformation file names must start with Test_Case_#.ktr [eg: Test_Case_1.ktr, Test_Case_34.ktr]

4. The Output file name must have the following format: Test_Case_#_Out_Actual.[txt|csv] [eg: Test_Case_1_Out_Actual.csv, Test_Case_34_Out_Actual.txt]
5. The sub-directories of the Golden directory must be numeric. [for example: 2.5.1, 3.0.1, 3.0.2]

6. The Golden directory is searched for the file to be compared against starting from the highest version then backwards. For example: If Test_Case_5_Out_Actual.txt exists in both 3.0.2 and 2.5.1, the search feature looks for the file 3.0.2 first and then goes to the directory with lower version numbers. The search stops as soon as the file is found in 3.0.2.
7. The exceptions encountered while executing a particular test case or while comparing Output file to Golden are logged and we would continue with next test case. To verify the results of the execution of the transformations, examine the logs generated.
8. doTests() method executes all the Transformations in the Transformation directory and compares all the files in Golden directory.

**Currently there is no support for testing Jobs.

9. The program looks to execute specific test cases or ALL test cases. To execute specific test cases, add the test case number to the testIDs property in the testing-config.xml file. For example: the configuration below executes test case number 1, 2, 3 and 7.


To run ALL test cases remove any entry in the value node. For example:


10. You can specify the logging level ranging from 1 to 6 in the testing-config.xml. For example:


Where 1 provides minimal logging and 6 provides max logging information.   

11. testing-config.xml holds the properties to be run for the Black Box Testing.

Sample testing-config.xml file:

    <test id="pdi3"
            <value>C:\Kettle Blackbox Tests\Transformations</value>
            <value>C:\Kettle Blackbox Tests\Golden</value>
            <value>C:\Kettle Blackbox Tests\Output</value>
        <!-- logLevel needs to be set to a value ranging from 0 to 6:
         with 0 being minimal logging and 6 being the most detailed logging -->

12. Sample Golden directory with the Golden files in 3.0.3 directory :

13. Sample Output directory (after executing the test cases 1,2,3,7):

14. Sample Log (after executing the test cases at level #1):

 INFO  27-11 13:47:00,062 (  -Running: C:\Kettle Blackbox Tests\Transformations\Test_Case_1.ktr
INFO  27-11 13:47:02,687 (  -Using "C:\DOCUME~1\User\LOCALS~1\Temp\vfs_cache" as temporary files store.
INFO  27-11 13:47:04,671 (  -Comparing files C:\Kettle Blackbox Tests\Output\Test_Case_1_Out_Actual.csv , C:\Kettle Blackbox Tests\Golden\3.0.3\Test_Case_1_Out_Actual.csv
INFO  27-11 13:47:04,687 (  -Comparison succeeded.
INFO  27-11 13:47:04,687 (  -Running: C:\Kettle Blackbox Tests\Transformations\Test_Case_2.ktr
INFO  27-11 13:47:06,125 (  -Comparing files C:\Kettle Blackbox Tests\Output\Test_Case_2_Out_Actual.txt , C:\Kettle Blackbox Tests\Golden\3.0.3\Test_Case_2_Out_Actual.txt
INFO  27-11 13:47:06,140 (  -Comparison succeeded.
INFO  27-11 13:47:06,140 (  -Running: C:\Kettle Blackbox Tests\Transformations\Test_Case_3.ktr
INFO  27-11 13:47:07,250 (  -Comparing files C:\Kettle Blackbox Tests\Output\Test_Case_3_Out_Actual.txt , C:\Kettle Blackbox Tests\Golden\3.0.3\Test_Case_3_Out_Actual.txt
INFO  27-11 13:47:07,250 (  -Comparison succeeded.
INFO  27-11 13:47:07,250 (  -Running: C:\Kettle Blackbox Tests\Transformations\Test_Case_7.ktr
INFO  27-11 13:47:08,484 (  -Comparing files C:\Kettle Blackbox Tests\Output\Test_Case_7_Out_Actual.txt , C:\Kettle Blackbox Tests\Golden\3.0.3\Test_Case_7_Out_Actual.txt
INFO  27-11 13:47:08,484 (  -Comparison succeeded.

  • No labels