The ETL Metadata Injection step is capable of injecting step metadata into a template transformation. So instead of statically entering ETL metadata in a step dialog, you pass it at run-time. It is possible to solve repetitive ETL workloads like loading of text files, data migration and so on.
The following steps support metadata injection (introduced version between brackets):
- Access Input (5.0)
- Concat Fields (5.1)
- CSV Input (4.1)
- Data Grid (5.1)
- Excel Input (4.1)
- Excel Output (5.1)
- Fixed file input (4.1)
- Get Data from XML (5.0)
- Group By (5.0)
- JSON output (5.2)
- Row Denormaliser (4.2)
- Row Normaliser (4.2)
- GZIP CSV Input (5.1)
- Pentaho Reporting Output (5.0)
- PostgreSQL Bulk Loader (5.1)
- Select Values (4.1)
- Split Field (5.0)
- Sort Rows (5.0)
- Table Output (5.1)
- Table Input (5.2)
- Text File Input (5.0)
- Text File Output (5.2)
- User Defined Java Expression (5.2)
- Avro Output (plugin 2.1.1 beta)
- Transformation template: in this section of the dialog you can specify the transformation to use as a template. When you have specified a transformation you can use the "Validate and Refresh" button (4.4). The "Edit" button will open the specified template in a new tab in Spoon.
- Source step to read from (optional): If you specify a step from the template here, then the output of the "ETL Metadata Injection" step will be the output from the source step.
- Optional target file (ktr after injection): For debugging or transformation generation you can save the resulting transformation filename, after metadata injection, to a file. If you want that specify a file name, for example "result.ktr".
- Don't execute resulting transformation: If you prefer to not execute the resulting transformation (after metadata injection) you can select that here.
- Field mapping: You can select any row in the metadata tree table with your mouse which will pop up a source step and field selection dialog.
Since version 5.1 this step is capable of streaming data from one transformation into another.
To pass data from your template transformation (after injection, during execution) to your current transformation, specify the "Template step to read from". You can also specify the expected output fields so that it's easier to design the steps which come after the ETL Metadata Injection step.
To pass data from a source step into the template transformation (again, after injection) you can specify the "Streaming source step" and the "Streaming target step" in the template transformation.