Hitachi Vantara Pentaho Community Wiki
Child pages
  • Expose key-value pairs from a .properties file as variables
Skip to end of metadata
Go to start of metadata

Expose the contents of a .properties file as variables

Abstract: Managing job configuration typically involves an initial phase where you set a bunch of configuration settings as variables so these can be referenced by subsequent jobs and transformations. Use this job to load the contents of a .properties (or .ini) file abd set them as variables.

Authors: Roland Bouman

License: LGPL

Kettle versions: 3.2 and up (perhaps earlier versions too)

Attachments

  • set-variables.ktr - A transformation that loads the properties in a config.properties file in a directory specified by the variable CONFIG_DIR and sets a variable for each key/value pair in the properties file.

Background

Often you need to run a reusable job against multiple environments or in different conditions. For example, the same job needs to run in a development environment, a test environment and a production environment. You can conveniently manage the configurations for these different contexts in a .properties or .ini file, and have an initial transformation that reads the properties and sets corresponding kettle variables so any subsequent jobs and transformations can access the configuration through variable references.

This transformation can be run as such an initial transformation in the main job of a ETL process. The transformation assumes a CONFIG_DIR variables (typically set by the caller of the main job through a kettle parameter) that points to the location where a config.properties file can be found. The transformation will read the key/value pairs from the file and will create a variable that having the key as name and the value as its value.

See also: http://rpbouman.blogspot.com/2011/05/managing-kettle-job-configuration.html

  • No labels

1 Comment

  1. Related feature requests:

    http://jira.pentaho.com/browse/PDI-6184 "As an ETL developer, I like to set variables by Key/Value pairs from a data stream (e.g. a properties file or table as a source)"

    http://jira.pentaho.com/browse/PDI-618 "As an ETL Designer, I want the ability to load variables defined in a properties file"