Hitachi Vantara Pentaho Community Wiki
Child pages
  • Security Considerations and Encryption with Kettle
Skip to end of metadata
Go to start of metadata

Security Considerations and Encryption with Kettle

Abstract: Kettle is used more and more in enterprises where the standard obfuscation of credentials is not sufficient enough. There are requirements to use strong encryption methods and even to store internal data encrypted (covered in PDI-6168 and PDI-6170). The above use cases inspired me to create some simple transformations to test and play around with encryption.

Note: Kettle already has job entries to encrypt and decrypt files with PGP.

Authors: Jens Bleuel

License: LGPL

Kettle versions: 4.1 and later

Update since Kettle 4.2: There are two steps in the experimental section: Secret key generator, Symmetric Cryptography that cover this use case.


Attention: When you want to use the given examples in production please be aware that they are made as a blue print and a proof of concept and need optimization.


Transformation cryptographyCreateSecretKey is used to create a secret key.
Use the transformations cryptographyEncrypt and cryptographyDecrypt for testing.

The named parameters and default values are:

  • cryptographyKeySize = 128
  • cryptographyMethod = AES

The transformations were tested with AES and 128 bit: Advanced Encryption Standard (AES) is a symmetric-key encryption standard, see also

To use AES with 192- and 256-bit key sizes, you need unlimited strength cryptography.
Due to import-control restrictions imposed by some countries, the jurisdiction policy files shipped
with the Java 2 SDK, v 1.4 only permit strong cryptography to be used. An unlimited strength version of these files
(that is, with no restrictions on cryptographic strength) is available for download, however.

Instead of storing the decrypted data to a file there are a lof of other options, e.g.:

  • use the decrypted data as credentials in subsequent steps or transformations
  • put the decrypted data into variables visible in a limited scope (e.g. parent job) and use them as credentials for databases, repository etc. (see PDI-6168)
  • and many more options

We may consider:

  • Symmetric-key algorithm vs. asymmetric key algorithms (public-key cryptography)
  • Diffie-Hellman key exchange is a specific method of exchanging keys.
  • Ensure integrity e.g. by hash-codes
  • Key file handling could be optimized in different ways.
  • Please keep in mind that unencrypted data is in RAM (see PDI-6170 for a circumvention to prevent heap dumps)
  • Beneath the binary or indexed storage type, an encrypted storage type may be possible in Kettle core.

For screen shots, some further background information and a test run, please see

  • No labels


  1. user-2ca73

    I used this sample to encrypt a field that is stored in MySQL, but when I tried to use MySQL's AES_DECRYPT() to get the data back out, it just returned NULLs. It turns out that MySQL uses a 16 byte key for it's AES encryption, regardless of what you pass in. If there are more than 16 bytes in your string, it will just OR in those bytes to the existing key bytes. Here's the blog post that explains this process and helped me resolve the issue:

    Hope this helps!

  2. user-1220a

    Hello, when I have more then 1 file in the directory that fits the regular expression to decrypt, it combined them both into one output file, and also leaves a bunch of junk lines at the bottom. Is there a way to decrypt each file into it's own output file? Thanks!