Hitachi Vantara Pentaho Community Wiki
Child pages
  • MapReduce Output
Skip to end of metadata
Go to start of metadata

(warning) PLEASE NOTE: This documentation applies to Pentaho 7.1 and earlier. For Pentaho 8.0 and later, see MapReduce Output on the Pentaho Enterprise Edition documentation site.

Description

This step defines the key/value pairs for Hadoop output. The output of this step will become the output to Hadoop, which changes depending on what the transformation is used for.


If this step is included in a transformation used as one of the following types:

  • Mapper and there is a combiner and/or reducer configured, the output will become the input pairs for the combiner and/or reducer. If there are no combiner or reducers configured the output is passed to the format configured for the job it was executed with. 
  • Combiner and there is a reducer configured, the output will become the input pairs for the reducer. If no reducer configured, the output is passed to the format configured for the job it was executed with.
  • Reducer, then the output is passed to the format configured for the job it was executed with.are not able to define the data type for the key or value here; it is defined earlier in your transformation. However, a reducer or combiner that takes this output as its input will have to know what the key and value data types are, so you may need to make note of them somehow.

Options

Option

Definition

Step name

The name of this step as it appears in the transformation workspace.

Key field

The Hadoop output field that represents the key in MapReduce terms.

Value field

The Hadoop output field that represents the value in MapReduce terms.

Metadata Injection Support (7.x and later)

All fields of this step support metadata injection. You can use this step with ETL Metadata Injection to pass metadata to your transformation at runtime.

  • No labels