Hitachi Vantara Pentaho Community Wiki

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Pentaho's Big Data story revolves around Pentaho Data Integration AKA Kettle. Kettle is a powerful Extraction, Transformation and Loading (ETL) engine that uses a metadata-driven approach. The kettle engine provides data services for and is embedded in many of the applications within the Pentaho BI suite. Kettle comes with a graphical, drag and drop design environment for designing and running Kettle Jobs and Transformations.

Wiki MarkupA quick 2 min video of PDI in action{red} *Kettle Transformations* !Simple Transform.png|align=right, vspace=4! A Kettle transformation consists of one or more _steps_ that perform core ETL work like reading data in the form of rows from a file or database, filtering rows, calculating new columns and sending the new data stream somewhere else. All steps in a transform execute simultaneously (usually in separate threads) and data is passed from step to step in parallel. The data is operated on in a continuous stream without having to be fully read into memory or staged. {color:red}(IN WORK action

Kettle Transformations
Image Added
A Kettle transformation consists of one or more steps that perform core ETL work like reading data in the form of rows from a file or database, filtering rows, calculating new columns and sending the new data stream somewhere else. All steps in a transform execute simultaneously (usually in separate threads) and data is passed from step to step in parallel. The data is operated on in a continuous stream without having to be fully read into memory or staged. The image to the right demonstrated a very simple kettle transformation - Read from a data source, do some transformation, in this case a filter and then write the data stream to another data source.

(IN WORK DM)

Note
titleThis is a closed wiki space

The only people with access are Pentaho Employees and Dave Reinke (Chris will need to sign up for the wiki and send me his user id)

This is a first shot at getting an open source collaboration space for Big Data. It will eventually be open but is currently a work in progress and a place to put the use cases, demo's etc. I completely pulled the structure and initial content from my arse and am not in love with any of it. It is a round lump of clay, waiting to be molded by the brilliant minds of the Big Ass Data Team.