Hitachi Vantara Pentaho Community Wiki
Child pages
  • Pentaho Software Architecture
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 15 Next »

Introduction

The purpose of this document is to provide a detailed view of the overall software components that when combined make up the entire Pentaho open source software suite as it exists today.

At a high level, the software components can be divided into a variety of forms.  In the following detailed list, the general organization includes third party libraries and components that Pentaho has needed to fork and maintain, common libraries and projects that are used in general ways, pillars that are core business analytics or data integration elements, tools that allow access to pillars, and plugins across the pillars that provide additional functionality.  These same components can be looked at from a architectural purpose point of view, including four general areas including information delivery, data management / integration, analytics / reporting, and platform services.  For each project below we categorize in both manners to give a multi-faceted view of the overall architecture of Pentaho.

Cross Cutting Architectures and Use Cases

This section discusses high level cross cutting software architectures and use cases.

Configuration Management

At this time, Pentaho utilizes a combination of SVN and GIT for managing the source.  Here are some related articles:

http://wiki.pentaho.com/display/PEOpen/Advanced+Git+Topics

Metadata Definitions

As we continue to build a community of projects, it's important that they share terminology and common metadata.  Here's the beginnings of capturing shared metadata to be used across all Pentaho projects:

http://wiki.pentaho.com/display/COM/Standard+MetaStore+Element+types

Detailed Software Listing

This detailed software listing is organized in the general order in which software components are dependent on one another, although it should not be used as the official build order of Pentaho.

Third Party Maintained Forks
Common Components
Pillars
Tools
Plugins

Third Party Maintained Forks

It is Pentaho's intention to avoid having to fork and maintain third party open source software, but on a few occasions it has been necessary.  The following list is of the current third party maintained forks that Pentaho includes in our product.

kettle-vfs

Kettle VFS is a maintained fork of Apache Commons VFS

Source Path: svn://source.pentaho.org/svnkettleroot/kettle-vfs

Architectural Owner: Matt Casters

Architectural Area: Data Management / Integration

hive

Due to the dynamic nature of Hadoop, Pentaho currently maintains our own Hive JDBC Driver implementation

Source Path: https://github.com/pentaho/hive

Architectural Owner: Will Gorman

Architectural Area: Data Management / Integration

pentaho-ofc4j

Pentaho ChartBeans Flash components, which are still used by Pentaho Dashboards and Action Sequences, are based on Open Flash Chart.  OFC4J is a Java to JSON converter that is used to generate the correct metadata for the charts on the server that is no longer maintained by the creator of the project.

Source Path: svn://source.pentaho.org/svnroot/pentaho-commons/pentaho-ofc4j

Architectural Owner: Will Gorman

Architectural Area: Information Delivery

Common Components

subfloor

Subfloor is Pentaho's common build system, based on ant and used by all projects for compilation, assembly, unit testing and code coverage.

Source Path: https://code.google.com/p/subfloor/  (Note that this location is out of date and should be transitioned to GitHub)

Architectural Owner: Will Gorman

Architectural Area: Engineering Operations

pentaho-commons-database

This commons project is a GWT thin client of the shared database dialog.  The submodule pentaho-database-model was an attempt at a thin Kettle DatabaseMeta implementation, which includes a dialect and JDBC Metadata architecture.

Source Path: https://github.com/pentaho/pentaho-commons-database

Architectural Owner: Will Gorman

Architectural Area: Data Management / Integration

pentaho-connections

This commons project provides an API for interacting with platform connections, usually within the context of xactions.  Pentaho Metadata also uses this API for providing access to metadata in result sets.

Source Path: svn://source.pentaho.org/svnroot/pentaho-commons/pentaho-connections

Architectural Owner: Will Gorman

Architectural Area: Platform Services

pentaho-hdfs-vfs

This commons project provides an apache-vfs implementation of Hadoop HDFS.  This is used by Kettle for accessing HDFS in a number of contexts.

Source Path: svn://source.pentaho.org/svnroot/pentaho-commons/pentaho-hdfs-vfs

Architectural Owner: Will Gorman

Architectural Area: Data Management / Integration

pentaho-vfs

This is a single xml file project that allows the loading of the platform's vfs driver, that maps to the scheme "solution:"

Source Path: svn://source.pentaho.org/svnroot/pentaho-commons/pentaho-vfs

Architectural Owner: Will Gorman

Architectural Area: Platform Services

pentaho-versionchecker

This project hosts the APIs and basic logic for checking the version information of a currently running platform.

Source Path: svn://source.pentaho.org/svnroot/pentaho-commons/pentaho-versionchecker

Architectural Owner: Will Gorman

Architectural Area: Platform Services

pentaho-palo-core

This project is an abstraction layer that insulates the uses of Palo within the product so there is no compile time dependency, due to Palo's GPL licensing.

Source Path: svn://source.pentaho.org/svnkettleroot/plugins/PaloCore

Architectural Owner: Matt Casters

Architectural Area: Data Management / Integration

pentaho-registry

The Pentaho Registry project was developed in the context of Instaview to provide an API for a system wide metadata index for purposes of understanding relationships and lineage.

Source Path: https://github.com/pentaho/pentaho-registry

Architectural Owner: Will Gorman

Architectural Area: Platform Services

metastore

The Pentaho Metastore is an API that defines a simple way for interacting with a global metadata repository, and will be used as the main API for managing and storing global metadata definitions, such as connections, etc.

Source Path: https://github.com/pentaho/metastore

Architectural Owner: Matt Casters

Architectural Area: Platform Services

pentaho-cwm (deprecated)

This is a legacy project that defines the Pentaho to CWM object model API mapping, the code in the project is mostly generated, and is deprecated with the new Pentaho Metadata thin API.  This module can be completely removed once Pentaho Metadata Editor is replaced by a new editor.

Source Path: svn://source.pentaho.org/svnroot/pentaho-commons/pentaho-cwm

Architectural Owner: Will Gorman

Architectural Area: Analytics / Reporting (Pentaho Metadata)

pentaho-metadata

The Pentaho Metadata project defines a reporting metadata architecture and implementations.  The core components of Pentaho Metadata API are the Business Query (MQL), Logical Model, and Physical Models (CSV and SQL).  Long term plans are to merge these concepts into Pentaho Mondrian and the Metastore, but this will be an evolutionary approach and will take time before Pentaho Metadata is deprecated.

Source Path: https://github.com/pentaho/pentaho-metadata

Architectural Owner: Will Gorman

Architectural Area: Analytics / Reporting

pentaho-commons-gwt-modules

The gwt-modules project consists of the gwt-widgets and gwt-widgets-samples submodules.  This is a common area to keep reused gwt capabilities for our downstream GWT-based projects, including at this time pentaho user console, data source wizard, and others.

Source Path: https://github.com/pentaho/pentaho-commons-gwt-modules

Architectural Owner: Nick Baker

Architectural Area: Information Delivery

pentaho-xul

This is a general UI framework developed by Pentaho to address our cross platform UI requirements, it includes an implementation for Swing, GWT and SWT.  One of the key requirements to this framework is pluggability supported via overlays.  Major projects using this framework include the Aggregation Designer, the Modeler, PUC, Report Designer, Spoon, MQL Editor, and others.

Source Path: https://github.com/pentaho/pentaho-commons-xul

Architectural Owner: Nick Baker

Architectural Area: Information Delivery

modeler

This project provides an easy to use Pentaho Modeler, which is able to generate both Pentaho Metadata and Mondrian file formats.  It was originally built within the context of Agile BI, but is also currently used in the Data Source Wizard.  The plan is to eventually replace Mondrian Schema Workbench and Pentaho Metadata Editor with this UI.

Source Path: https://github.com/pentaho/modeler

Architectural Owner: Nick Baker

Architectural Area: Analytics / Reporting

pentaho-mql-editor

This project provides a simple UI for business users to define Pentaho Metadata Queries.  It is based on XUL so it can be used in Swing, SWT and GWT.

*Source Path: *svn://source.pentaho.org/svnroot/pentaho-commons/pentaho-mql-editor

Architectural Owner: Nick Baker

Architectural Area: Information Delivery

pentaho-actionsequence-dom

This project provides a document object model for core action sequence file parsing.  It also manages validation of action sequence files (.xaction).  In the long term Pentaho is replacing Action Sequence functionality with Kettle Transformations and Job capabilities.

Source Path: svn://source.pentaho.org/svnroot/pentaho-commons/pentaho-actionsequence-dom

Architectural Owner: Will Gorman

Architectural Area: Platform Services

chartbeans

TODO

publisher

TODO

Pillars

Tools

Plugins

  • No labels