Hitachi Vantara Pentaho Community Wiki

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

Introduction

5.1 includes several large changes which bring new features to the platform today while setting the stage for larger things to come. Before we dive-in to the specific enhancements made, it's important to understand the broader context in which the changes are being and the motivations behind them.

Platform migration to OSGI

Introduction

For some years now it’s been our desire to transition away from our limited home-grown plugin system to an industry standard modular system based-on OSGI. In 5.0 we began laying the groundwork for this transition. 5.1 sees us forge a clear migration path and rolls-out the first stages of OSGI in the Platform.

Background

First Why is this migration necessary? The answer requires a little background on why these changes were necessary. the architecture of the Platform.

The Platform at one time was designed around a central service-locator backed by a singular ObjectFactory ( Spring in production ). Unfortunately this service locator didn’t support collections of objects, querying, or the ability to extend it without editing a bunch of core files. There was no clean way to drop something into the platform and have it integrate well with the system. Consequently the IPluginManager came to prominence. PluginManager IPluginManager and PlatformPlugins provided a good “vehicle” for getting new features into the system. Since then the Platform has steadily moved to a more plugin-centric model. This is not wholly bad, its one-step closer to a modular architecture.Unfortunately, our plugin system is not that sophisticatedhas been a double-edged sword. One one side the extensibility of the Platform without having the modify the core increased greatly. On the other, "core" code became a second-class citizen in it’s own house. To work around this many of the "core" features had to be registered through placeholder plugins such as the “default-plugin” and "admin-plugin".

We also increasingly find ourselves running into the limitations of the Platform Plugin system. Classes cannot be shared between plugins, limiting the extension of the platform and hurting cohesion between plugins. Since As Classes cannot be shared out of Plugins , no new concepts can be added to the system. We actually have multiple instances of plugins communicating between each other Calls between plugins are diffucult to perform, often relying using web services, content-generators and even reflection to work-around this limitation. Some things can only be supplied through plugins, making core code a second-class citizen in it’s own house. To work around this many of the "core" features are registered through the “default-plugin”. OSGI addresses all of these concerns, but how can be get there?

The transition involves leveling the field between legacy plugins, core code and OSGI Bundles. Once on an equal footing, Platform Plugins can be migrated to OSGI Bundles over a period of time. 5.1 accomplishes precisely this

Leveling the field

Early-on we identified that the dual APIs of PentahoSystem.getXXX() and IPluginManager.getXXX needed to be consolidated. The choice was made to enhance the PentahoSystem's backing IPentahoObjectFactory APIs to support collections of objects per type (Class/Interface) as well as adding a filtering mechanism (find IContentGenerator for type 'prpt'). The combination of these two would allow for a transition away from the IPluginManager API. 5.0 delivered these enhancements.

Extensible ObjectFactory for PentahoSystem
5.0 also replaced the singular Spring-based IPentahoObjectFactory with an Aggregating ObjectFactory implementation. This new AggregateObjectFactory supports the registration of any number of IPentahoObjectFactory instances. Queries to the Aggregate call down to all configured "sub"-factories for matches. The results are filtered by any query conditions, ordered by priority and returned back. The AggregateObjectFactory can be thought of as a single Logical view over many Physicals underneath. The main Spring ApplicationContent (pentaho-spring-beans.xml) is registered as well as any plugin.spring.xml files publishing beans (link to 5.0 document). 5.1 sees two new sub-factories registered by default, RuntimeObjectFactory and OSGIObjectFactory

ObjectReferences

Another change made in the 5.0 release was the introduction of IPentahoObjectReference (source). These are wrappers, or more appropriately descriptors, for an Object available in the ObjectFactory. The main advantage these bring is the ability to provide attribute metadata which powers the ordering and filtering capabilities. It also allows for the lazy creation of the actual objects described by the reference. The same lazy mechanism also makes it possible for an ObjectReference to be a factory.

OSGI ObjectFactory

The OSGIObjectFactory delegates down to the OSGI Service Registry to find matching ServiceReferences. These ServiceReferences map almost directly to IPentahoObjectReferences. If your bundle registers a
Service, it will be found through PentahoSystem.getXXX calls by the same Classes/Interfaces and properties as it was registered with in OSGI.

Code Block
titleIn OSGI Bundle

BundleContext cxt = ...
IContentInfo infoInstance = new ContentInfo();
Hashtable props = new Hashtable( { put("prop1", "value"); } );
cxt.registerService( IContentInfo.class, infoInstance, props );
Code Block
titleIn Pentaho Code

IContentInfo fromOsgi = PentahoSystem.get( IContentInfo.class, null, Collections.singletonMap( "prop1", "value" ) );
Runtime ObjectFactory

The RuntimeObjectFactory is an implementation of the IPentahoRegistrableObjectFactory interface (source). This interface extends the IPentahoObjectFactory with new methods allowing the registration of Objects and IPentahoObjectReferences at runtime. Some built-in IPentahoObjectReference implementations are available for common factory scenarios ( SingletonPentahoObjectReference, SessionBoundPentahoObjectReference, SingletonPentahoObjectReference). Each of these has a static Builder available to make construction easier.

Code Block
titleObject Registration


import static org.pentaho.platform.api.engine.IPentahoRegistrableObjectFactory.Types;
...
// "Testing" will be registered under all Classes in the inheritance chain (String, Object).
PentahoSystem.registerObject( "Testing", Types.CLASSES );

// "Testing" will be registered under all implementing Interfaces ( CharSequence an Serializable )
PentahoSystem.registerObject( "Testing", Types.INTERFACES );

// "Testing" will be registered under all implemented Interfaces and all Classes in the inheritance chain (String, Object, CharSequence, Serializable).
PentahoSystem.registerObject( "Testing", Types.ALL );

// "Testing" will be registered under just the String Class.
PentahoSystem.registerObject( "Testing", String.class);

// "Testing" will be registered under the String Class and CharSequence Interface
PentahoSystem.registerObject( "Testing", String.class, CharSequence.class);

IPentahoObjectReference registration is much the same as Object registration. Below are some examples of some of the built-in ObjectReference types available out-of-the-box.

Code Block
titleSingletonPentahoObjectReference

import static org.pentaho.platform.api.engine.IPentahoRegistrableObjectFactory.Types;
...

// Singleton will always return the same Object it was constructed with
SingletonPentahoObjectReference reference = new SingletonPentahoObjectReference.Builder<String>( String.class ).object( "Hello" ).attributes(
    Collections.<String, Object>singletonMap( "attr1", "value1" ) ).build()
PentahoSystem.registerReference( reference, Types.CLASSES );

Code Block
titlePrototypePentahoObjectReference

import static org.pentaho.platform.api.engine.IPentahoRegistrableObjectFactory.Types;
...

// Prototype requires an IObjectCreator which is called for each call to reference.getObject()
PrototypePentahoObjectReference protoReference = new PrototypePentahoObjectReference.Builder<UUID>( UUID.class ).creator(
  new IObjectCreator<UUID>() {
    @Override public UUID create( IPentahoSession session ) {
      return UUID.randomUUID();
    }
  }).build();
PentahoSystem.registerReference( protoReference, UUID.class );

Code Block
titlePrototypePentahoObjectReference

import static org.pentaho.platform.api.engine.IPentahoRegistrableObjectFactory.Types;
...

// Session-Based will call the IObjectCreator for each unique IPentahoSession it's called with. Subsequent calls with the same session will return the same Object
SessionBoundPentahoObjectReference sessionReference = new SessionBoundPentahoObjectReference.Builder<UUID>( UUID.class ).creator(
  new IObjectCreator<UUID>() {
    @Override public UUID create( IPentahoSession session ) {
      return UUID.randomUUID();
    }
  }).build();
PentahoSystem.registerReference( sessionReference, UUID.class );

IPentahoObjectRegistration

Both PentahoSystem.registerObject and PentahoSystem.registerReference return a IPentahoObjectRegistration handle. This can be used later to de-register the Object or Reference

Code Block
titleRegistration Handles

IPentahoObjectRegistration handle = PentahoSystem.registerObject( "Testing", Types.CLASSES );

// later when you want to remove this registration
handle.remove();
EEPluginManager

The new EEPluginManager uses these methods to store plugin objects in the PentahoSystem instead of keeping them hidden within, as was done in the DefaultPluginManager. The result is that you no longer have to make calls to the IPluginManager and can instead query the PentahoSystem:

Code Block

// instead of this
IPluginManager pluginManager = PentahoSystem.get( IPluginManager.class );
List<XulOverlay> overlays = pluginManager.getOverlays();

// you can call this
List<XulOverlay> overlays = PentahoSystem.getAll( XulOverlay.class )

The significance of this change is that this second call will return XulOverlays not just from the PlatformPlugins, but also from OSGI bundles, configured Spring ApplicationContexts or anyone registered with the AggregateObjectFactory! This is the key to transitioning to OSGI!

Why OSGI?

OSGI is a industry-standard Modular Software framework. It's design supersedes that of a plugin architecture. Core code and extensions are built using the same modular design, Bundles, which most closely resemble the concept of Plugins. Everything is on the same footing in OSGI. Versionsed Classes can be freely shared between bundles allowing for the controlled extension of the platform. The OSGI environment has built-in resilience to change at runtime for bundles installing and uninstalling.

In truth the Migration to OSGI is not exchanging one plugin system to another. Rather it is moving to an architecture which does not require a Plugin system at all.

5.0 and 5.1 Enhancements preparing for OSGI

You can read about the steps taken to in 5.0 and 5.1 which set the stage for this migration here:
Platform enhancements in 5.0 and 5.1

6.0 and Beyond -the next steps

The goal for 5.2 is to be able to migrate PlatformPlugins to Bundles. Several Jira cases have been created to encapsulate the work to be done in 6.0

BISERVER-11539 - Support the ability to provide Platform Plugins from OSGI
Our home-grown plugin system is limiting our ability to deliver compelling product

BISERVER-11540 - Deprecate the IPluginManager as a Service Locator
Have the PluginManager register objects with PentahoSystem instead so we have a single point for service location

BISERVER-11541 - IPluginResourceLoader needs to be modified to work with resources provided by OSGI Bundles
The ResourceLoader api is separate from PlatformPlugins and The IPluginManager but one that needs to be made to work with OSGI Bundles

MARKET-151 - The Marketplace needs to be updated to work with OSGI Bundle plugins

BISERVER-11542 - Sample OSGI Bundles need to be written demonstrating PlatformPlugin-like behavior

BISERVER-11543 - A Type-Tracking system needs to be added to the Pentaho Platform API
As the new system allows PlatformPlugins and OSGI Bundles to be de/registered at runtime, we need a way to notify other areas of the system of the events

BISERVER-11544 - PentahoSystem's AggregateObjectFactory needs to be updated to cache query results

BISERVER-11545 - The Pentaho Spring Extensions need to be updated for Spring 3.2.5 so they can be used within OSGI Bundle Blueprint files

7.0 Goals

Once we have the ability for plugins migrate the OSGI, the next step is to move the "core" code into OSGI as well. This involves moving the OSGI container from an isolated embedded space to become the top-level container of the system. This migration is easier than it sounds. The "core" code left after 5.2 is all contained in the "pentaho" WAR. OSGI supports the deployment of WARs as OSGI bundles. The entire pentaho WAR will become one large WAB (Web Application Bundle). From there the work of separating out parts to their own distinct bundles can begin.

Also in 6.0 is the switch to the OSGI Service Registry away from PentahoSystem. All IPentahoObjectFactories in the 6.0 timeframe will be updated to register with the OSGI ServiceRegistry. Once done the PentahoSystem.getXXX() calls can be migrated away from our code to simply call into OSGI. Once this is done we can start the process of deprecating the Pentaho-Specific APIs and transition to plain OSGI.

7.0 Architecture:

Image Added

Continue on to Platform enhancements in 5.0 and 5.1