Hitachi Vantara Pentaho Community Wiki
Skip to end of metadata
Go to start of metadata

Wingman Infrastructure

The wingman system is composed of a virtual machine hosting 3 main services, with the addition of GitHub.

  • A Jenkins service for scheduling
  • An OSGI application, pentaho-wingman, which handles the orchestration and analysis of the builds.
  • A docker container, which hosts the builds to run in a sandbox.


There is a basic Jenkins instance doing the orchestration and management. It has 3 jobs. 

  • wingman-trigger: This is the scheduled process. 
  • wingman: This is the main job which calls the Orchestrator service.
  • wingman-update: This is a management process. It fetches, compiles and updates the pentaho-wingman service (see below).


The trigger is a scheduled job. It runs each X minutes and asks GitHub for a list of the current pull requests for all projects in a list of organizations known to GitHub. 

It was designed as part of task ENGOPS-1586. Physically, it consists of a python script. The script can be seen here. It runs periodically on the Jenkins server and takes the following parameters.




A comma separated list of the names of the organizations to scan for a list of projects. Usually something like: "pentaho,webdetails".


The API token from GitHub.


A comma separated list of the names of the branches to scan. All pull requests for these branches will be considered.


A comma separated list of organization names to which a user must belong to in order for his pull request to be built.

A typical invocation of this script should look something like this:

python -o webdetails,pentaho -t 0123456789abcdef -b master,6.1,6.0 -ao webdetails,pentaho


This is the main job on Jenkins which communicates with the Wingman  service. It is a pretty simple job which gets invoked by the wingman-trigger job. It can also be launched manually. 

This job takes a few parameters.




The name of the organization which owns the pull request we want built.


The name of the repository which owns the pull request we want built.


The number of the pull request to build.

With these parameters, Jenkins simply invokes the REST endpoint of the wingman service by adding the parameters supplied. A typical invocation should look something like this:


JSON=`echo "{\"SourceRetriever\": { \"SourceControlType\": \"github\", \"Organization\": \"${WINGMAN_ORG}\", \"Repository\": \"${WINGMAN_REPO}\", \"PullRequest\": \"${WINGMAN_PR_NUMBER}\"}, \"StatusUpdater\": {\"WingmanUrl\": \"$BUILD_URL\"}}"`
curl -N -s -S -H "Content-Type: application/json"\
     -X POST -d "$JSON"\


This job updates the Wingman service. The update job performs a few tasks.

  • Build wingman from the latest code
  • Stop the wingman service
  • Replace the current build with the new one
  • Start the wingman service

This is done through the script present within the project.


This system is composed of an OSGI application. The code lives here. It contains numerous modules which fall in one of those categories.

  • Source control modules
  • Builders
  • Analyzers
  • A single Orchestrator module

To get the code built and tested, please refer to the project's README file.

Once deployed, the service is invoked through a URL.


This service takes a few parameters. To invoke it, do a POST operation on its endpoint and pass the following JSON message. Notice the presence of a few variables. Replace these tokens by their proper values.

  • No labels