Wingman Infrastructure

Skip to end of metadata
Go to start of metadata

Wingman Infrastructure

The wingman system is composed of a virtual machine hosting 3 main services, with the addition of GitHub.

  • A Jenkins service for scheduling
  • An OSGI application, pentaho-wingman, which handles the orchestration and analysis of the builds.
  • A docker container, which hosts the builds to run in a sandbox.

Jenkins

There is a basic Jenkins instance doing the orchestration and management. It has 3 jobs. 

  • wingman-trigger: This is the scheduled process. 
  • wingman: This is the main job which calls the Orchestrator service.
  • wingman-update: This is a management process. It fetches, compiles and updates the pentaho-wingman service (see below).

wingman-trigger

The trigger is a scheduled job. It runs each X minutes and asks GitHub for a list of the current pull requests for all projects in a list of organizations known to GitHub. 

It was designed as part of task ENGOPS-1586. Physically, it consists of a python script. The script can be seen here. It runs periodically on the Jenkins server and takes the following parameters.

Parameter Description
-o A comma separated list of the names of the organizations to scan for a list of projects. Usually something like: "pentaho,webdetails".
-t The API token from GitHub.
-b A comma separated list of the names of the branches to scan. All pull requests for these branches will be considered.
-ao A comma separated list of organization names to which a user must belong to in order for his pull request to be built.

A typical invocation of this script should look something like this:

python PullRequestRecon.py -o webdetails,pentaho -t 5334afa5de50ea3e4357d73ce3544ac3e8f5987 -b master,6.1,6.0 -ao webdetails,pentaho

wingman

This is the main job on Jenkins which communicates with the Wingman  service. It is a pretty simple job which gets invoked by the wingman-trigger job. It can also be launched manually. 

This job takes a few parameters.

Parameter Description
WINGMAN_ORG The name of the organization which owns the pull request we want built.
WINGMAN_REPO The name of the repository which owns the pull request we want built.
WINGMAN_PR_NUMBER The number of the pull request to build.

With these parameters, Jenkins simply invokes the REST endpoint of the wingman service by adding the parameters supplied. A typical invocation should look something like this:

#!/bin/bash

JSON=`echo "{\"SourceRetriever\": { \"SourceControlType\": \"github\", \"Organization\": \"${WINGMAN_ORG}\", \"Repository\": \"${WINGMAN_REPO}\", \"PullRequest\": \"${WINGMAN_PR_NUMBER}\"}, \"StatusUpdater\": {\"WingmanUrl\": \"$BUILD_URL\"}}"`
curl -N -s -S -H "Content-Type: application/json"\
     -X POST -d "$JSON"\
     http://localhost:8181/cxf/orchestrator/orchestrate

wingman-updater

This job updates the Wingman service. The update job performs a few tasks.

  • Build wingman from the latest code
  • Stop the wingman service
  • Replace the current build with the new one
  • Start the wingman service

This is done through the deploy.sh script present within the project.

pentaho-wingman

This system is composed of an OSGI application. The code lives here. It contains numerous modules which fall in one of those categories.

  • Source control modules
  • Builders
  • Analyzers
  • A single Orchestrator module

To get the code built and tested, please refer to the project's README file.

Once deployed, the service is invoked through a URL.

http://localhost:8181/cxf/orchestrator/orchestrate

This service takes a few parameters. To invoke it, do a POST operation on its endpoint and pass the following JSON message. Notice the presence of a few variables. Replace these tokens by their proper values.

{  
   "SourceRetriever":{  
      "SourceControlType":"github",
      "Organization":"${WINGMAN_ORG}",
      "Repository":"${WINGMAN_REPO}",
      "PullRequest":"${WINGMAN_PR_NUMBER}"
   },
   "StatusUpdater":{  
      "WingmanUrl":"http://wingman.pentaho.com:8080/job/wingman/106/"
   }
}
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.