Access Keys:
Skip to content (Access Key - 0)

Introduction

In version 3.0.3 and 3.1.0-M1, we added the ability to launch job entries in parallel.  This makes it easier to fire off jobs and transformations in parallel on the same machine or even remotely.

Enabling parallel execution

You can ask a job entries to launch the next job entries in parallel.  As such, you can click right on a job entry and select the "Launch next entries in parallel" option.

 
Once selected, the arrows to the next job entries will be shown in dashed lines and a check-box will appear next to the entry in the pop-up menu.

Functionality

All the job entries following the one where you enabled the "launch in parallel" option are going to be executed in parallel.  Since the job entries following that one are dependent on it, these will also be executed in parallel. (because of the backtracking algorithm that the job execution engine uses).

Limitation

The execution model cited above makes it harder to to execute a certain number of job entries in parallel and then simply continue with something else in sequence.
To do this, we suggest you wrap up the parallel work in a separate Job.

Warning

The following warning will be shown when you enable the feature:



  1. Jun 09, 2008

    Mark Cahill says:

    I get a page full of errors when I change my job to run in parallel like this. W...

    I get a page full of errors when I change my job to run in parallel like this. What could be causing this?

    2008/06/09 13:42:14 - kettle - ERROR (version 3.0.3, build 569 from 2008/04/25 12:30:04) : ERROR executing query: org.pentaho.di.core.exception.KettleDatabaseException: 2008/06/09 13:42:14 - kettle - ERROR (version 3.0.3, build 569 from 2008/04/25 12:30:04) : offending row : [value String], [key Integer]2008/06/09 13:42:14 - kettle - ERROR (version 3.0.3, build 569 from 2008/04/25 12:30:04) : 2008/06/09 13:42:14 - kettle - ERROR (version 3.0.3, build 569 from 2008/04/25 12:30:04) : Error setting value #1 [value String] on prepared statement (String)2008/06/09 13:42:14 - kettle - ERROR (version 3.0.3, build 569 from 2008/04/25 12:30:04) : java.sql.SQLException: PreparedStatement has been closed. No further operations allowed.2008/06/09 13:42:14 - kettle - ERROR (version 3.0.3, build 569 from 2008/04/25 12:30:04) : PreparedStatement has been closed. No further operations allowed.2008/06/09 13:42:14 - kettle - ERROR (version 3.0.3, build 569 from 2008/04/25 12:30:04) : ERROR in part: P Set values

  2. Oct 22, 2008

    Jens Bleuel says:

    Mark, if the problem still persists in 3.1 GA, please file a bug report at JIRA...

    Mark,

    if the problem still persists in 3.1 GA, please file a bug report at JIRA or discuss on the forum (better then commenting here).

    Thanks,

    Jens

  3. Nov 13, 2008

    Matt Casters says:

    Mark, this error is caused by a concurrency problem in the repository handler.&n...

    Mark, this error is caused by a concurrency problem in the repository handler.  As Jens suspected it's fixed in the mean time.

  4. Jan 02, 2009

    Jens Bleuel says:

    Just for the records: It is fixed in 3.1.1 by http://jira.pentaho.com/browse/PDI...

    Just for the records: It is fixed in 3.1.1 by http://jira.pentaho.com/browse/PDI-1756

  5. Mar 06, 2009

    COLAS Benoit says:

    Where can we find 3.1.1 ? Only 3.1.0-826 is available on sourceforge. In t...

    Where can we find 3.1.1 ? Only 3.1.0-826 is available on sourceforge.

    In this version, 3.1.0-826,  the problem still exists:

    INFO  06-03 12:16:51,537 - End_Load_GTI - Starting entry [alim_evt_archi_ton_quot]
    INFO  06-03 12:16:51,538 - End_Load_GTI - Launched job entry [alim_evt_archi_ton_quot] in parallel.
    INFO  06-03 12:16:51,539 - End_Load_GTI - Starting entry [alim_cgo_quot]
    INFO  06-03 12:16:51,541 - End_Load_GTI - Launched job entry [alim_cgo_quot] in parallel.
    ERROR 06-03 12:16:51,609 - alim_cgo_quot - Error running job entry 'job' : org.pentaho.di.core.exception.KettleException:
    An error occurred reading job [alim_cgo_quot] from the repository

  6. Dec 15, 2009

    Jens Bleuel says:

    > Where can we find 3.1.1 ? Fix releases are available for customers and als...

    > Where can we find 3.1.1 ?

    Fix releases are available for customers and also covered in the Kettle Developer Support, please see http://www.pentaho.com/products/buy_bi_suite.php

  7. May 08, 2012

    Andrew Jones says:

    When 'Launch next entries in parallel' is checked just prior to a subjob that ex...

    When 'Launch next entries in parallel' is checked just prior to a subjob that executes once for each input row, does the subjob execute in parallel or only a single thread?

This documentation is maintained by the Pentaho community, and members are encouraged to create new pages in the appropriate spaces, or edit existing pages that need to be corrected or updated.

Please do not leave comments on Wiki pages asking for help. They will be deleted. Use the forums instead.

Adaptavist Theme Builder (4.2.0) Powered by Atlassian Confluence 3.3.3, the Enterprise Wiki