Hitachi Vantara Pentaho Community Wiki
Child pages
  • .14 Logging
Skip to end of metadata
Go to start of metadata

Logging

Logging Description


 PDI is configured to provide helpful log messages to help provide understanding in how a job or transformation is running.  Logging can be configured to provide minimal logging information, just to know whether a job or transformation failed or was successful, or detailed in providing errors or warnings such as network issues or mis-configurations.

  • Nothing: Don't show any output
  • Error: Only show errors
  • Minimal: Only use minimal logging
  • Basic: This is the default basic logging level
  • Detailed: Give detailed logging output
  • Debug: For debugging purposes, very detailed output.
  • Row level: Logging at a row level, this can generate a lot of data.

When executing a job or transformation from within the Spoon development environment, a "Logging" tab is available, showing any log messages that have been generated.  Any error messages are shown with red text, to easily identify the cause of any errors in the file.

Step Metrics

The Step Metrics tab provides a view on the execution of a transformation or job. Transformation steps and Job entries will be highlighted in red if they fail during execution.

Transformation Step Metrics Details

The log grid displays the following details for each step running in the transformation:

Option

Description

Stepname

The name of the step

Copynr

Copy number of the step

Read

Number of lines read from input-streams

Written

Number of lines written to output-streams

Input

Number of lines read from file or database

Output

Number of lines written to file or database

Updated

Number of lines updated in the database

Rejected

 

Errors

Number of errors that occurred

Active

The status of the step: running, finished or stopped

Time

The number of seconds that the step has been running.

Speed

The speed in rows per second at which the step processes rows.

 

Priority of the step (10=highest, 1=lowest), nr of rows in the input-stream(s), nr of rows in the output-stream(s).

input/output

Sleep time (get/put) is the time that a step had to go to sleep (in nano seconds) because the input buffer was empty (get) or the output buffer was full (put).

Note: The system is tuning the steps priority in such a way that the slowest steps get the highest priority.

Job Step Metrics

The log grid displays the following details for each job entry executing in the Job:

Option

Description

Job/Job Entry

The name of the job / job entry

Comment

A comment on the state of the entry execution

Result

The result (success or failure) of the job entry

Reason

Reason: why was this job entry started?

Nr

The value of the nr variable in the result object (available in evaluation Javascript)

Log date

Log date: logging date, corresponds with the start or end of the job entry.

Buttons

Transformation Buttons

Start

This button starts the transformation. Please note that Spoon tries to launch this from the XML-file or repository. It is therefore necessary that the transformation is saved. The output of the execution is displayed in the Log Text part of the Log View.

Preview (debug)

This button launches the Transformation Debug dialog allowing you to specify the number of rows to preview and define conditional breakpoints for the preview execution. After configuring the debug information, click the 'Quick Launch' button to begin the preview execution for the currently selected step. The output of the execution is displayed in the Log Text part of the Log View.

Debug Options

The following table provides a detailed description of the debug options:

Option

Description

Step List

The step list on the left displays a list of available steps from the current transformation. Select a step to begin configuring related options like number of rows and break-points.

Number of rows to retrieve

Enter the rows per step you want to preview for the selected step. After the requested rows are obtained from the different steps, the transformation is ended and the result is shown.

Note: This option will only take effect when the 'Retrieve first rows' option is checked.

Retrieve first rows (preview)

Enable this to restrict the preview size to the number of rows specified above.

Pause transformation on condition

Enable this option to cause the transformation to pause if one of the conditional break-points evaluates to true during execution.

Break-point/pause condition

Enter conditions based on comparing one field to another field or value.

Show error lines

This button displays all lines of the Log Text that contain the word ERROR (lower- or uppercase). You can then choose to edit the source step of the error.

Clear log

This clears the text in the Log Text Window.

Log Settings

If you put a text in the filter field, only the lines that contain this text will be shown in the Log Text window. The "Log level" setting allows you to select the logging level. You can choose one of these:

  •  

If the "Enable time" option is enabled, all lines in the logging will be preceded by the time of day.

Hide inactive

Checking this hides all steps that finished processing.

Safe mode

Places the transformation in Safe Mode. Additional row checking is enabled at runtime, see also Safe mode.

Job Buttons

Start job

This button begins execution of the current Job. Please note that Spoon launches attempts to launch the job from an XML file or the Kettle repository. It is therefore necessary that the job is saved. The output of the execution is displayed in the Log Text part of the Logging tab.

Stop job

This button stops a running job.

Refresh log

Refreshes the log window.

Clear log

This clears the text in the Log Text Window.

Auto-refresh

Enable this option to disable the logging window from updating all the time. You might want to do this when you're using a remote desktop (VNC, X11) over a slow network connection.

1 Comment

  1. user-72358

    I would like to point out an error on this page:

    Under: Log Grid -> Transformation Log Grid Details -> Starting after the Updated row, the Option/Description pairs are mismatched. It looks like the Rejected description needs to be inserted in there.