Hitachi Vantara Pentaho Community Wiki
Child pages
  • First clustering results
Skip to end of metadata
Go to start of metadata

Splits up into a master and executes automatically usining CTRL-U in the latest Spoon.

Execution results (Master):

Step name

Copy Nr

Read

Written

Input

Output

Updated

Errors

Active

Time

Speed

pr/in/out

Sleeps

Reader : 60000)

0

0

10000

10000

0

0

0

Finished

22.5

445.0

-

0/23845

Assemble

0

20000

20000

0

0

0

0

Finished

24.0

834.9

-

24533/82638

Reader : 60001)

0

0

10000

10000

0

0

0

Finished

24.0

417.4

-

0/112479

Read

0

0

10000

10001

0

0

0

Finished

15.2

657.8

-

0/109743

Writer : 60002)

0

10000

0

0

0

0

0

Finished

15.8

633.6

-

119571/0

Writer : 60003)

0

10000

0

0

0

0

0

Finished

16.1

620.1

-

2366/0

Read 2

0

0

10000

10001

0

0

0

Finished

15.2

658.4

-

0/112458

Get zipcode-region

0

0

2774

0

0

0

0

Finished

7.4

376.9

-

0/7347

Writer : 60004)

0

2774

0

0

0

0

0

Finished

8.2

337.5

-

854/0

Writer : 60005)

0

2774

0

0

0

0

0

Finished

7.7

359.3

-

5282/0

Table output

0

20000

0

0

20000

0

0

Finished

23.9

836.0

-

38059/0

And Two slaves: 

One:

Step name

Copy Nr

Read

Written

Input

Output

Updated

Errors

Active

Time

Speed

pr/in/out

Sleeps

Look up Region using zipcode

0

12774

10000

0

0

0

0

Finished

18.4

692.8

-

5930/221122

Writer : 60001)

0

10000

0

0

0

0

0

Finished

20.5

488.9

-

68926/0

Reader : 60003)

0

0

10000

10000

0

0

0

Finished

15.8

634.3

-

0/80362

Calc

0

10000

10000

0

0

0

0

Finished

17.3

577.6

-

34376/107801

Value Mapper

0

10000

10000

0

0

0

0

Finished

18.4

544.2

-

2914/260912

Reader : 60005)

0

0

2774

2774

0

0

0

Finished

5.6

498.7

-

0/0

Two:

Step name

Copy Nr

Read

Written

Input

Output

Updated

Errors

Active

Time

Speed

pr/in/out

Sleeps

Look up Region using zipcode

0

12774

10000

0

0

0

0

Finished

12.6

1010.1

-

62589/152

Writer : 60000)

0

10000

0

0

0

0

0

Finished

13.9

721.1

-

111172/0

Reader : 60002)

0

0

10000

10000

0

0

0

Finished

12.6

792.7

-

0/4485

Calc

0

10000

10000

0

0

0

0

Finished

12.6

791.8

-

81480/29648

Value Mapper

0

10000

10000

0

0

0

0

Finished

12.6

792.1

-

79631/34977

Reader : 60004)

0

0

2774

2774

0

0

0

Finished

4.5

611.4

-

0/0

  • No labels

3 Comments

  1. So what was the total time for each of the two configurations?

    James

  2. From the data it looks like it took 24 seconds in total.  However, this is not a performance driven example, rather than a functionality test.

    The test here was to see if the system could handle remote stream lookup driven by local data.  In the case where you have a remote server with lots of RAM, that might make sense.

    This sample is obviously limited by the text file input and table output steps, NOT the stream lookup.