Hitachi Vantara Pentaho Community Wiki
Child pages
  • First clustering results
Skip to end of metadata
Go to start of metadata

Splits up into a master and executes automatically usining CTRL-U in the latest Spoon.

Execution results (Master):

Step name

Copy Nr

Read

Written

Input

Output

Updated

Errors

Active

Time

Speed

pr/in/out

Sleeps

Reader : 60000)

0

0

10000

10000

0

0

0

Finished

22.5

445.0

-

0/23845

Assemble

0

20000

20000

0

0

0

0

Finished

24.0

834.9

-

24533/82638

Reader : 60001)

0

0

10000

10000

0

0

0

Finished

24.0

417.4

-

0/112479

Read

0

0

10000

10001

0

0

0

Finished

15.2

657.8

-

0/109743

Writer : 60002)

0

10000

0

0

0

0

0

Finished

15.8

633.6

-

119571/0

Writer : 60003)

0

10000

0

0

0

0

0

Finished

16.1

620.1

-

2366/0

Read 2

0

0

10000

10001

0

0

0

Finished

15.2

658.4

-

0/112458

Get zipcode-region

0

0

2774

0

0

0

0

Finished

7.4

376.9

-

0/7347

Writer : 60004)

0

2774

0

0

0

0

0

Finished

8.2

337.5

-

854/0

Writer : 60005)

0

2774

0

0

0

0

0

Finished

7.7

359.3

-

5282/0

Table output

0

20000

0

0

20000

0

0

Finished

23.9

836.0

-

38059/0

And Two slaves: 

One:

Step name

Copy Nr

Read

Written

Input

Output

Updated

Errors

Active

Time

Speed

pr/in/out

Sleeps

Look up Region using zipcode

0

12774

10000

0

0

0

0

Finished

18.4

692.8

-

5930/221122

Writer : 60001)

0

10000

0

0

0

0

0

Finished

20.5

488.9

-

68926/0

Reader : 60003)

0

0

10000

10000

0

0

0

Finished

15.8

634.3

-

0/80362

Calc

0

10000

10000

0

0

0

0

Finished

17.3

577.6

-

34376/107801

Value Mapper

0

10000

10000

0

0

0

0

Finished

18.4

544.2

-

2914/260912

Reader : 60005)

0

0

2774

2774

0

0

0

Finished

5.6

498.7

-

0/0

Two:

Step name

Copy Nr

Read

Written

Input

Output

Updated

Errors

Active

Time

Speed

pr/in/out

Sleeps

Look up Region using zipcode

0

12774

10000

0

0

0

0

Finished

12.6

1010.1

-

62589/152

Writer : 60000)

0

10000

0

0

0

0

0

Finished

13.9

721.1

-

111172/0

Reader : 60002)

0

0

10000

10000

0

0

0

Finished

12.6

792.7

-

0/4485

Calc

0

10000

10000

0

0

0

0

Finished

12.6

791.8

-

81480/29648

Value Mapper

0

10000

10000

0

0

0

0

Finished

12.6

792.1

-

79631/34977

Reader : 60004)

0

0

2774

2774

0

0

0

Finished

4.5

611.4

-

0/0

  • No labels

3 Comments

  1. user-25d0d

    So what was the total time for each of the two configurations?

    James

  2. user-dc351

    From the data it looks like it took 24 seconds in total.  However, this is not a performance driven example, rather than a functionality test.

    The test here was to see if the system could handle remote stream lookup driven by local data.  In the case where you have a remote server with lots of RAM, that might make sense.

    This sample is obviously limited by the text file input and table output steps, NOT the stream lookup.