Hitachi Vantara Pentaho Community Wiki
Child pages
  • Helpful Commands for Working with Hadoop Configurations

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migration of unmigrated content due to installation of a new plugin

Excerpt

!Common Images^new-icon.png!Image Added Helpful Scripts, Commands, etc. for Working with Hadoop Configurations.

...

Code Block
find . -wholename "*pentaho-big-data-plugin/plugin.properties" -exec ls {} \; -exec grep -o "active\.hadoop\.configuration=[0-9A-Za-z\-]*" {} \; | cut -f2 -d=

Alias to find the shims directory and change your current directory to it:

Code Block
alias goshims="cd \`find . -wholename \"*pentaho-big-data-plugin/hadoop-configurations\"\`"

Alias to find the active shim's directory and change your current directory to it:


Code Block
alias goshim='SHIM_PROP="`find . -wholename \"*pentaho-big-data-plugin/plugin.properties\" -exec ls {} 2>/dev/null \; | head --lines=1`"; \
                          ACTIVE_SHIM="`grep -o \"active\.hadoop\.configuration=[0-9A-Za-z\-]*\" \"$SHIM_PROP\" | sed \"s/.*=//g\"`"; \
                          HADOOP_CONFIG_PATH="`grep -o \"hadoop\.configurations\.path=[0-9A-Za-z\-]*\" \"$SHIM_PROP\" | sed \"s/.*=//g\"`"; \
                          cd "`dirname \"$SHIM_PROP\"`/$HADOOP_CONFIG_PATH/$ACTIVE_SHIM"'

Change the ResourceManager information from Hortonworks Sandbox to my.resourcemanager.com (replace this with your RM's hostname):

Code Block

find . -name "*-site.xml" -exec sed -i "s/sandbox.hortonworks.com/my.resourcemanager.com/g" {} \;

Change the ResourceManager information from Cloudera QuickStart VM to my.resourcemanager.com (replace this with your RM's hostname):

Code Block

find . -name "*-site.xml" -exec sed -i "s/clouderamanager.cdh5.test/my.resourcemanager.com/g" {} \;

Increase the Mondrian query timeout:

Analyzer reports using Hive datasources (or anything that generates a MapReduce job) can exceed the timeout, here's how to change the timeout quickly from 300 seconds (5 mins) to 600 seconds (10 mins), you can change the 600 to whatever you want:

Code Block

sed -i "s/^mondrian.rolap.queryTimeout=300$/mondrian.rolap.queryTimeout=600/" pentaho-solutions/system/mondrian/mondrian.properties