[jira] [Created] (HIVE-16515) Hive MapRedTask should has its own OPTS environment variable

["chuanjie.duan (JIRA)" <ji...@apache.org>]

chuanjie.duan created HIVE-16515:
------------------------------------

             Summary: Hive MapRedTask should has its own OPTS environment variable 
                 Key: HIVE-16515
                 URL: https://issues.apache.org/jira/browse/HIVE-16515
             Project: Hive
          Issue Type: Improvement
          Components: HiveServer2
    Affects Versions: 1.2.1
            Reporter: chuanjie.duan
            Priority: Minor


Hiveserver2 submit a mapred job using Hiveserver's variable  HADOOP_OPTS and HADOOP_CLIENT_OPTS, it's weird. eg: hiveserver started with HADOOP_CLIENT_OPTS="-Xms12000m -Xmx12000m", hive mr sub process would need 12g memory allocate. I think  maybe beeline should have another environment variable like HIVE_HEAPSIZE to overwrite HADOOP_CLIENT_OPTS when hive mr job submit

MapRedTask.java
public int execute(DriverContext driverContext) {
     ……
if (ShimLoader.getHadoopShims().isLocalMode(conf)) {
        ......
      } else {
        // nothing to do - we are not running in local mode - only submitting
        // the job via a child process. in this case it's appropriate that the
        // child jvm use the same memory as the parent jvm
      }
     ……
String jarCmd = hiveJar + " " + ExecDriver.class.getName() + libJarsOption;

      String cmdLine = hadoopExec + " jar " + jarCmd + " -plan "
          + planPath.toString() + " " + isSilent + " " + hiveConfArgs;
      ……
     }
      // Run ExecDriver in another JVM
      executor = Runtime.getRuntime().exec(cmdLine, env, new File(workDir));

${HADOOP_HOME}/bin/hadoop
......
# Always respect HADOOP_OPTS and HADOOP_CLIENT_OPTS
    HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"
......
    export CLASSPATH=$CLASSPATH
    exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)