You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Jonathan Ellithorpe (JIRA)" <ji...@apache.org> on 2015/11/17 06:15:10 UTC
[jira] [Updated] (HADOOP-12574) Multiple Xmx Parameters to Java
Process
[ https://issues.apache.org/jira/browse/HADOOP-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellithorpe updated HADOOP-12574:
-----------------------------------------
Description:
When launching hadoop, the setting of environment variable HADOOP_HEAPMAX results in multiple Xmx parameters being passed to the java application launcher. This is because a default Xmx setting included in the HADOOP_CLIENT_OPTS in etc/hadoop/hadoop-env.sh, line 61:
{noformat}
# The following applies to multiple commands (fs, dfs, fsck, distcp etc)
export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"
{noformat}
is merged into the variable HADOOP_OPTS in bin/hadoop, line 160:
{noformat}
# Always respect HADOOP_OPTS and HADOOP_CLIENT_OPTS
HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"
{noformat}
Orthogonally, the HADOOP_HEAPMAX environment variable triggers the setting of JAVA_HEAP_MAX in hadoop-config.sh, line 192:
{noformat}
# check envvars which might override default args
if [ "$HADOOP_HEAPSIZE" != "" ]; then
#echo "run with heapsize $HADOOP_HEAPSIZE"
JAVA_HEAP_MAX="-Xmx""$HADOOP_HEAPSIZE""m"
#echo $JAVA_HEAP_MAX
fi
{noformat}
Which is ultimately passed to the java application launcher in bin/hadoop, line 166:
{noformat}
exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
{noformat}
The result: two Xmx settings passed to the Java application launcher. Which one is respected depends on the JVM, but for HotSpot the rightmost setting is respected, and so the HADOOP_HEAPMAX environment variable has no effect and the default setting in HADOOP_CLIENT_OPTS wins.
was:
When launching hadoop, the setting of environment variable HADOOP_HEAPMAX results in multiple Xmx parameters being passed to the java application launcher. This is because a default Xmx setting included in the HADOOP_CLIENT_OPTS in etc/hadoop/hadoop-env.sh, line 61:
# The following applies to multiple commands (fs, dfs, fsck, distcp etc)
export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"
is merged into the variable HADOOP_OPTS in bin/hadoop, line 160:
# Always respect HADOOP_OPTS and HADOOP_CLIENT_OPTS
HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"
Orthogonally, the HADOOP_HEAPMAX environment variable triggers the setting of JAVA_HEAP_MAX in hadoop-config.sh, line 192:
# check envvars which might override default args
if [ "$HADOOP_HEAPSIZE" != "" ]; then
#echo "run with heapsize $HADOOP_HEAPSIZE"
JAVA_HEAP_MAX="-Xmx""$HADOOP_HEAPSIZE""m"
#echo $JAVA_HEAP_MAX
fi
Which is ultimately passed to the java application launcher in bin/hadoop, line 166:
exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
The result: two Xmx settings passed to the Java application launcher. Which one is respected depends on the JVM, but for HotSpot the rightmost setting is respected, and so the HADOOP_HEAPMAX environment variable has no effect and the default setting in HADOOP_CLIENT_OPTS wins.
> Multiple Xmx Parameters to Java Process
> ---------------------------------------
>
> Key: HADOOP-12574
> URL: https://issues.apache.org/jira/browse/HADOOP-12574
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.7.1
> Reporter: Jonathan Ellithorpe
>
> When launching hadoop, the setting of environment variable HADOOP_HEAPMAX results in multiple Xmx parameters being passed to the java application launcher. This is because a default Xmx setting included in the HADOOP_CLIENT_OPTS in etc/hadoop/hadoop-env.sh, line 61:
> {noformat}
> # The following applies to multiple commands (fs, dfs, fsck, distcp etc)
> export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"
> {noformat}
> is merged into the variable HADOOP_OPTS in bin/hadoop, line 160:
> {noformat}
> # Always respect HADOOP_OPTS and HADOOP_CLIENT_OPTS
> HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"
> {noformat}
> Orthogonally, the HADOOP_HEAPMAX environment variable triggers the setting of JAVA_HEAP_MAX in hadoop-config.sh, line 192:
> {noformat}
> # check envvars which might override default args
> if [ "$HADOOP_HEAPSIZE" != "" ]; then
> #echo "run with heapsize $HADOOP_HEAPSIZE"
> JAVA_HEAP_MAX="-Xmx""$HADOOP_HEAPSIZE""m"
> #echo $JAVA_HEAP_MAX
> fi
> {noformat}
> Which is ultimately passed to the java application launcher in bin/hadoop, line 166:
> {noformat}
> exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
> {noformat}
> The result: two Xmx settings passed to the Java application launcher. Which one is respected depends on the JVM, but for HotSpot the rightmost setting is respected, and so the HADOOP_HEAPMAX environment variable has no effect and the default setting in HADOOP_CLIENT_OPTS wins.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)