You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Allen Wittenauer (JIRA)" <ji...@apache.org> on 2009/07/23 22:07:14 UTC
[jira] Created: (HADOOP-6168) HADOOP_HEAPSIZE cannot be done
per-server easily
HADOOP_HEAPSIZE cannot be done per-server easily
------------------------------------------------
Key: HADOOP-6168
URL: https://issues.apache.org/jira/browse/HADOOP-6168
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 0.18.3
Reporter: Allen Wittenauer
The hadoop script forces a heap that cannot be easily overridden if one wants to push the same config everywhere.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-6168) HADOOP_HEAPSIZE cannot be done
per-server easily
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer resolved HADOOP-6168.
--------------------------------------
Resolution: Duplicate
agreed that this a dupe.
> HADOOP_HEAPSIZE cannot be done per-server easily
> ------------------------------------------------
>
> Key: HADOOP-6168
> URL: https://issues.apache.org/jira/browse/HADOOP-6168
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 0.18.3
> Reporter: Allen Wittenauer
>
> The hadoop script forces a heap that cannot be easily overridden if one wants to push the same config everywhere.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6168) HADOOP_HEAPSIZE cannot be done
per-server easily
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734893#action_12734893 ]
Vinod K V commented on HADOOP-6168:
-----------------------------------
Oh, and I think this issue is a duplicate of HADOOP-6167.
> HADOOP_HEAPSIZE cannot be done per-server easily
> ------------------------------------------------
>
> Key: HADOOP-6168
> URL: https://issues.apache.org/jira/browse/HADOOP-6168
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 0.18.3
> Reporter: Allen Wittenauer
>
> The hadoop script forces a heap that cannot be easily overridden if one wants to push the same config everywhere.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6168) HADOOP_HEAPSIZE cannot be done
per-server easily
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734759#action_12734759 ]
Allen Wittenauer commented on HADOOP-6168:
------------------------------------------
So bin/hadoop has this logic:
JAVA_HEAP_MAX=-Xmx1000m
# check envvars which might override default args
if [ "$HADOOP_HEAPSIZE" != "" ]; then
#echo "run with heapsize $HADOOP_HEAPSIZE"
JAVA_HEAP_MAX="-Xmx""$HADOOP_HEAPSIZE""m"
#echo $JAVA_HEAP_MAX
fi
This makes it impossible to create a hadoop-env.sh that is truly generic to all nodes. It would be better to do this HEAPSIZE management in hadoop-env.sh to allow it to be easily override-able.
> HADOOP_HEAPSIZE cannot be done per-server easily
> ------------------------------------------------
>
> Key: HADOOP-6168
> URL: https://issues.apache.org/jira/browse/HADOOP-6168
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 0.18.3
> Reporter: Allen Wittenauer
>
> The hadoop script forces a heap that cannot be easily overridden if one wants to push the same config everywhere.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6168) HADOOP_HEAPSIZE cannot be done
per-server easily
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734892#action_12734892 ]
Vinod K V commented on HADOOP-6168:
-----------------------------------
Though not the cleanest, there is a way to do this using various HADOOP_*_OPTS. We can set heap size also in these opts, for e.g. HADOOP_JOBTRACKER_OPTS=-Xmx2000m, it will be eventually picked up by the running command. It works this way because jvm takes up the last -Xmx value that occurs in the command line which would be from HADOOP_OPTS.
{code} exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"{code}
> HADOOP_HEAPSIZE cannot be done per-server easily
> ------------------------------------------------
>
> Key: HADOOP-6168
> URL: https://issues.apache.org/jira/browse/HADOOP-6168
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 0.18.3
> Reporter: Allen Wittenauer
>
> The hadoop script forces a heap that cannot be easily overridden if one wants to push the same config everywhere.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.