You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Tsuyoshi OZAWA (JIRA)" <ji...@apache.org> on 2015/01/16 11:23:35 UTC
[jira] [Commented] (MAPREDUCE-6209) Implement a heuristic to
auto-size Java heap of MRAppMaster container
[ https://issues.apache.org/jira/browse/MAPREDUCE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14280062#comment-14280062 ]
Tsuyoshi OZAWA commented on MAPREDUCE-6209:
-------------------------------------------
[~jira.shegalov] Tez is doing similar approach: task memory * 0.8 is used to avoid OoM. It looks good to me. Do you have additional ideas?
> Implement a heuristic to auto-size Java heap of MRAppMaster container
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-6209
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6209
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
> Reporter: Gera Shegalov
>
> The size of Java heap required by the AM is linearly proportional to the size of the MR job (number of mappers/splits and reducers). it would be nice if users did not have to adjust the AM container size when transitioning from testing job on a small sample to a production job on a full-scale dataset.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)