You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Maysam Yabandeh (JIRA)" <ji...@apache.org> on 2014/10/02 23:57:34 UTC

[jira] [Created] (MAPREDUCE-6118) Uber-mode decision does not consider -Xmx

Maysam Yabandeh created MAPREDUCE-6118:
------------------------------------------

             Summary: Uber-mode decision does not consider -Xmx
                 Key: MAPREDUCE-6118
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6118
             Project: Hadoop Map/Reduce
          Issue Type: Bug
            Reporter: Maysam Yabandeh
            Priority: Minor


Current the decision on using uber-mode is based on the mapper container size and the AM container size. However the actual memory at AM is limited by -Xmx options passed via yarn.app.mapreduce.am.command-opts.

For example: AM memory: 4G, yarn.app.mapreduce.am.command-opts=-Xmx2048GB, mapper memory: 3GB. Here the uber job could face OOM whereas a non-uber execution would not.

We faced this problem recently and forced to disable uber-mode to circumvent the problem. One idea, although a bit ugly, is to parse jvm opts, extract Xmx, and take it into account for uber-mode decision.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)