[jira] [Created] (AMBARI-10979) MRv2 Config Screen allows me to allocate sort buffers larger than heap.

["Srimanth Gunturi (JIRA)" <ji...@apache.org>]

Srimanth Gunturi created AMBARI-10979:
-----------------------------------------

             Summary: MRv2 Config Screen allows me to allocate sort buffers larger than heap.
                 Key: AMBARI-10979
                 URL: https://issues.apache.org/jira/browse/AMBARI-10979
             Project: Ambari
          Issue Type: Bug
          Components: contrib
    Affects Versions: 2.1.0
            Reporter: Srimanth Gunturi
            Assignee: Srimanth Gunturi
             Fix For: 2.1.0


{{mapreduce.task.io.sort.mb}} needs to be smaller than the min Xmx for maps and reduces (also applies to Tez).

Ambari allows me to specify an invalid state without any warnings.

The formulas should be as follows
{code}
    'mapreduce.map.memory.mb' = int(configurations["yarn-site"]["properties"]["yarn.scheduler.minimum-allocation-mb"])
    'mapreduce.reduce.memory.mb' = min(2*int(configurations["yarn-site"]["properties"]["yarn.scheduler.minimum-allocation-mb"]), int(nodemanagerMinRam))
    mapredMapXmx = int(0.8*int(configurations["mapred-site"]["properties"]["mapreduce.map.memory.mb"]));
    'mapreduce.map.java.opts' = "-Xmx" + str(mapredMapXmx) + "m"
    'mapreduce.reduce.java.opts' = "-Xmx" + str(int(0.8*int(configurations["mapred-site"]["properties"]["mapreduce.reduce.memory.mb"]))) + "m"
    'mapreduce.task.io.sort.mb' = str(min(int(0.7*mapredMapXmx), 2047))
{code}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)