[jira] [Resolved] (AMBARI-10979) MRv2 Config Screen allows me to allocate sort buffers larger than heap.

["Srimanth Gunturi (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMBARI-10979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Srimanth Gunturi resolved AMBARI-10979.
---------------------------------------
    Resolution: Fixed

Tested on live cluster and stack-advisor tests pass.

> MRv2 Config Screen allows me to allocate sort buffers larger than heap.
> -----------------------------------------------------------------------
>
>                 Key: AMBARI-10979
>                 URL: https://issues.apache.org/jira/browse/AMBARI-10979
>             Project: Ambari
>          Issue Type: Bug
>          Components: contrib
>    Affects Versions: 2.1.0
>            Reporter: Srimanth Gunturi
>            Assignee: Srimanth Gunturi
>             Fix For: 2.1.0
>
>         Attachments: AMBARI-10979.patch
>
>
> {{mapreduce.task.io.sort.mb}} needs to be smaller than the min Xmx for maps and reduces (also applies to Tez).
> Ambari allows me to specify an invalid state without any warnings.
> The formulas should be as follows
> {code}
>     'mapreduce.map.memory.mb' = int(configurations["yarn-site"]["properties"]["yarn.scheduler.minimum-allocation-mb"])
>     'mapreduce.reduce.memory.mb' = min(2*int(configurations["yarn-site"]["properties"]["yarn.scheduler.minimum-allocation-mb"]), int(nodemanagerMinRam))
>     mapredMapXmx = int(0.8*int(configurations["mapred-site"]["properties"]["mapreduce.map.memory.mb"]));
>     'mapreduce.map.java.opts' = "-Xmx" + str(mapredMapXmx) + "m"
>     'mapreduce.reduce.java.opts' = "-Xmx" + str(int(0.8*int(configurations["mapred-site"]["properties"]["mapreduce.reduce.memory.mb"]))) + "m"
>     'mapreduce.task.io.sort.mb' = str(min(int(0.7*mapredMapXmx), 2047))
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)