[jira] [Commented] (YARN-389) Infinitely assigning containers when the required resource exceeds the cluster's absolute capacity

["Omkar Vinit Joshi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718504#comment-13718504 ] 

Omkar Vinit Joshi commented on YARN-389:
----------------------------------------

[~zjshen] [~bikassaha] I think we should reject the problematic requests at allocate call but not when it is accepted. As that will be a problem.
* For allocate call today we are only rejecting requests if their request is more than what cluster has but we don't do any validation w.r.t. how much a single container will need to run. I think we should add that check. SchedulerUtils#validateResourceRequest().. thoughts??
* We can not reject requests once they are accepted. How the AM will come to know which requests were rejected later? is there anyway we can inform AM about the accepted (earlier) but now rejected requests? One more thing to be considered here is that Node manager having large amount of resources may go down and come back in short span.. (node reconnect or..node removed and added back after very small time)..in whichever case we should not reject that request if it was accepted....large jobs will definitely suffer if few nodes restart in very short span.. thoughts?
                
> Infinitely assigning containers when the required resource exceeds the cluster's absolute capacity
> --------------------------------------------------------------------------------------------------
>
>                 Key: YARN-389
>                 URL: https://issues.apache.org/jira/browse/YARN-389
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Zhijie Shen
>            Assignee: Omkar Vinit Joshi
>
> I've run wordcount example on branch-2 and trunk. I've set yarn.nodemanager.resource.memory-mb to 1G and yarn.app.mapreduce.am.resource.mb to 1.5G. Therefore, resourcemanager is to assign a 2G AM container for AM. However, the nodemanager doesn't have enough memory to assign the container. The problem is that the assignment operation will be repeated infinitely, if the assignment cannot be accomplished. Logs follow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira