You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Bikas Saha (JIRA)" <ji...@apache.org> on 2013/03/22 19:29:16 UTC
[jira] [Commented] (YARN-389) Infinitely assigning containers when
the required resource exceeds the cluster's absolute capacity
[ https://issues.apache.org/jira/browse/YARN-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13611037#comment-13611037 ]
Bikas Saha commented on YARN-389:
---------------------------------
bq. I've checked reason why the requested AM size is larger than yarn.scheduler.maximum-allocation-mb. In fact, the AM size is not checked before requesting a container. It is because MR jobs are submitted through ResourceMgrDelegate and managed by MRAppMaster, while checking AM size against the maximum resource capability is only implemented in Client and ApplicationMaster of the distributed shell.
The AM launcher checking limits is great but its the RM's responsibility to not accept requests that it cannot fulfill. From reading the comments, it not clear to me what exactly the root issue is in the RM itself. Is it that the RM is accepting container requests that are greater than the maximum resource available on any 1 node? I dont think we should be comparing against the entire cluster resource since a container request needs to be satisfied within a single node.
> Infinitely assigning containers when the required resource exceeds the cluster's absolute capacity
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-389
> URL: https://issues.apache.org/jira/browse/YARN-389
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Zhijie Shen
> Assignee: Zhijie Shen
>
> I've run wordcount example on branch-2 and trunk. I've set yarn.nodemanager.resource.memory-mb to 1G and yarn.app.mapreduce.am.resource.mb to 1.5G. Therefore, resourcemanager is to assign a 2G AM container for AM. However, the nodemanager doesn't have enough memory to assign the container. The problem is that the assignment operation will be repeated infinitely, if the assignment cannot be accomplished. Logs follow.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira