You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Tan, Wangda (JIRA)" <ji...@apache.org> on 2019/05/22 20:29:00 UTC
[jira] [Commented] (YARN-9576) ResourceUsageMultiNodeLookupPolicy
may cause Application starve forever
[ https://issues.apache.org/jira/browse/YARN-9576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846213#comment-16846213 ]
Tan, Wangda commented on YARN-9576:
-----------------------------------
[~jutia], actually this behavior is not caused by multi-node lookup policy, it is caused by resource fragmentation. There's no good solution for this except queue priority based preemption. See YARN-5864.
> ResourceUsageMultiNodeLookupPolicy may cause Application starve forever
> ------------------------------------------------------------------------
>
> Key: YARN-9576
> URL: https://issues.apache.org/jira/browse/YARN-9576
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: tianjuan
> Priority: Major
>
> eems that ResourceUsageMultiNodeLookupPolicy in YARN-7494 may cause Application starve forever
> for example, there are 10 nodes(h1,h2,...h9,h10), each has 8G memory in cluster, and two queues A,B, each is configured with 50% capacity.
> firstly there are 10 jobs (each requests 6G respurce) is submited to queue A, and each node of the 10 nodes will have a contianer allocated.
> Afterwards, another job JobB which requests 3G resource is submited to queue B, and there will be one container with 3G size reserved on node h1,
> with ResourceUsageMultiNodeLookupPolicy, the order policy will always be h1,h2,..h9,h10, and there will always be one container re-reverved on node h1, no other reservation happen, JobB will hang forever, [~sunilg] what's ypur thought about this situation?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org