You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-issues@hadoop.apache.org by "Karthik Kambatla (JIRA)" <ji...@apache.org> on 2015/07/07 08:13:05 UTC

[jira] [Commented] (YARN-3891) Apps are starved for resource due to default FairSharePolicy in Fair Scheduler

    [ https://issues.apache.org/jira/browse/YARN-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616243#comment-14616243 ] 

Karthik Kambatla commented on YARN-3891:
----------------------------------------

In these cases, MR should preempt reducers and spawn mappers. A bug in headroom calculation (YARN-3485) leads to this MR issue. Could this be the same issue? MAPREDUCE-6302 is related. 

> Apps are starved for resource due to default FairSharePolicy in Fair Scheduler
> ------------------------------------------------------------------------------
>
>                 Key: YARN-3891
>                 URL: https://issues.apache.org/jira/browse/YARN-3891
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Siqi Li
>            Priority: Critical
>
> Based on FairSharePolicy, FSLeafQueue will try to schedule an app that has the lowest resource consumption first.
> However, we have seen some apps that have several thousand reducers stuck at waiting for some newly scheduled mappers to launch. However, it would be very difficult for such apps to get resources if there are many other jobs with low resource consumption and the queue is full.
> If the queue keeps getting new applications, then the stuck reducers would never finish.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)