You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Karthik Kambatla (JIRA)" <ji...@apache.org> on 2015/01/15 07:07:34 UTC
[jira] [Commented] (YARN-2990) FairScheduler's delay-scheduling
always waits for node-local and rack-local delays, even for off-rack-only
requests
[ https://issues.apache.org/jira/browse/YARN-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14278300#comment-14278300 ]
Karthik Kambatla commented on YARN-2990:
----------------------------------------
Looked more closely, unfortunately this appears to be by design. Each application has an "allowed locality level" - initially node-local, and transitions to rack-local and off-switch after the corresponding delays. Instead, it might be better to track this allowed locality level per {{ResourceRequest}}. I propose:
# In the short-term, to address the case where the AM has to go through the node-local and rack-local delays, we could start with the default locality level of off-switch and reset it node-local after the AM is allocated.
# In the long-term, let us augment ResourceRequest to include allowed locality level.
Thoughts?
> FairScheduler's delay-scheduling always waits for node-local and rack-local delays, even for off-rack-only requests
> -------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-2990
> URL: https://issues.apache.org/jira/browse/YARN-2990
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 2.6.0
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: yarn-2990-test.patch
>
>
> Looking at the FairScheduler, it appears the node/rack locality delays are used for all requests, even those that are only off-rack.
> More details in comments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)