You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Karthik Kambatla (JIRA)" <ji...@apache.org> on 2016/10/27 21:22:00 UTC
[jira] [Commented] (YARN-3960) When a leaf queue changes to a
parent queue, allow handling application submissions gracefully
[ https://issues.apache.org/jira/browse/YARN-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15613260#comment-15613260 ]
Karthik Kambatla commented on YARN-3960:
----------------------------------------
Discussed this with [~jrottinghuis] offline.
The problem is - when a leaf queue is changed to be a parent queue with leaf queues, all applications submitted to the queue (through scripts or otherwise) will now fail. This can happen especially when the user does not set any queue at all.
Creating a "default" leaf queue might work in some cases. To ensure the other leaf queues are not at a disadvantage, it might make sense to set the weight to 0. However, that would lead to poorer performance for the jobs without an explicit notification to the user. And, different installations might want to address this differently.
Using queue placement policies should alleviate some of this, at least for the case where the user does not specify a queue at all. We could add more placement rules. The benefit of using placement policies is the admin could set the intended behavior for their cluster.
Thoughts?
> When a leaf queue changes to a parent queue, allow handling application submissions gracefully
> ----------------------------------------------------------------------------------------------
>
> Key: YARN-3960
> URL: https://issues.apache.org/jira/browse/YARN-3960
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: fairscheduler
> Reporter: Siqi Li
> Assignee: Siqi Li
> Labels: oct16-easy
> Attachments: YARN-3960.v1.patch
>
>
> For a tier queue in fair scheduler, if you submits a job without specifying the exact child queue. The job will fail on launching.
> Especially for folks to transition from flat queues to tiered queues, many jobs would end up failing
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org