[jira] [Commented] (YARN-4212) FairScheduler: Parent queues is not allowed to be 'Fair' policy if its children have the "drf" policy

["Karthik Kambatla (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363498#comment-15363498 ] 

Karthik Kambatla commented on YARN-4212:
----------------------------------------

Took a quick look at the patch. 

One high-level question: instead of checking for parents in AllocationFileLoaderService, would it make sense to check for the parent compatibility while creating the queue? That way, we would be able to handle dynamically created queues in addition to those created using the allocation file. Also, we will not have to include a separate map to keep all the parents. 

In general, I am not a fan of the current approach of having all those maps with information about queues outside of FSQueue hierarchy. Having all the information regarding a queue in one place seems the right way to go. I know [~rchiang] was interested in improving this. 

> FairScheduler: Parent queues is not allowed to be 'Fair' policy if its children have the "drf" policy
> -----------------------------------------------------------------------------------------------------
>
>                 Key: YARN-4212
>                 URL: https://issues.apache.org/jira/browse/YARN-4212
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Arun Suresh
>            Assignee: Yufei Gu
>              Labels: fairscheduler
>         Attachments: YARN-4212.002.patch, YARN-4212.003.patch, YARN-4212.004.patch, YARN-4212.1.patch
>
>
> The Fair Scheduler, while performing a {{recomputeShares()}} during an {{update()}} call, uses the parent queues policy to distribute shares to its children.
> If the parent queues policy is 'fair', it only computes weight for memory and sets the vcores fair share of its children to 0.
> Assuming a situation where we have 1 parent queue with policy 'fair' and multiple leaf queues with policy 'drf', Any app submitted to the child queues with vcore requirement > 1 will always be above fairshare, since during the recomputeShare process, the child queues were all assigned 0 for fairshare vcores.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org