Re: YARN Fair Scheduler - maxResources limit for parent and child queues with inheritance

[Ashwin Sai Shankar <as...@netflix.com>]

Yes this is a issue with webui. maxResources would be enforced at
*someParentQueue,
*so childQueues of it will get atmost ${maxResources} depending
on usage of its siblings.

On Tue, Sep 29, 2015 at 8:29 AM, David Fox <da...@gmail.com> wrote:

> Hello,
>
> I would like to know how it works with Fair Scheduler in YARN-hadoop. I'm
> trying to configure parent queue with MaxResorces in allocations file, e.i.
> fair-scheduler.xml. Then I want to create child queues. As documentation
> about fair scheduler says "Queues can be arranged in a hierarchy to divide
> resources and configured with weights to share the cluster in specific
> proportions"
> http://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/FairScheduler.html
> I suppose if I configure parent queue with Max Resources, the child queues
> will be limited to max of the parent resources with fair shares. Is my
> undersatnding of queue hierarchy mechanism right?
>
> Example to be more concrete:
> {code}
> <?xml version="1.0"?>
> <allocations>
>     <queue name="someParentQueue" type="parent">
>         <maxResources>111000 mb,100 vcores, 100 disks</maxResources>
>     </queue>
>     <queuePlacementPolicy>
>         <rule name="nestedUserQueue" create="true">
>             <rule name="specified" create="false"/>
>         </rule>
>         <rule name="user" create="true"/>
>     </queuePlacementPolicy>
> </allocations>
> {code}
>
> The queues someParentQueue.user1 has different Max Resources in web UI. Is
> it issue of web UI or intended behavoior of Fair Scheduler?
>
> Thanks
>