[jira] [Comment Edited] (YARN-10807) Parents node labels are incorrectly added to child queues in weight mode

["Qi Zhu (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-10807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358486#comment-17358486 ] 

Qi Zhu edited comment on YARN-10807 at 6/7/21, 9:54 AM:
--------------------------------------------------------

Thanks [~bteke] for this work.

If we'd better skip the not existed node label also in sum logic,  even thought we will sum 0 when it not exisit:
{code:java}
    for (CSQueue queue : childQueues) {
            float weight = Math.max(0,
                queue.getQueueCapacities().getWeight(nodeLabel));
            sumOfWeight += weight;
     }
{code}
Other things LGTM.

Thanks.


was (Author: zhuqi):
Thanks [~bteke] for this work.

If we can skip the not existed also in sum logic,  even thought we will sum 0 when it not exisit:
{code:java}
    for (CSQueue queue : childQueues) {
            float weight = Math.max(0,
                queue.getQueueCapacities().getWeight(nodeLabel));
            sumOfWeight += weight;
   }
{code}
Other things LGTM.

Thanks.

> Parents node labels are incorrectly added to child queues in weight mode 
> -------------------------------------------------------------------------
>
>                 Key: YARN-10807
>                 URL: https://issues.apache.org/jira/browse/YARN-10807
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Benjamin Teke
>            Assignee: Benjamin Teke
>            Priority: Major
>         Attachments: YARN-10807.001.patch
>
>
> In ParentQueue.updateClusterResource when calculating the normalized weights CS will iterate through the parent's nodelabels. If the parent has a node label that a specific child doesn't it will incorrectly added to the child's node label list through the queueCapacities.setNormalizedWeights(label, weight) call:
> {code:java}
> // Normalize weight of children
>       if (getCapacityConfigurationTypeForQueues(childQueues)
>           == QueueCapacityType.WEIGHT) {
>         for (String nodeLabel : queueCapacities.getExistingNodeLabels()) {
>           float sumOfWeight = 0;
>           for (CSQueue queue : childQueues) {
>             float weight = Math.max(0,
>                 queue.getQueueCapacities().getWeight(nodeLabel));
>             sumOfWeight += weight;
>           }
>           // When sum of weight == 0, skip setting normalized_weight (so
>           // normalized weight will be 0).
>           if (Math.abs(sumOfWeight) > 1e-6) {
>             for (CSQueue queue : childQueues) {
>                 queue.getQueueCapacities().setNormalizedWeight(nodeLabel,
>                     queue.getQueueCapacities().getWeight(nodeLabel) / sumOfWeight);
>             }
>           }
>         }
>       }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org