[jira] [Updated] (YARN-4484) Available Resource calculation for a queue is not correct when used with labels

["Sunil G (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-4484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sunil G updated YARN-4484:
--------------------------
    Attachment: 0004-YARN-4484.patch

Hi [~leftnoteasy]
Thank you for sharing the comments.

bq.a queue has >0 available resource from default partition and another partition, when container allocated/released, we should modify available resource from both partitions
I have test case named {{testQueueMetricsWithLabelsOnDefaultLabelNode}} for handling scenario where we has Default and another label configured for one Queue. And it shares few resource from default-partition and from partition=x. And I am getting expected results.

To clear out the scenario:
1. 1 node with 10GB for label "x" and another node for "default_partition"
2. Allocated 1GB AM container from node2 ("default_partition")
3. Allocated 5 * 1GB containers from node1 (label = x)
4. Allocate 1GB container from node2 ("default_partition")

Now capacity distribution is as follows.
QueueA 
     --->  label X to take 50%
     ---> Default Partition to take 25%

So QueueA can get 5Gb from Node1(partition x) and 2.5GB from (default_partition).

Test verifies below for Queue A (metrics)
Total Available resource is 0.5GB (this from node2)
Total Allocated resource is 7GB

This metric calculation is updated only on QueueMetrics. For label, we are already handling in QueuCapacity. Ref:: {{CSQueueUtils#updateUsedCapacity}}

Updating test case by adding more clarity and rebasing. Could you please help to check the same.

> Available Resource calculation for a queue is not correct when used with labels
> -------------------------------------------------------------------------------
>
>                 Key: YARN-4484
>                 URL: https://issues.apache.org/jira/browse/YARN-4484
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacity scheduler
>    Affects Versions: 2.7.1
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-4484.patch, 0002-YARN-4484.patch, 0003-YARN-4484-v2.patch, 0003-YARN-4484.patch, 0004-YARN-4484.patch
>
>
> To calculate available resource for a queue, we have to get the total resource allocated for all labels in queue compare to its usage. 
> Also address the comments given in [YARN-4304-comments|https://issues.apache.org/jira/browse/YARN-4304?focusedCommentId=15064874&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15064874 ] given by [~leftnoteasy] for same.
> ClusterMetrics related issues will also get handled once we fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)