[jira] [Commented] (YARN-6492) Generate queue metrics for each partition

["Manikandan R (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16282261#comment-16282261 ] 

Manikandan R commented on YARN-6492:
------------------------------------

[~sunilg] Thanks for your comments.

Attached v0 patch. Few notes are

1. Introduced a new PartitionMetrics class similar to QueueMetrics to handle metrics computation based on above discussed structure. Please refer https://issues.apache.org/jira/browse/YARN-6492?focusedCommentId=16113219&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16113219 for structure.
2. Currently, focus is to compute below metrics: 
allocatedMB,allocatedVCores,allocatedContainers,aggregateContainersAllocated,aggregateContainersReleased,availableMB,availableMB, pendingMB, pendingVCores, pendingContainers,reservedMB, reservedMB, reservedContainers.
This list has been inherited from YARN-6467.
3. In addtion to above metrics, incorporated metrics for resource vectors as well. These metrics would be available as AvailableResource.memory-mb, AvailableResource.vcores, PendingResource.memory-mb, PendingResource.vcores etc in addition to above set.
4. Similar to YARN-6467, focus is on Capacity scheduler metrics, Partition Metrics won't be computed in FairScheduler for now.
5. Introduced a property in CS configuration to turn this feature ON or OFF similar to user metrics.
6. User Metrics switch would be honoured with in Partition Metrics as well.
7. As discussed in earlier comments, existing queue metrics would be preserved for backward compatibility.


> Generate queue metrics for each partition
> -----------------------------------------
>
>                 Key: YARN-6492
>                 URL: https://issues.apache.org/jira/browse/YARN-6492
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacity scheduler
>            Reporter: Jonathan Hung
>            Assignee: Manikandan R
>         Attachments: YARN-6492.001.patch, partition_metrics.txt
>
>
> We are interested in having queue metrics for all partitions. Right now each queue has one QueueMetrics object which captures metrics either in default partition or across all partitions. (After YARN-6467 it will be in default partition)
> But having the partition metrics would be very useful.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org