[jira] [Updated] (YARN-4851) Metric improvements for ATS v1.5 storage

["Li Lu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-4851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Li Lu updated YARN-4851:
------------------------
    Attachment: YARN-4851-trunk.001.patch

First draft of the patch. In this patch I've added some metrics for the ATS v1.5 storage. Specifically:

For overall system usage:
- Number of read requests to summary storage
- Number of read requests to detail storage
- Number of entities scanned by EntityGroupFS storage into the summary storage. 
- Accumulated time spent on scanning new apps in the active directory
- Accumulated time spent on reading summary data into the summary storage

Caching performance:
- Number of cache storage refreshes (cache reloads). This can be compared to the number of read requests to detail storage to understand how useful a caching layer is for specific cluster workload. 
- Accumulated time spent on refreshing cache storages. 

Log cleaner/purging:
- Number of dirs purged by the storage
- Accumulated time for log cleaning. 

> Metric improvements for ATS v1.5 storage
> ----------------------------------------
>
>                 Key: YARN-4851
>                 URL: https://issues.apache.org/jira/browse/YARN-4851
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-4851-trunk.001.patch
>
>
> We can add more metrics to the ATS v1.5 storage systems, including purging, cache hit/misses, read latency, etc. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)