[jira] [Comment Edited] (PHOENIX-5436) Histogram view of Global Client Metrics

["abhishek sen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-5436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919828#comment-16919828 ] 

abhishek sen edited comment on PHOENIX-5436 at 8/30/19 7:20 PM:
----------------------------------------------------------------

Includes the class GlobalClientHistogramMetrics which uses [MutableTimeRangeHistogram|https://hbase.apache.org/2.0/devapidocs/index.html?org/apache/hadoop/metrics2/lib/MutableTimeHistogram.html] to maintain a histogram of metrics values. This patch maintains the histogram for these metrics only - MUTATION_BYTES, MUTATION_COMMIT_TIME, SCAN_BYTES, TASK_QUEUE_WAIT_TIME, TASK_END_TO_END_TIME, TASK_EXECUTION_TIME. The class retrieves the same Metric Registry as used by the GlobalClientMetrics class. It adds new attributes values in the same MBean which currently has the global client metrics as the attribute values. These attribute values are - mean, median, max, nth percentile latency, value in each bin of the histogram etc - added by the [MutableRangeHistogram|http://hbase.apache.org/2.0/devapidocs/org/apache/hadoop/metrics2/lib/MutableRangeHistogram.html] and [MutableHistogram|https://hbase.apache.org/2.0/devapidocs/index.html?org/apache/hadoop/metrics2/lib/MutableHistogram.html] classes while taking a snapshot. 

 A new config phoenix.query.global.histogram.metrics.enabled is added to enable/disable the histogram metrics.

[~vincentpoon]  

 


was (Author: abhishek.sen):
Includes the class GlobalClientHistogramMetrics which uses [MutableTimeRangeHistogram|https://hbase.apache.org/2.0/devapidocs/index.html?org/apache/hadoop/metrics2/lib/MutableTimeHistogram.html] to maintain a histogram of metrics values. This patch maintains the histogram for these metrics only - MUTATION_BYTES, MUTATION_COMMIT_TIME, SCAN_BYTES, TASK_QUEUE_WAIT_TIME, TASK_END_TO_END_TIME, TASK_EXECUTION_TIME. The class retrieves the same Metric Registry as used by the GlobalClientMetrics class. It adds new attributes values in the same MBean which currently has the global client metrics as the attribute values. These attribute values are - mean, median, max, nth percentile latency, value in each bin of the histogram etc - added by the [MutableRangeHistogram|http://hbase.apache.org/2.0/devapidocs/org/apache/hadoop/metrics2/lib/MutableRangeHistogram.html] and [MutableHistogram|https://hbase.apache.org/2.0/devapidocs/index.html?org/apache/hadoop/metrics2/lib/MutableHistogram.html] classes while taking a snapshot. 

 A new config phoenix.query.global.histogram.metrics.enabled is added to enable/disable the histogram metrics.

 

> Histogram view of Global Client Metrics
> ---------------------------------------
>
>                 Key: PHOENIX-5436
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5436
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: abhishek sen
>            Priority: Minor
>              Labels: histogram, metric-collector, metrics
>         Attachments: Histogram_View_of_some_Global_Client_metrics.patch
>
>   Original Estimate: 336h
>  Remaining Estimate: 336h
>
> Currently the [GlobalClientMetrics|https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/monitoring/GlobalClientMetrics.java] enum-class defines Phoenix Global Client Metrics as an enum and each metric-enum tracks a [Phoenix Metric|https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/monitoring/Metric.java] object implemented by [AtomicMetric|https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/monitoring/AtomicMetric.java] class. This tracks a single counter, but in some use cases we want the distribution of a particular metric value over some period of time. One example could be the metric [TASK_EXECUTION_TIME|https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/monitoring/MetricType.java#L47]. The Global Client metric tracks the aggregate value for the execution time of the task. More useful information to monitor would be the distribution of task execution time instead. 
>  Now, in order to incorporate histogram view of the metric, we can use the [Histogram Implementation|https://github.com/apache/hbase/blob/master/hbase-metrics/src/main/java/org/apache/hadoop/hbase/metrics/impl/HistogramImpl.java] from hbase-metrics-api package. The current GlobalClientMetric also ultimately adapts to hbase-metrics-api interface but it [implements|https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/monitoring/GlobalClientMetrics.java#L155] the [Gauge|https://github.com/apache/hbase/blob/master/hbase-metrics-api/src/main/java/org/apache/hadoop/hbase/metrics/Gauge.java] class which can only keep track of a single value not a histogram. So one way could be to create a new GlobalClientHistogramMetrics enum class which keeps track of a Histogram (based on [MutableTimeHistogram|https://github.com/apache/hbase/blob/master/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MutableTimeHistogram.java]) for each enum-metric instead of a single counter as it is now. The updating of the these new metric can go [here|https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/job/JobManager.java#L289] within JobManager class where the currently available Global as well as request-level metrics are being updated.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)