[jira] [Commented] (PHOENIX-3655) Metrics for PQS

["Karan Mehta (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554787#comment-16554787 ] 

Karan Mehta commented on PHOENIX-3655:
--------------------------------------

[~elserj] Getting back on this one after a long time. Need some more clarification before I put up the implementation patch.

We have two phases of the patch, first is metric objects and second is exporting those objects.

Approach for first part, we need to use hbase-metrics and hbase-metrics-api modules (and the metrics offered by them). These includes all classes that implement this {{org.apache.hadoop.hbase.metrics.Metric}} interface.

We need to create a new {{org.apache.hadoop.hbase.metrics.MetricRegistryInfo}} and use the static method {{MetricRegistries.global().create(info);}} to get the corresponding {{org.apache.hadoop.hbase.metrics.MetricRegistry}}. This registry object allows us to register new metrics, which in our case, will be conversion of {{org.apache.phoenix.monitoring.GlobalMetric}} to {{org.apache.hadoop.hbase.metrics.Gauge}}.

Approach for second part, this is the part where we need to define the logic that metrics from the registry defined above is pushed to JMX (through Hadoop Metrics 2, the default reporter) and configure others if required. {{HBaseMetrics2HadoopMetricsAdapter}} class is supposed to help us with that, since it converts hbase-metrics to hadoop-metrics. This adapter is an implementation of MetricsSource class from hadoop2-metrics. We need to register it with {{DefaultMetricsSystem}}. This happens as a part of the thread that {{GlobalMetricRegistriesAdapter}} runs every 10 seconds. The initialization for this one happens as a part of {{BaseSourceImpl}} constructor, which is the class we want to move away from, if I understand correctly. Whats the best approach then? Should I manually register the adapter with Hadoop's DefaultMetricsSystem? 

Let me know if this is not clear.

[~apurtell] [~xucang] would appreciate your insights as well.

> Metrics for PQS
> ---------------
>
>                 Key: PHOENIX-3655
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3655
>             Project: Phoenix
>          Issue Type: New Feature
>    Affects Versions: 4.8.0
>         Environment: Linux 3.13.0-107-generic kernel, v4.9.0-HBase-0.98
>            Reporter: Rahul Shrivastava
>            Assignee: Karan Mehta
>            Priority: Major
>             Fix For: 4.15.0
>
>         Attachments: MetricsforPhoenixQueryServerPQS.pdf
>
>   Original Estimate: 240h
>  Remaining Estimate: 240h
>
> Phoenix Query Server runs a separate process compared to its thin client. Metrics collection is currently done by PhoenixRuntime.java i.e. at Phoenix driver level. We need the following
> 1. For every jdbc statement/prepared statement/ run by PQS , we need capability to collect metrics at PQS level and push the data to external sink i.e. file, JMX , other external custom sources. 
> 2. Besides this global metrics could be periodically collected and pushed to the sink. 
> 2. PQS can be configured to turn on metrics collection and type of collect ( runtime or global) via hbase-site.xml
> 3. Sink could be configured via an interface in hbase-site.xml. 
> All metrics definition https://phoenix.apache.org/metrics.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)