[jira] [Reopened] (HBASE-12911) Client-side metrics

["Nick Dimiduk (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nick Dimiduk reopened HBASE-12911:
----------------------------------

Reopening for 0.98 backport.

> Client-side metrics
> -------------------
>
>                 Key: HBASE-12911
>                 URL: https://issues.apache.org/jira/browse/HBASE-12911
>             Project: HBase
>          Issue Type: New Feature
>          Components: Client, Operability, Performance
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>             Fix For: 2.0.0, 1.2.0, 1.3.0
>
>         Attachments: 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 0001-HBASE-12911-Client-side-metrics.patch, 12911-0.98.00.patch, 12911-branch-1.00.patch, 12911.yammer.jpg, 12911.yammer.v00.patch, 12911.yammer.v01.patch, 12911.yammer.v02.patch, 12911.yammer.v02.patch, 12911.yammer.v03.branch-1.patch, 12911.yammer.v03.patch, 12911.yammer.v03.patch, 12911.yammer.v04.0.98.patch, 12911.yammer.v04.branch-1.2.patch, 12911.yammer.v04.branch-1.2.patch, 12911.yammer.v04.branch-1.patch, 12911.yammer.v04.branch-1.patch, 12911.yammer.v04.patch, 12911.yammer.v04.patch, am.jpg, client metrics RS-Master.jpg, client metrics client.jpg, conn_agg.jpg, connection attributes.jpg, ltt.jpg, standalone.jpg
>
>
> There's very little visibility into the hbase client. Folks who care to add some kind of metrics collection end up wrapping Table method invocations with {{System.currentTimeMillis()}}. For a crude example of this, have a look at what I did in {{PerformanceEvaluation}} for exposing requests latencies up to {{IntegrationTestRegionReplicaPerf}}. The client is quite complex, there's a lot going on under the hood that is impossible to see right now without a profiler. Being a crucial part of the performance of this distributed system, we should have deeper visibility into the client's function.
> I'm not sure that wiring into the hadoop metrics system is the right choice because the client is often embedded as a library in a user's application. We should have integration with our metrics tools so that, i.e., a client embedded in a coprocessor can report metrics through the usual RS channels, or a client used in a MR job can do the same.
> I would propose an interface-based system with pluggable implementations. Out of the box we'd include a hadoop-metrics implementation and one other, possibly [dropwizard/metrics|https://github.com/dropwizard/metrics].
> Thoughts?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)