[jira] [Commented] (HBASE-15161) Umbrella: Miscellaneous improvements from production usage

["Enis Soztutar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-15161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15228877#comment-15228877 ] 

Enis Soztutar commented on HBASE-15161:
---------------------------------------

In sub tasks of these for new metrics, especially per-table, per-user and hdfs IO metrics, we discussed briefly about possible perf impact. General agreement is that we should be careful in adding new metrics to not affect the hot code paths. For example, HBASE-15160 adds the fs IO latency metrics back which was removed from before. The way it works is the hfile block reader emits the latency and offers it to a array blocking queue, and later the metrics collector which runs periodically drains these queues and adds it to the latency histogram metrics. Instead of this, I was suggesting to increment the metrics directly inline in the hot code paths directly, because in-theory using {{Counter}}s should be better than using {{ArrayBlockingQueue}} 

To quantify whether it is the case or not, I've written a JMH suite for testing Counter vs AtomicLong vs MetricHistogram vs ArrayBlockingQueue. 
https://github.com/enis/hbase-counters-benchmark/blob/master/src/main/java/org/apache/hbase/CountersBenchmark.java 

Running these with 1,2,4,8,16,24 threads, some of the results are expected and good confirmation (Counter is really far better than AtomicLong), it seems that MutableHistogram is very expensive:
https://docs.google.com/spreadsheets/d/1C_V0Y59Ct4gM_1xhbiaTFk-5OkyJHkR6UkF85NSMWdA/edit?usp=sharing



> Umbrella: Miscellaneous improvements from production usage
> ----------------------------------------------------------
>
>                 Key: HBASE-15161
>                 URL: https://issues.apache.org/jira/browse/HBASE-15161
>             Project: HBase
>          Issue Type: Umbrella
>            Reporter: Yu Li
>            Assignee: Yu Li
>
> We use HBase to (mainly) build index for our search engine in Alibaba. Recently we are upgrading our online cluster from 0.98.12 to 1.x and I'd like to take the opportunity to contribute a bunch of our private patches to community (better late than never, I hope :-)). This is an umbrella to track this effort.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)