[jira] [Created] (HBASE-18151) Improvement on Histogram

["Phil Yang (JIRA)" <ji...@apache.org>]

Phil Yang created HBASE-18151:
---------------------------------

             Summary: Improvement on Histogram
                 Key: HBASE-18151
                 URL: https://issues.apache.org/jira/browse/HBASE-18151
             Project: HBase
          Issue Type: Improvement
            Reporter: Phil Yang


There are some bugs and some places we can improvement in the new native Histogram.

The default maxExpected value in HistogramImpl is Integer.MAX_VALUE << 2 (https://github.com/apache/hbase/blob/master/hbase-metrics/src/main/java/org/apache/hadoop/hbase/metrics/impl/HistogramImpl.java#L41 ) which is overflow... and will be regarded as -4... And binsMax pf FastLongHistogram will be 255(by the reason of this bug) which is too low (In FastLongHistogram#L75 it says the expected binsMax is 1000? I think it may be also too low. Even if there is no overflow, Integer.MAX_VALUE*4 is too large?

We may need pass the right expected min/max value when we use it because the value may be different in scenarios. And the default value should not be 1000 or Integer.MAX_VALUE*4.

There is an assumption that the value is in uniform distribution. But if the value is latency which is the most use case, the distribution is not uniform. The lower the value, the higher the probability.

Because of these reasons, the 75/90/99 or other percentages may have big error than we expected. We need find a way to reduce the error.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)