You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2013/12/13 18:47:07 UTC
[jira] [Created] (HBASE-10158) Add sync rate histogram to HLogPE
stack created HBASE-10158:
-----------------------------
Summary: Add sync rate histogram to HLogPE
Key: HBASE-10158
URL: https://issues.apache.org/jira/browse/HBASE-10158
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
HLogPE emits a few metrics that help comparing WAL implementations -- sync and append counts. This patch adds a histogram around time between sync calls to ensure implementations are not clumping sync invocations. Helps comparing implementations.
It also makes it so the just committed HBASE-8755 code calls postSync and postAppend methods.
[~eclark] Mind reviewing? Also, are these additions actually needed? Could I dump the metrics we already collect if I only did proper startup invocation; i.e. use our metrics rather than these yammer ones?
Here is what the output looks like on a run of HLogPE:
{code}
...
2013-12-13 09:41:15,641 INFO [StoreOpener-a77539a833048fa441d3195c853cf6ea-1] hfile.CacheConfig: Allocating LruBlockCache with maximum size 395 M
2013-12-13 09:41:15,652 INFO [StoreOpener-a77539a833048fa441d3195c853cf6ea-1] compactions.CompactionConfiguration: size [134217728, 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; delete expired; major period 604800000, major jitter 0.500000
2013-12-13 09:41:15,662 INFO [StoreOpener-a77539a833048fa441d3195c853cf6ea-1] util.ChecksumType: Checksum using org.apache.hadoop.util.PureJavaCrc32
2013-12-13 09:41:15,663 INFO [StoreOpener-a77539a833048fa441d3195c853cf6ea-1] util.ChecksumType: Checksum can use org.apache.hadoop.util.PureJavaCrc32C
2013-12-13 09:41:15,666 DEBUG [main] regionserver.HRegion: Found 0 recovered edits file(s) under file:/Users/stack/checkouts/trunk/target/test-data/a62eb348-138b-4275-b33a-12ebb12c20b9/HLogPerformanceEvaluation/data/default/HLogPerformanceEvaluation/a77539a833048fa441d3195c853cf6ea
2013-12-13 09:41:15,669 INFO [main] regionserver.HRegion: Onlined a77539a833048fa441d3195c853cf6ea; next sequenceid=1
12/13/13 9:42:15 AM ============================================================
org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation:
appendMeter:
count = 1173705160
mean rate = 19180.80 bytes/ms
1-minute rate = 16437.82 bytes/ms
5-minute rate = 12716.38 bytes/ms
15-minute rate = 11783.97 bytes/ms
syncMeter:
count = 1726060
mean rate = 28.20 syncs/ms
1-minute rate = 24.16 syncs/ms
5-minute rate = 18.68 syncs/ms
15-minute rate = 17.30 syncs/ms
org.apache.hadoop.hbase.regionserver.wal.HLogPerformanceEvaluation.nanos-between-syncs:
syncHistogram:
min = 13000.00
max = 32438000.00
mean = 34754.07
stddev = 38993.58
median = 29000.00
75% <= 37000.00
95% <= 56000.00
98% <= 84420.00
99% <= 99000.00
99.9% <= 267709.00
2013-12-13 09:42:25,040 INFO [t0] wal.HLogPerformanceEvaluation: t0 took 69.369s 28831.320ops/s
2013-12-13 09:42:25,040 INFO [main] wal.HLogPerformanceEvaluation: Summary: threads=1, iterations=2000000, syncInterval=0 took 69.369s 28831.320ops/s
...
{code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)