You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Andrew Wong (Code Review)" <ge...@cloudera.org> on 2021/03/04 02:00:45 UTC
[kudu-CR] wip [tools] add a histogram diffs to metrics parser
Andrew Wong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17150
Change subject: wip [tools] add a histogram diffs to metrics parser
......................................................................
wip [tools] add a histogram diffs to metrics parser
Histogram counts get reported in one big go with all counts of the
histogram reported in each metrics record. Reporting on this can be
useful for shortlived histograms, but as the histogram grows larger,
changes in distribution become significantly less visible.
This patch diffs the counts between metric records, and posts
percentiles of the diffs.
$ /data/3/awong/Repositories/kudu/build/release/bin/kudu diagnose parse_metrics kudu-tserver.kudu-worker12.company.com.kudu.diagnostics.20210123-201217.0.74565 --histogram_metrics=server.handler_latency_kudu_tserver_TabletServerService_Scan:scan_rpc_us
I0303 17:46:17.438558 169329 diagnostics_log_parser.cc:292] collecting histogram metric server.handler_latency_kudu_tserver_TabletServerService_Scan as scan_rpc_us
timestamp scan_rpc_us_count scan_rpc_us_min scan_rpc_us_p50 scan_rpc_us_p75 scan_rpc_us_p95 scan_rpc_us_p99 scan_rpc_us_p99_99 scan_rpc_us_max scan_rpc_us_diff_count scan_rpc_us_diff_min scan_rpc_us_diff_p50 scan_rpc_us_diff_p75 scan_rpc_us_diff_p95 scan_rpc_us_diff_p99 scan_rpc_us_diff_p99_99 scan_rpc_us_diff_max
1611432793767488 434170147 2 1215 1639 3711 12927 501759 8650751 434170147 2 1215 1639 3711 12927 501759 8650751
1611432853767552 434198546 2 1215 1639 3711 12927 501759 8650751 28399 2 1815 2399 4735 25215 501759 503807
1611432913767616 434227285 2 1215 1639 3711 12927 501759 8650751 28739 2 1607 1975 2991 15679 354303 501759
1611432973767689 434255021 2 1215 1639 3711 12927 501759 8650751 27736 2 1823 2431 7039 26879 501759 501759
1611433033767772 434283184 2 1215 1639 3711 12927 501759 8650751 28163 2 1727 2271 4927 7071 415743 473087
1611433093767841 434309627 2 1215 1639 3711 12927 501759 8650751 26443 2 1719 2191 5599 41727 466943 501759
1611433153767902 434339928 2 1215 1639 3711 12927 501759 8650751 30301 2 1519 1831 2895 24447 309247 331775
1611433213767967 434366776 2 1215 1639 3711 12927 501759 8650751 26848 2 1439 1743 2447 4351 501759 501759
1611433273768032 434394555 2 1215 1639 3711 12927 501759 8650751 27779 2 1495 1887 2959 24959 501759 501759
1611433333768067 434420683 2 1215 1639 3711 12927 501759 8650751 26128 2 1511 1887 6047 9087 501759 501759
1611433393768130 434447991 2 1215 1639 3711 12927 501759 8650751 27308 2 1647 2207 4319 66047 503807 507903
1611433453768205 434476348 2 1215 1639 3711 12927 501759 8650751 28357 2 1775 2463 6143 16255 440319 460799
1611433513768270 434498551 2 1215 1639 3711 12927 501759 8650751 22203 2 1607 2271 7199 22015 501759 501759
wip:
- haven't decided whether to do this by default or expose a gflag for it
- the current implementation keeps track of a single diff between the current
and previous metrics record. could consider tracking multiple and expanding the
histogram window.
- needs tests
Change-Id: I5e96849b03812dd845aead6a489b4727ce25c365
---
M src/kudu/tools/diagnostics_log_parser.cc
1 file changed, 68 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/17150/1
--
To view, visit http://gerrit.cloudera.org:8080/17150
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5e96849b03812dd845aead6a489b4727ce25c365
Gerrit-Change-Number: 17150
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>