You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Will Berkeley (Code Review)" <ge...@cloudera.org> on 2018/11/19 23:56:02 UTC
[kudu-CR] [tools] ksck: Add information about replica counts to plain ksck output
Will Berkeley has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11958
Change subject: [tools] ksck: Add information about replica counts to plain ksck output
......................................................................
[tools] ksck: Add information about replica counts to plain ksck output
This adds some information about replica counts on tablet servers to the
output of ksck when ksck is in PLAIN_* mode (i.e. not JSON output). In
PLAIN_CONCISE mode, it outputs a 5-number summary of the distribution of
replicas and lists any outliers:
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1646
First Quartile | 3672
Median | 4075
Third Quartile | 4242
Maximum | 4600
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------------------+---------------
Small | cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 1646
In PLAIN_FULL mode it outputs the replica count for every tablet server:
Tablet Replica Count by Tablet Server
UUID | Host | Replica Count
----------------------------------+--------------------------------+---------------
09d6bf7a02124145b43f43cb7a667b3d | vc1314.halxg.cloudera.com:7050 | 100
23d473f441674d43807fd9e631862bfd | vc1308.halxg.cloudera.com:7050 | 100
2fb5cdac22b0418bb2df456906e42eb4 | vc1306.halxg.cloudera.com:7050 | 101
70f7ee61ead54b1885d819f354eb3405 | vc1316.halxg.cloudera.com:7050 | 95
72fcec63e96f4248ae39d114eb3cd7c9 | vc1318.halxg.cloudera.com:7050 | 94
86708813b37a44bd8e92c711211c8685 | vc1310.halxg.cloudera.com:7050 | 96
a662440710624c02bd5612df32cb0235 | vc1302.halxg.cloudera.com:7050 | 101
c9633273962a4521a32d5e177a118a84 | vc1312.halxg.cloudera.com:7050 | 101
cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 76
I also tested it against an empty cluster.
There's no unit tests added, just because our current testing setup for
ksck makes it really painful to add one for this, and it seemed easy
enough to check out manually. Probably, a follow up should straighten
out ksck-test to make testing ksck changes easier.
Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
---
M src/kudu/tools/ksck_results.cc
M src/kudu/tools/ksck_results.h
2 files changed, 130 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/11958/1
--
To view, visit http://gerrit.cloudera.org:8080/11958
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Gerrit-Change-Number: 11958
Gerrit-PatchSet: 1
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
[kudu-CR] [tools] ksck: Add information about replica counts to plain ksck output
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11958 )
Change subject: [tools] ksck: Add information about replica counts to plain ksck output
......................................................................
[tools] ksck: Add information about replica counts to plain ksck output
This adds some information about replica counts on tablet servers to the
output of ksck when ksck is in PLAIN_* mode (i.e. not JSON output). It
outputs a 5-number summary of the distribution of replicas and lists any
outliers:
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1646
First Quartile | 3672
Median | 4075
Third Quartile | 4242
Maximum | 4600
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------------------+---------------
Small | cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 1646
In PLAIN_FULL mode it additionally outputs the replica count for every
tablet server:
Tablet Replica Count by Tablet Server
UUID | Host | Replica Count
----------------------------------+--------------------------------+---------------
09d6bf7a02124145b43f43cb7a667b3d | vc1314.halxg.cloudera.com:7050 | 100
23d473f441674d43807fd9e631862bfd | vc1308.halxg.cloudera.com:7050 | 100
2fb5cdac22b0418bb2df456906e42eb4 | vc1306.halxg.cloudera.com:7050 | 101
70f7ee61ead54b1885d819f354eb3405 | vc1316.halxg.cloudera.com:7050 | 95
72fcec63e96f4248ae39d114eb3cd7c9 | vc1318.halxg.cloudera.com:7050 | 94
86708813b37a44bd8e92c711211c8685 | vc1310.halxg.cloudera.com:7050 | 96
a662440710624c02bd5612df32cb0235 | vc1302.halxg.cloudera.com:7050 | 101
c9633273962a4521a32d5e177a118a84 | vc1312.halxg.cloudera.com:7050 | 101
cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 76
I also tested it against an empty cluster.
There's no unit tests added, just because our current testing setup for
ksck makes it really painful to add one for this, and it seemed easy
enough to check out manually. Probably, a follow up should straighten
out ksck-test to make testing ksck changes easier.
Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Reviewed-on: http://gerrit.cloudera.org:8080/11958
Reviewed-by: Andrew Wong <aw...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/tools/ksck_results.cc
M src/kudu/tools/ksck_results.h
2 files changed, 131 insertions(+), 0 deletions(-)
Approvals:
Andrew Wong: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/11958
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Gerrit-Change-Number: 11958
Gerrit-PatchSet: 5
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Fengling Wang <fw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mitch Barnett <mb...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [tools] ksck: Add information about replica counts to plain ksck output
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Fengling Wang, Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, Mitch Barnett,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11958
to look at the new patch set (#4).
Change subject: [tools] ksck: Add information about replica counts to plain ksck output
......................................................................
[tools] ksck: Add information about replica counts to plain ksck output
This adds some information about replica counts on tablet servers to the
output of ksck when ksck is in PLAIN_* mode (i.e. not JSON output). It
outputs a 5-number summary of the distribution of replicas and lists any
outliers:
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1646
First Quartile | 3672
Median | 4075
Third Quartile | 4242
Maximum | 4600
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------------------+---------------
Small | cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 1646
In PLAIN_FULL mode it additionally outputs the replica count for every
tablet server:
Tablet Replica Count by Tablet Server
UUID | Host | Replica Count
----------------------------------+--------------------------------+---------------
09d6bf7a02124145b43f43cb7a667b3d | vc1314.halxg.cloudera.com:7050 | 100
23d473f441674d43807fd9e631862bfd | vc1308.halxg.cloudera.com:7050 | 100
2fb5cdac22b0418bb2df456906e42eb4 | vc1306.halxg.cloudera.com:7050 | 101
70f7ee61ead54b1885d819f354eb3405 | vc1316.halxg.cloudera.com:7050 | 95
72fcec63e96f4248ae39d114eb3cd7c9 | vc1318.halxg.cloudera.com:7050 | 94
86708813b37a44bd8e92c711211c8685 | vc1310.halxg.cloudera.com:7050 | 96
a662440710624c02bd5612df32cb0235 | vc1302.halxg.cloudera.com:7050 | 101
c9633273962a4521a32d5e177a118a84 | vc1312.halxg.cloudera.com:7050 | 101
cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 76
I also tested it against an empty cluster.
There's no unit tests added, just because our current testing setup for
ksck makes it really painful to add one for this, and it seemed easy
enough to check out manually. Probably, a follow up should straighten
out ksck-test to make testing ksck changes easier.
Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
---
M src/kudu/tools/ksck_results.cc
M src/kudu/tools/ksck_results.h
2 files changed, 131 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/11958/4
--
To view, visit http://gerrit.cloudera.org:8080/11958
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Gerrit-Change-Number: 11958
Gerrit-PatchSet: 4
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Fengling Wang <fw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mitch Barnett <mb...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [tools] ksck: Add information about replica counts to plain ksck output
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Fengling Wang, Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, Mitch Barnett,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11958
to look at the new patch set (#2).
Change subject: [tools] ksck: Add information about replica counts to plain ksck output
......................................................................
[tools] ksck: Add information about replica counts to plain ksck output
This adds some information about replica counts on tablet servers to the
output of ksck when ksck is in PLAIN_* mode (i.e. not JSON output). It
outputs a 5-number summary of the distribution of replicas and lists any
outliers:
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1646
First Quartile | 3672
Median | 4075
Third Quartile | 4242
Maximum | 4600
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------------------+---------------
Small | cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 1646
In PLAIN_FULL mode it additionally outputs the replica count for every
tablet server:
Tablet Replica Count by Tablet Server
UUID | Host | Replica Count
----------------------------------+--------------------------------+---------------
09d6bf7a02124145b43f43cb7a667b3d | vc1314.halxg.cloudera.com:7050 | 100
23d473f441674d43807fd9e631862bfd | vc1308.halxg.cloudera.com:7050 | 100
2fb5cdac22b0418bb2df456906e42eb4 | vc1306.halxg.cloudera.com:7050 | 101
70f7ee61ead54b1885d819f354eb3405 | vc1316.halxg.cloudera.com:7050 | 95
72fcec63e96f4248ae39d114eb3cd7c9 | vc1318.halxg.cloudera.com:7050 | 94
86708813b37a44bd8e92c711211c8685 | vc1310.halxg.cloudera.com:7050 | 96
a662440710624c02bd5612df32cb0235 | vc1302.halxg.cloudera.com:7050 | 101
c9633273962a4521a32d5e177a118a84 | vc1312.halxg.cloudera.com:7050 | 101
cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 76
I also tested it against an empty cluster.
There's no unit tests added, just because our current testing setup for
ksck makes it really painful to add one for this, and it seemed easy
enough to check out manually. Probably, a follow up should straighten
out ksck-test to make testing ksck changes easier.
Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
---
M src/kudu/tools/ksck_results.cc
M src/kudu/tools/ksck_results.h
2 files changed, 130 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/11958/2
--
To view, visit http://gerrit.cloudera.org:8080/11958
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Gerrit-Change-Number: 11958
Gerrit-PatchSet: 2
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Fengling Wang <fw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mitch Barnett <mb...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [tools] ksck: Add information about replica counts to plain ksck output
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11958 )
Change subject: [tools] ksck: Add information about replica counts to plain ksck output
......................................................................
Patch Set 1:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/11958/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11958/1//COMMIT_MSG@11
PS1, Line 11: it outputs a 5-number summary of the distribution of
: replicas and lists any outliers
> Maybe your answer to this lies in what you've implemented, but do you think
Yeah, why not? It's the verbose output.
http://gerrit.cloudera.org:8080/#/c/11958/1//COMMIT_MSG@26
PS1, Line 26: Small
> Hrm, I might've missed this. What is this referring to? Also, how are you q
It should be pretty clear what it means, I think. The reason I even include it is thatthe table will be sorted Small before Big and then within Small sorted by most to least replicas and within Big by least to most, so having the Small/Big label helps make the order understandable.
http://gerrit.cloudera.org:8080/#/c/11958/1//COMMIT_MSG@28
PS1, Line 28: In PLAIN_FULL mode it outputs the replica count for every tablet server:
:
: Tablet Replica Count by Tablet Server
: UUID | Host | Replica Count
: ----------------------------------+--------------------------------+---------------
: 09d6bf7a02124145b43f43cb7a667b3d | vc1314.halxg.cloudera.com:7050 | 100
: 23d473f441674d43807fd9e631862bfd | vc1308.halxg.cloudera.com:7050 | 100
: 2fb5cdac22b0418bb2df456906e42eb4 | vc1306.halxg.cloudera.com:7050 | 101
: 70f7ee61ead54b1885d819f354eb3405 | vc1316.halxg.cloudera.com:7050 | 95
: 72fcec63e96f4248ae39d114eb3cd7c9 | vc1318.halxg.cloudera.com:7050 | 94
: 86708813b37a44bd8e92c711211c8685 | vc1310.halxg.cloudera.com:7050 | 96
: a662440710624c02bd5612df32cb0235 | vc1302.halxg.cloudera.com:7050 | 101
: c9633273962a4521a32d5e177a118a84 | vc1312.halxg.cloudera.com:7050 | 101
: cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 76
:
: I also tested it against an empty cluster.
> Did you test it with the -tables, -tablets configurations? From the impleme
Yes. Actually this output comes from using the -tables.
http://gerrit.cloudera.org:8080/#/c/11958/1/src/kudu/tools/ksck_results.cc
File src/kudu/tools/ksck_results.cc:
http://gerrit.cloudera.org:8080/#/c/11958/1/src/kudu/tools/ksck_results.cc@576
PS1, Line 576: tservers_sorted_by_replica_count.emplace_back(entry.first,
> warning: 'emplace_back' is called inside a loop; consider pre-allocating th
Done
--
To view, visit http://gerrit.cloudera.org:8080/11958
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Gerrit-Change-Number: 11958
Gerrit-PatchSet: 1
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Fengling Wang <fw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mitch Barnett <mb...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Tue, 20 Nov 2018 07:22:46 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] ksck: Add information about replica counts to plain ksck output
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11958 )
Change subject: [tools] ksck: Add information about replica counts to plain ksck output
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11958/3/src/kudu/tools/ksck_results.h
File src/kudu/tools/ksck_results.h:
http://gerrit.cloudera.org:8080/#/c/11958/3/src/kudu/tools/ksck_results.h@358
PS3, Line 358: in the latter (default)
: // case, only a quartile summary of the counts will be printed.
> nit: maybe generalize this to just mention that it prints a summary of the
Done
--
To view, visit http://gerrit.cloudera.org:8080/11958
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Gerrit-Change-Number: 11958
Gerrit-PatchSet: 3
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Fengling Wang <fw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mitch Barnett <mb...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Tue, 20 Nov 2018 08:50:45 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] ksck: Add information about replica counts to plain ksck output
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11958 )
Change subject: [tools] ksck: Add information about replica counts to plain ksck output
......................................................................
Patch Set 1:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/11958/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/11958/1//COMMIT_MSG@11
PS1, Line 11: it outputs a 5-number summary of the distribution of
: replicas and lists any outliers
Maybe your answer to this lies in what you've implemented, but do you think this is useful to include in PLAIN_FULL?
http://gerrit.cloudera.org:8080/#/c/11958/1//COMMIT_MSG@26
PS1, Line 26: Small
Hrm, I might've missed this. What is this referring to? Also, how are you quantifying what an outlier is?
Ah, from the patch it's clear, but mind also noting this in the commit message?
Upon googling things, I guess the answer here is obvious...
http://gerrit.cloudera.org:8080/#/c/11958/1//COMMIT_MSG@28
PS1, Line 28: In PLAIN_FULL mode it outputs the replica count for every tablet server:
:
: Tablet Replica Count by Tablet Server
: UUID | Host | Replica Count
: ----------------------------------+--------------------------------+---------------
: 09d6bf7a02124145b43f43cb7a667b3d | vc1314.halxg.cloudera.com:7050 | 100
: 23d473f441674d43807fd9e631862bfd | vc1308.halxg.cloudera.com:7050 | 100
: 2fb5cdac22b0418bb2df456906e42eb4 | vc1306.halxg.cloudera.com:7050 | 101
: 70f7ee61ead54b1885d819f354eb3405 | vc1316.halxg.cloudera.com:7050 | 95
: 72fcec63e96f4248ae39d114eb3cd7c9 | vc1318.halxg.cloudera.com:7050 | 94
: 86708813b37a44bd8e92c711211c8685 | vc1310.halxg.cloudera.com:7050 | 96
: a662440710624c02bd5612df32cb0235 | vc1302.halxg.cloudera.com:7050 | 101
: c9633273962a4521a32d5e177a118a84 | vc1312.halxg.cloudera.com:7050 | 101
: cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 76
:
: I also tested it against an empty cluster.
Did you test it with the -tables, -tablets configurations? From the implementation, it seems like this should be handled, but it's probably worth a look that it looks ok.
--
To view, visit http://gerrit.cloudera.org:8080/11958
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Gerrit-Change-Number: 11958
Gerrit-PatchSet: 1
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Fengling Wang <fw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mitch Barnett <mb...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 20 Nov 2018 02:01:55 +0000
Gerrit-HasComments: Yes
[kudu-CR] [tools] ksck: Add information about replica counts to plain ksck output
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11958 )
Change subject: [tools] ksck: Add information about replica counts to plain ksck output
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/11958
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Gerrit-Change-Number: 11958
Gerrit-PatchSet: 4
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Fengling Wang <fw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mitch Barnett <mb...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Tue, 20 Nov 2018 08:52:17 +0000
Gerrit-HasComments: No
[kudu-CR] [tools] ksck: Add information about replica counts to plain ksck output
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Fengling Wang, Tidy Bot, Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, Mitch Barnett,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11958
to look at the new patch set (#3).
Change subject: [tools] ksck: Add information about replica counts to plain ksck output
......................................................................
[tools] ksck: Add information about replica counts to plain ksck output
This adds some information about replica counts on tablet servers to the
output of ksck when ksck is in PLAIN_* mode (i.e. not JSON output). It
outputs a 5-number summary of the distribution of replicas and lists any
outliers:
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1646
First Quartile | 3672
Median | 4075
Third Quartile | 4242
Maximum | 4600
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+--------------------------------+---------------
Small | cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 1646
In PLAIN_FULL mode it additionally outputs the replica count for every
tablet server:
Tablet Replica Count by Tablet Server
UUID | Host | Replica Count
----------------------------------+--------------------------------+---------------
09d6bf7a02124145b43f43cb7a667b3d | vc1314.halxg.cloudera.com:7050 | 100
23d473f441674d43807fd9e631862bfd | vc1308.halxg.cloudera.com:7050 | 100
2fb5cdac22b0418bb2df456906e42eb4 | vc1306.halxg.cloudera.com:7050 | 101
70f7ee61ead54b1885d819f354eb3405 | vc1316.halxg.cloudera.com:7050 | 95
72fcec63e96f4248ae39d114eb3cd7c9 | vc1318.halxg.cloudera.com:7050 | 94
86708813b37a44bd8e92c711211c8685 | vc1310.halxg.cloudera.com:7050 | 96
a662440710624c02bd5612df32cb0235 | vc1302.halxg.cloudera.com:7050 | 101
c9633273962a4521a32d5e177a118a84 | vc1312.halxg.cloudera.com:7050 | 101
cc32936bc8594948a04fd4240da36aed | vc1304.halxg.cloudera.com:7050 | 76
I also tested it against an empty cluster.
There's no unit tests added, just because our current testing setup for
ksck makes it really painful to add one for this, and it seemed easy
enough to check out manually. Probably, a follow up should straighten
out ksck-test to make testing ksck changes easier.
Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
---
M src/kudu/tools/ksck_results.cc
M src/kudu/tools/ksck_results.h
2 files changed, 131 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/11958/3
--
To view, visit http://gerrit.cloudera.org:8080/11958
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Gerrit-Change-Number: 11958
Gerrit-PatchSet: 3
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Fengling Wang <fw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mitch Barnett <mb...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [tools] ksck: Add information about replica counts to plain ksck output
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11958 )
Change subject: [tools] ksck: Add information about replica counts to plain ksck output
......................................................................
Patch Set 3: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11958/3/src/kudu/tools/ksck_results.h
File src/kudu/tools/ksck_results.h:
http://gerrit.cloudera.org:8080/#/c/11958/3/src/kudu/tools/ksck_results.h@358
PS3, Line 358: in the latter (default)
: // case, only a quartile summary of the counts will be printed.
nit: maybe generalize this to just mention that it prints a summary of the replica counts, since the quartiles are somewhat an implementation detail, and this also doesn't mention outliers. Also the fact that PLAIN_CONCISE is the default probably doesn't need to be here (and maybe shouldn't, in case that changes?)
--
To view, visit http://gerrit.cloudera.org:8080/11958
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7e5373033ab84c1e34f9519eb9bd4e04a652c595
Gerrit-Change-Number: 11958
Gerrit-PatchSet: 3
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Fengling Wang <fw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mitch Barnett <mb...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Tue, 20 Nov 2018 07:44:13 +0000
Gerrit-HasComments: Yes