You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Yifan Zhang (Code Review)" <ge...@cloudera.org> on 2021/12/27 09:28:49 UTC
[kudu-CR] KUDU-3346: fix rebalancer tool fails to run with '--ignored tservers'
Yifan Zhang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18114
Change subject: KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
......................................................................
KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
Before this patch, the validity of 'ignored_tservers' was checked in
'BuildClusterinfo', which leads to a failure when the 'raw_info' only
contains information of tservers on a specific location.
This patch fix it by moving the parameter validity check into
'KsckResultsToClusterRawInfo' because ksck results always contains all
tservers in the cluster. And also, I add 'needs_tservers_to_empty_info'
to 'BuildClusterInfo', we only need this information when:
- print the state of whole cluster before and after the rebalancer runs.
- get the state of whole cluster during running IgnoredTServersRunner.
In these two cases 'raw_info' contains all tablet servers info in the
cluster so every ignored tservers could be found in 'raw_info'.
Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
---
M src/kudu/rebalance/rebalance-test.cc
M src/kudu/rebalance/rebalancer.cc
M src/kudu/rebalance/rebalancer.h
M src/kudu/tools/rebalancer_tool-test.cc
M src/kudu/tools/rebalancer_tool.cc
M src/kudu/tools/rebalancer_tool.h
6 files changed, 189 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/14/18114/1
--
To view, visit http://gerrit.cloudera.org:8080/18114
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
Gerrit-Change-Number: 18114
Gerrit-PatchSet: 1
Gerrit-Owner: Yifan Zhang <ch...@163.com>
[kudu-CR] KUDU-3346: fix rebalancer tool fails to run with '--ignored tservers'
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18114 )
Change subject: KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
......................................................................
KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
Prior to this patch the validity of 'ignored_tservers' was checked when
'BuildClusterinfo', which leads to a failure when the 'raw_info' only contains
contains information of tservers on a specific location. This patch fix it by
moving the parameter validity check into 'KsckResultsToClusterRawInfo', because
ksck results contain original cluster information.
I noticed 'ClusterInfo::tservers_to_empty' is not necessary to be built when
'BuildClusterInfo', because we use this info only for printing cluster's stats
and running IgnoredTserverRunner. This should be refactored in follow-up patch.
This patch adds a regression test for the issue and I also verified this fix on
a real cluster.
Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
Reviewed-on: http://gerrit.cloudera.org:8080/18114
Reviewed-by: Andrew Wong <aw...@cloudera.com>
Tested-by: Andrew Wong <aw...@cloudera.com>
---
M src/kudu/rebalance/rebalancer.cc
M src/kudu/tools/rebalancer_tool-test.cc
M src/kudu/tools/rebalancer_tool.cc
M src/kudu/tools/rebalancer_tool.h
4 files changed, 151 insertions(+), 46 deletions(-)
Approvals:
Andrew Wong: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/18114
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
Gerrit-Change-Number: 18114
Gerrit-PatchSet: 4
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
[kudu-CR] KUDU-3346: fix rebalancer tool fails to run with '--ignored tservers'
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has removed a vote on this change.
Change subject: KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
......................................................................
Removed -Verified by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/18114
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
Gerrit-Change-Number: 18114
Gerrit-PatchSet: 3
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-3346: fix rebalancer tool fails to run with '--ignored tservers'
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has removed a vote on this change.
Change subject: KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/18114
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
Gerrit-Change-Number: 18114
Gerrit-PatchSet: 3
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-3346: fix rebalancer tool fails to run with '--ignored tservers'
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Andrew Wong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18114
to look at the new patch set (#3).
Change subject: KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
......................................................................
KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
Prior to this patch the validity of 'ignored_tservers' was checked when
'BuildClusterinfo', which leads to a failure when the 'raw_info' only contains
contains information of tservers on a specific location. This patch fix it by
moving the parameter validity check into 'KsckResultsToClusterRawInfo', because
ksck results contain original cluster information.
I noticed 'ClusterInfo::tservers_to_empty' is not necessary to be built when
'BuildClusterInfo', because we use this info only for printing cluster's stats
and running IgnoredTserverRunner. This should be refactored in follow-up patch.
This patch adds a regression test for the issue and I also verified this fix on
a real cluster.
Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
---
M src/kudu/rebalance/rebalancer.cc
M src/kudu/tools/rebalancer_tool-test.cc
M src/kudu/tools/rebalancer_tool.cc
M src/kudu/tools/rebalancer_tool.h
4 files changed, 151 insertions(+), 46 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/14/18114/3
--
To view, visit http://gerrit.cloudera.org:8080/18114
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
Gerrit-Change-Number: 18114
Gerrit-PatchSet: 3
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-3346: fix rebalancer tool fails to run with '--ignored tservers'
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Andrew Wong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18114
to look at the new patch set (#2).
Change subject: KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
......................................................................
KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
Before this patch, the validity of 'ignored_tservers' was checked in
'BuildClusterinfo', which leads to a failure when the 'raw_info' only contains
contains information of tservers on a specific location. This patch fix it by
moving the parameter validity check into 'KsckResultsToClusterRawInfo', because
ksck results always contains all tservers' information in the cluster.
Since 'ClusterInfo::tservers_to_empty' is only for printing ignored tservers'
stats now, I add 'needs_tservers_to_empty_info' to 'BuildClusterInfo',
which is false by default.
I test this fix on a real cluster and also add a regression test.
Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
---
M src/kudu/rebalance/rebalance-test.cc
M src/kudu/rebalance/rebalance_algo.h
M src/kudu/rebalance/rebalancer.cc
M src/kudu/rebalance/rebalancer.h
M src/kudu/tools/rebalancer_tool-test.cc
M src/kudu/tools/rebalancer_tool.cc
M src/kudu/tools/rebalancer_tool.h
7 files changed, 188 insertions(+), 53 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/14/18114/2
--
To view, visit http://gerrit.cloudera.org:8080/18114
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
Gerrit-Change-Number: 18114
Gerrit-PatchSet: 2
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] KUDU-3346: fix rebalancer tool fails to run with '--ignored tservers'
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/18114 )
Change subject: KUDU-3346: fix rebalancer tool fails to run with '--ignored_tservers'
......................................................................
Patch Set 3: Verified+1 Code-Review+2
Thanks for fixing this!
--
To view, visit http://gerrit.cloudera.org:8080/18114
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1361f562f3e886077a79c3de8ea5fb2ebb8df6e9
Gerrit-Change-Number: 18114
Gerrit-PatchSet: 3
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 04 Jan 2022 18:58:53 +0000
Gerrit-HasComments: No