You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Yifan Zhang (Code Review)" <ge...@cloudera.org> on 2019/11/19 12:05:16 UTC
[kudu-CR] rebalancer: fix on Runner::UpdateMovesInProgressStatus()
Yifan Zhang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14742
Change subject: rebalancer: fix on Runner::UpdateMovesInProgressStatus()
......................................................................
rebalancer: fix on Runner::UpdateMovesInProgressStatus()
This patch added a new parameter 'has_pending_moves' for
Runner::UpdateMovesInProgressStatus(), see the comments for
this method.
Before this patch, the return value of this method was ambiguous,
and there was a bug in PolicyFixer: When no pending operation
completed, the rebalancer tool would re-synchronize the state of
cluster. Now, the re-synchronization of the cluster state happens
only in case of an error or timeout or no pending movement.
Change-Id: I8a25574d13f36a8f4f98f4340aa2086711b1c741
---
M src/kudu/rebalance/rebalancer.h
M src/kudu/tools/rebalancer_tool.cc
M src/kudu/tools/rebalancer_tool.h
3 files changed, 39 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/42/14742/1
--
To view, visit http://gerrit.cloudera.org:8080/14742
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8a25574d13f36a8f4f98f4340aa2086711b1c741
Gerrit-Change-Number: 14742
Gerrit-PatchSet: 1
Gerrit-Owner: Yifan Zhang <ch...@163.com>
[kudu-CR] rebalancer: fix on Runner::UpdateMovesInProgressStatus()
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/14742 )
Change subject: rebalancer: fix on Runner::UpdateMovesInProgressStatus()
......................................................................
Patch Set 1:
(6 comments)
Thank you for fixing the bug!
The patch looks good, just a few nits.
http://gerrit.cloudera.org:8080/#/c/14742/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14742/1//COMMIT_MSG@14
PS1, Line 14: When no pending operation
: completed, the rebalancer tool would re-synchronize the state of
: cluster
nit: I don't think this is what was happening. I think in that case the rebalancer would try to get information on the next move from the algorithm and perform it. Direct re-synchronization of the cluster state happened only if case of error. At least, that's what I can see from the existing code in https://github.com/apache/kudu/blob/master/src/kudu/tools/rebalancer_tool.cc#L499
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/rebalance/rebalancer.h
File src/kudu/rebalance/rebalancer.h:
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/rebalance/rebalancer.h@180
PS1, Line 180: its
their
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/rebalance/rebalancer.h@181
PS1, Line 181: operations were
operation was
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/rebalance/rebalancer.h@181
PS1, Line 181: some
nit: at least one
This way it's easier to comprehend.
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/tools/rebalancer_tool.cc
File src/kudu/tools/rebalancer_tool.cc:
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/tools/rebalancer_tool.cc@529
PS1, Line 529: auto has_pending_moves = false;
Maybe, move this variable into the scope of the 'while' loop below, closer to the place of its usage? It's not referenced anywhere else outside.
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/tools/rebalancer_tool.cc@566
PS1, Line 566: Or there was no pending move left, a new set of
: // moves is also required
nit: Also, do the same if not a single pending move has left.
--
To view, visit http://gerrit.cloudera.org:8080/14742
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8a25574d13f36a8f4f98f4340aa2086711b1c741
Gerrit-Change-Number: 14742
Gerrit-PatchSet: 1
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 20 Nov 2019 00:03:37 +0000
Gerrit-HasComments: Yes
[kudu-CR] rebalancer: fix on Runner::UpdateMovesInProgressStatus()
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14742 )
Change subject: rebalancer: fix on Runner::UpdateMovesInProgressStatus()
......................................................................
rebalancer: fix on Runner::UpdateMovesInProgressStatus()
This patch added a new parameter 'has_pending_moves' for
Runner::UpdateMovesInProgressStatus(), see the comments for
this method.
Before this patch, the return value of this method was ambiguous,
and there was a bug in PolicyFixer: When no pending operation
completed, the rebalancer tool would try to get a new set of moves
from the algorithm and perform it. Now, the rebalancer would do this
only in case of an error or no pending movement.
Change-Id: I8a25574d13f36a8f4f98f4340aa2086711b1c741
Reviewed-on: http://gerrit.cloudera.org:8080/14742
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/rebalance/rebalancer.h
M src/kudu/tools/rebalancer_tool.cc
M src/kudu/tools/rebalancer_tool.h
3 files changed, 36 insertions(+), 20 deletions(-)
Approvals:
Alexey Serbin: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/14742
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I8a25574d13f36a8f4f98f4340aa2086711b1c741
Gerrit-Change-Number: 14742
Gerrit-PatchSet: 3
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
[kudu-CR] rebalancer: fix on Runner::UpdateMovesInProgressStatus()
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Yifan Zhang has posted comments on this change. ( http://gerrit.cloudera.org:8080/14742 )
Change subject: rebalancer: fix on Runner::UpdateMovesInProgressStatus()
......................................................................
Patch Set 2:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/14742/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/14742/1//COMMIT_MSG@14
PS1, Line 14: When no pending operation
: completed, the rebalancer tool would try to get a new set of moves
: from th
> nit: I don't think this is what was happening. I think in that case the re
Done
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/rebalance/rebalancer.h
File src/kudu/rebalance/rebalancer.h:
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/rebalance/rebalancer.h@180
PS1, Line 180: the
> their
Done
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/rebalance/rebalancer.h@181
PS1, Line 181: at l
> nit: at least one
Done
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/rebalance/rebalancer.h@181
PS1, Line 181: ss move operati
> operation was
Done
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/tools/rebalancer_tool.cc
File src/kudu/tools/rebalancer_tool.cc:
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/tools/rebalancer_tool.cc@529
PS1, Line 529: while (!is_timed_out) {
> Maybe, move this variable into the scope of the 'while' loop below, closer
Done
http://gerrit.cloudera.org:8080/#/c/14742/1/src/kudu/tools/rebalancer_tool.cc@566
PS1, Line 566: Also, do the same if not a single pending move has left.
: break;
> nit: Also, do the same if not a single pending move has left.
Done
--
To view, visit http://gerrit.cloudera.org:8080/14742
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8a25574d13f36a8f4f98f4340aa2086711b1c741
Gerrit-Change-Number: 14742
Gerrit-PatchSet: 2
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 20 Nov 2019 03:36:18 +0000
Gerrit-HasComments: Yes
[kudu-CR] rebalancer: fix on Runner::UpdateMovesInProgressStatus()
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/14742 )
Change subject: rebalancer: fix on Runner::UpdateMovesInProgressStatus()
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/14742
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8a25574d13f36a8f4f98f4340aa2086711b1c741
Gerrit-Change-Number: 14742
Gerrit-PatchSet: 2
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yifan Zhang <ch...@163.com>
Gerrit-Comment-Date: Wed, 20 Nov 2019 03:43:14 +0000
Gerrit-HasComments: No
[kudu-CR] rebalancer: fix on Runner::UpdateMovesInProgressStatus()
Posted by "Yifan Zhang (Code Review)" <ge...@cloudera.org>.
Hello Alexey Serbin, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/14742
to look at the new patch set (#2).
Change subject: rebalancer: fix on Runner::UpdateMovesInProgressStatus()
......................................................................
rebalancer: fix on Runner::UpdateMovesInProgressStatus()
This patch added a new parameter 'has_pending_moves' for
Runner::UpdateMovesInProgressStatus(), see the comments for
this method.
Before this patch, the return value of this method was ambiguous,
and there was a bug in PolicyFixer: When no pending operation
completed, the rebalancer tool would try to get a new set of moves
from the algorithm and perform it. Now, the rebalancer would do this
only in case of an error or no pending movement.
Change-Id: I8a25574d13f36a8f4f98f4340aa2086711b1c741
---
M src/kudu/rebalance/rebalancer.h
M src/kudu/tools/rebalancer_tool.cc
M src/kudu/tools/rebalancer_tool.h
3 files changed, 36 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/42/14742/2
--
To view, visit http://gerrit.cloudera.org:8080/14742
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8a25574d13f36a8f4f98f4340aa2086711b1c741
Gerrit-Change-Number: 14742
Gerrit-PatchSet: 2
Gerrit-Owner: Yifan Zhang <ch...@163.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)