You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2018/10/20 06:55:15 UTC
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11745
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
[rebalancer] location-aware rebalancer (part 4/n)
Refactored Rebalancer and Runner classes, separating common base for
a runner of the rebalancing process.
Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
---
M src/kudu/tools/rebalance_algo.cc
M src/kudu/tools/rebalancer.cc
M src/kudu/tools/rebalancer.h
3 files changed, 483 insertions(+), 377 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/45/11745/1
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11745 )
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
[rebalancer] location-aware rebalancer (part 4/n)
Refactored Rebalancer and Runner classes, separating common base for
a runner of the rebalancing process.
Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Reviewed-on: http://gerrit.cloudera.org:8080/11745
Tested-by: Alexey Serbin <as...@cloudera.com>
Reviewed-by: Will Berkeley <wd...@gmail.com>
---
M src/kudu/tools/rebalancer.cc
M src/kudu/tools/rebalancer.h
2 files changed, 487 insertions(+), 374 deletions(-)
Approvals:
Alexey Serbin: Verified
Will Berkeley: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11745 )
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Mon, 29 Oct 2018 23:01:36 +0000
Gerrit-HasComments: No
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11745 )
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
Patch Set 1:
(3 comments)
Thanks for splitting the patches up. It's much easier to review in bites.
http://gerrit.cloudera.org:8080/#/c/11745/1/src/kudu/tools/rebalancer.h
File src/kudu/tools/rebalancer.h:
http://gerrit.cloudera.org:8080/#/c/11745/1/src/kudu/tools/rebalancer.h@322
PS1, Line 322: std::vector<std::string>* tablet_ids)
These are ts UUIDs, i.e. replica UUIDs, right?
http://gerrit.cloudera.org:8080/#/c/11745/1/src/kudu/tools/rebalancer.h@324
PS1, Line 324: // Filter out move operations at the tablets which already have operations
: // in progress. The 'replica_moves' cannot be null.
I'm confused. What does this do, and what does it return in the out param? It populates 'replica_moves' with the moves in 'scheduled_moves'?
http://gerrit.cloudera.org:8080/#/c/11745/1/src/kudu/tools/rebalancer.cc
File src/kudu/tools/rebalancer.cc:
http://gerrit.cloudera.org:8080/#/c/11745/1/src/kudu/tools/rebalancer.cc@596
PS1, Line 596:
Extra line.
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Mon, 22 Oct 2018 21:25:26 +0000
Gerrit-HasComments: Yes
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Will Berkeley, Tidy Bot, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11745
to look at the new patch set (#2).
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
[rebalancer] location-aware rebalancer (part 4/n)
Refactored Rebalancer and Runner classes, separating common base for
a runner of the rebalancing process.
Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
---
M src/kudu/tools/rebalancer.cc
M src/kudu/tools/rebalancer.h
2 files changed, 487 insertions(+), 374 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/45/11745/2
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/11745 )
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
Removed reviewer Kudu Jenkins with the following votes:
* Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11745 )
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
Patch Set 2: Verified+1
Unrelated flake in org.apache.kudu.spark.kudu.DefaultSourceTest
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Tue, 23 Oct 2018 18:26:22 +0000
Gerrit-HasComments: No
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change. ( http://gerrit.cloudera.org:8080/11745 )
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Mon, 29 Oct 2018 17:05:17 +0000
Gerrit-HasComments: No
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/11745 )
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
Removed reviewer Kudu Jenkins with the following votes:
* Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11745 )
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/11745/1/src/kudu/tools/rebalancer.h
File src/kudu/tools/rebalancer.h:
http://gerrit.cloudera.org:8080/#/c/11745/1/src/kudu/tools/rebalancer.h@322
PS1, Line 322: ty with the result status of Status::
> These are ts UUIDs, i.e. replica UUIDs, right?
Nope, these are tablet UUIDs. The source and the destination replicas are determined by tserver UUIDs: TableReplicaMove::from and TableReplicaMove::to. I updated the comment.
http://gerrit.cloudera.org:8080/#/c/11745/1/src/kudu/tools/rebalancer.h@324
PS1, Line 324: const ClusterRawInfo& raw_info,
: std::vector<std::string>
> I'm confused. What does this do, and what does it return in the out param?
It filters move operations in 'replica_moves', removing all operations that would involve moving replicas of tablets which are in 'scheduled_moves'.
I updated the comment.
http://gerrit.cloudera.org:8080/#/c/11745/1/src/kudu/tools/rebalancer.cc
File src/kudu/tools/rebalancer.cc:
http://gerrit.cloudera.org:8080/#/c/11745/1/src/kudu/tools/rebalancer.cc@596
PS1, Line 596: Rebalancer::BaseRunner::BaseRunner(Rebalancer* rebalancer,
> Extra line.
Done
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Tue, 23 Oct 2018 05:26:32 +0000
Gerrit-HasComments: Yes
[kudu-CR] [rebalancer] location-aware rebalancer (part 4/n)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/11745 )
Change subject: [rebalancer] location-aware rebalancer (part 4/n)
......................................................................
Patch Set 3: Verified+1
Unrelated flake in DeleteTableITest.TestNoDeleteTombstonedTablets
--
To view, visit http://gerrit.cloudera.org:8080/11745
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id47183fc853573390b22ec714751adec93e0ea3a
Gerrit-Change-Number: 11745
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-Comment-Date: Mon, 29 Oct 2018 22:50:16 +0000
Gerrit-HasComments: No