You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2017/09/28 02:38:21 UTC
[kudu-CR] [catalog manager] introduce replica selector
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/8161
Change subject: [catalog manager] introduce replica selector
......................................................................
[catalog manager] introduce replica selector
Introduced replica selector for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The selector will be used for listing tablet replicas once NON_VOTER
replicas are introduced in a follow-up commit.
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
---
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
9 files changed, 113 insertions(+), 47 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/8161/1
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
[kudu-CR] [catalog manager] introduce replica type matching policy
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica type matching policy
......................................................................
Patch Set 7:
(7 comments)
overall lgtm
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/catalog_manager.h
File src/kudu/master/catalog_manager.h:
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/catalog_manager.h@722
PS7, Line 722: include_non_voters
flag name changed
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/catalog_manager.h@723
PS7, Line 723: inconsistency
unclear to me what this means
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/catalog_manager.h@727
PS7, Line 727: sel
how about naming this 'filter' ?
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master.proto
File src/kudu/master/master.proto:
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master.proto@366
PS7, Line 366: ReplicaMatchPolicy
How about ReplicaTypeFilter
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master.proto@378
PS7, Line 378: match_policy
replica_type_filter ?
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master.proto@463
PS7, Line 463: match_policy
same
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master_service.cc
File src/kudu/master/master_service.cc:
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master_service.cc@241
PS7, Line 241: unknown
todd (per https://github.com/apache/kudu/blame/master/src/kudu/master/master_service.cc#L241 )
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 7
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Wed, 29 Nov 2017 03:15:22 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] introduce replica selector
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8161
to look at the new patch set (#3).
Change subject: [catalog manager] introduce replica selector
......................................................................
[catalog manager] introduce replica selector
Introduced replica selector for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The selector is used to specify replica membership matching policy
while populating the result of the GetTabletLocations RPC.
As of now, only VOTER_REPLICA policy is used for all GetTabletLocations
invocations and only VOTER replicas are returned in the result.
As soon as NON_VOTER replicas appear in a follow-up commit, it will
be possible to use ANY_REPLICA policy with GetTabletLocation request
to receive information on NON_VOTER replicas as well.
The 'regular' client operations should work with voter replicas only.
The use case for getting all replicas is when the kudu CLI tool fetches
that information to report on replica membership status for listings
in 'table list --tablets' and 'ksck' sub-commands.
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
---
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
10 files changed, 164 insertions(+), 49 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/8161/3
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] [catalog manager] introduce replica type filter
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica type filter
......................................................................
[catalog manager] introduce replica type filter
Introduced a replica type filter for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The filter is used to specify what type of tablet replicas to include
while populating the result of the GetTabletLocations RPC.
When not specified otherwise or when using earlier versions of the
GetTable[t]LocationsRequestPB protobuf messages, VOTER_REPLICA policy
is used. To locate all replicas, specify ANY_REPLICA policy in the
'replica_type_filter' field of the GetTable[t]LocationsRequestPB. The
latter is done now unconditionally for already existing non-voter
replica-related tests. This will change by a follow-up commit:
the 'regular' client operations should work with voter replicas only,
so the use cases for getting all tablet replicas will be just
the following:
* the kudu CLI tool fetching that information to report on replica
membership status for listings in 'table list --tablets' and
'ksck' sub-commands.
* various tests exercising the functionality related to
non-voter tablet replicas
Added an integration test to verify the functionality of the replica
type filter for GetTableLocations and GetTabletLocations RPC.
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Reviewed-on: http://gerrit.cloudera.org:8080/8161
Tested-by: Alexey Serbin <as...@cloudera.com>
Reviewed-by: Mike Percy <mp...@apache.org>
---
M src/kudu/client/meta_cache.cc
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/delete_table-itest.cc
M src/kudu/integration-tests/flex_partitioning-itest.cc
M src/kudu/integration-tests/raft_config_change-itest.cc
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/integration-tests/tablet_copy-itest.cc
M src/kudu/integration-tests/tombstoned_voting-itest.cc
M src/kudu/integration-tests/ts_itest-base.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
18 files changed, 242 insertions(+), 64 deletions(-)
Approvals:
Alexey Serbin: Verified
Mike Percy: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 11
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] [catalog manager] introduce replica selector
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica selector
......................................................................
Removed reviewer Kudu Jenkins.
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
[kudu-CR] [catalog manager] introduce replica type filter
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica type filter
......................................................................
Patch Set 8:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/8161/8//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/8161/8//COMMIT_MSG@9
PS8, Line 9: replica type filter policy
> s/policy//; a replica type filter
Done
http://gerrit.cloudera.org:8080/#/c/8161/8//COMMIT_MSG@11
PS8, Line 11: locate
> include
Done
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 8
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 30 Nov 2017 00:52:50 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] introduce replica type matching policy
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Mike Percy, Kudu Jenkins, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8161
to look at the new patch set (#6).
Change subject: [catalog manager] introduce replica type matching policy
......................................................................
[catalog manager] introduce replica type matching policy
Introduced replica type matching policy for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The policy is used to specify what type of tablet replicas to locate
while populating the result of the GetTabletLocations RPC.
When not specified otherwise or when using earlier versions of the
GetTable[t]LocationsRequestPB protobuf messages, VOTER_REPLICA policy
is used. To locate all replicas, specify ANY_REPLICA policy in the
match_policy field of the GetTable[t]LocationsRequestPB. The latter
is done now unconditionally to enable already existing non-voter
replica-related tests pass. This will change by a follow-up commit:
the 'regular' client operations should work with voter replicas only,
so the use cases for getting all tablet replicas will be just
the following:
* the kudu CLI tool fetching that information to report on replica
membership status for listings in 'table list --tablets' and
'ksck' sub-commands.
* various tests exercising the functionality related to
non-voter tablet replicas
Added an integration test to verify the functionality of the replica
matching policy for GetTableLocations and GetTabletLocations RPC.
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
---
M src/kudu/client/meta_cache.cc
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/delete_table-itest.cc
M src/kudu/integration-tests/flex_partitioning-itest.cc
M src/kudu/integration-tests/raft_config_change-itest.cc
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/integration-tests/tablet_copy-itest.cc
M src/kudu/integration-tests/tombstoned_voting-itest.cc
M src/kudu/integration-tests/ts_itest-base.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
18 files changed, 232 insertions(+), 59 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/8161/6
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] [catalog manager] introduce replica selector
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica selector
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/8161/3/src/kudu/master/master.proto
File src/kudu/master/master.proto:
http://gerrit.cloudera.org:8080/#/c/8161/3/src/kudu/master/master.proto@378
PS3, Line 378: ReplicaSelector
> Bike-shedding, but "ReplicaSelector" (especially when seen in C++) suggests
That's a good observation, thanks.
I just found we already have ReplicaSelection enumeration in Kudu C++ client API, so maybe TabletReplicaType would fit better.
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 05 Oct 2017 20:08:37 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] introduce replica type filter
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica type filter
......................................................................
Patch Set 10: Verified+1
unrelated flake in flags-test
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 10
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 30 Nov 2017 01:39:23 +0000
Gerrit-HasComments: No
[kudu-CR] [catalog manager] introduce replica selector
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica selector
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/8161/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/8161/2//COMMIT_MSG@18
PS2, Line 18: to receive information on NON_VOTER replicas as well.
> The 'regular' client operations should work with voter replicas only, as I
k, would be good to include this in the commit message as well
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 29 Sep 2017 17:51:36 +0000
Gerrit-HasComments: Yes
[kudu-CR] WIP [catalog manager] introduce replica type matching policy
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Adar Dembo, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8161
to look at the new patch set (#4).
Change subject: WIP [catalog manager] introduce replica type matching policy
......................................................................
WIP [catalog manager] introduce replica type matching policy
Introduced replica type matching policy for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The policy is used to specify what type of tablet replicas to locate
while populating the result of the GetTabletLocations RPC.
When not specified otherwise or when using prior version of the
GetTable[t]LocationsRequestPB protobuf messages, VOTER_REPLICA policy
is used. To locate all replicas, specify ANY_REPLICA policy in the
match_policy field of the GetTable[t]LocationsRequestPB.
The 'regular' client operations should work with voter replicas only.
The use case for getting all replicas is when the kudu CLI tool fetches
that information to report on replica membership status for listings
in 'table list --tablets' and 'ksck' sub-commands.
WIP: perhaps, add more tests
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
---
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
11 files changed, 158 insertions(+), 47 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/8161/4
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] [catalog manager] introduce replica type matching policy
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Mike Percy, Kudu Jenkins, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8161
to look at the new patch set (#7).
Change subject: [catalog manager] introduce replica type matching policy
......................................................................
[catalog manager] introduce replica type matching policy
Introduced replica type matching policy for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The policy is used to specify what type of tablet replicas to locate
while populating the result of the GetTabletLocations RPC.
When not specified otherwise or when using earlier versions of the
GetTable[t]LocationsRequestPB protobuf messages, VOTER_REPLICA policy
is used. To locate all replicas, specify ANY_REPLICA policy in the
match_policy field of the GetTable[t]LocationsRequestPB. The latter
is done now unconditionally to enable already existing non-voter
replica-related tests pass. This will change by a follow-up commit:
the 'regular' client operations should work with voter replicas only,
so the use cases for getting all tablet replicas will be just
the following:
* the kudu CLI tool fetching that information to report on replica
membership status for listings in 'table list --tablets' and
'ksck' sub-commands.
* various tests exercising the functionality related to
non-voter tablet replicas
Added an integration test to verify the functionality of the replica
matching policy for GetTableLocations and GetTabletLocations RPC.
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
---
M src/kudu/client/meta_cache.cc
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/delete_table-itest.cc
M src/kudu/integration-tests/flex_partitioning-itest.cc
M src/kudu/integration-tests/raft_config_change-itest.cc
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/integration-tests/tablet_copy-itest.cc
M src/kudu/integration-tests/tombstoned_voting-itest.cc
M src/kudu/integration-tests/ts_itest-base.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
18 files changed, 231 insertions(+), 59 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/8161/7
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 7
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] [catalog manager] introduce replica type filter
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed Kudu Jenkins from this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica type filter
......................................................................
Removed reviewer Kudu Jenkins with the following votes:
* Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 10
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] [catalog manager] introduce replica type filter
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica type filter
......................................................................
Patch Set 7:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
File src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc:
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc@198
PS7, Line 198: GetTabletLocation
> GetTabletLocations
Done
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/catalog_manager.h
File src/kudu/master/catalog_manager.h:
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/catalog_manager.h@722
PS7, Line 722: include_non_voters
> flag name changed
Done
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/catalog_manager.h@723
PS7, Line 723: inconsistency
> unclear to me what this means
yeah, it reads better without that detail, given that the InvalidArgument most likely corresponds to a programming error.
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/catalog_manager.h@727
PS7, Line 727: sel
> how about naming this 'filter' ?
Done
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master.proto
File src/kudu/master/master.proto:
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master.proto@366
PS7, Line 366: ReplicaMatchPolicy
> How about ReplicaTypeFilter
Done
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master.proto@378
PS7, Line 378: match_policy
> replica_type_filter ?
Done
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master.proto@463
PS7, Line 463: match_policy
> same
Done
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master_service.cc
File src/kudu/master/master_service.cc:
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/master/master_service.cc@241
PS7, Line 241: unknown
> todd (per https://github.com/apache/kudu/blame/master/src/kudu/master/maste
Done
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 7
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Wed, 29 Nov 2017 23:44:52 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] introduce replica type filter
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica type filter
......................................................................
Patch Set 10: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 10
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 01 Dec 2017 09:05:39 +0000
Gerrit-HasComments: No
[kudu-CR] WIP [catalog manager] introduce replica type matching policy
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: WIP [catalog manager] introduce replica type matching policy
......................................................................
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/8161/4/src/kudu/integration-tests/raft_consensus-itest.cc
File src/kudu/integration-tests/raft_consensus-itest.cc:
http://gerrit.cloudera.org:8080/#/c/8161/4/src/kudu/integration-tests/raft_consensus-itest.cc@119
PS4, Line 119: using kudu::master::ANY_REPLICA;
> warning: using decl 'ANY_REPLICA' is unused [misc-unused-using-decls]
Done
http://gerrit.cloudera.org:8080/#/c/8161/4/src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
File src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc:
http://gerrit.cloudera.org:8080/#/c/8161/4/src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc@73
PS4, Line 73: using kudu::master::VOTER_REPLICA;
> warning: using decl 'VOTER_REPLICA' is unused [misc-unused-using-decls]
Done
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 17 Nov 2017 22:01:45 +0000
Gerrit-HasComments: Yes
[kudu-CR] WIP [catalog manager] introduce replica type matching policy
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Mike Percy, Kudu Jenkins, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8161
to look at the new patch set (#5).
Change subject: WIP [catalog manager] introduce replica type matching policy
......................................................................
WIP [catalog manager] introduce replica type matching policy
Introduced replica type matching policy for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The policy is used to specify what type of tablet replicas to locate
while populating the result of the GetTabletLocations RPC.
When not specified otherwise or when using earlier versions of the
GetTable[t]LocationsRequestPB protobuf messages, VOTER_REPLICA policy
is used. To locate all replicas, specify ANY_REPLICA policy in the
match_policy field of the GetTable[t]LocationsRequestPB. The latter
is done now unconditionally to enable already existing non-voter
replica-related tests pass. This will change by a follow-up commit:
the 'regular' client operations should work with voter replicas only,
so the use cases for getting all tablet replicas will be just
the following:
* the kudu CLI tool fetching that information to report on replica
membership status for listings in 'table list --tablets' and
'ksck' sub-commands.
* various tests exercising the functionality related to
non-voter tablet replicas
WIP: perhaps, add more tests
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
---
M src/kudu/client/meta_cache.cc
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/integration-tests/ts_itest-base.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
13 files changed, 164 insertions(+), 47 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/8161/5
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] [catalog manager] introduce replica selector
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica selector
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/8161/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/8161/2//COMMIT_MSG@18
PS2, Line 18: to receive information on NON_VOTER replicas as well.
can you elaborate on the use case here? What client operations need to specify a selection policy instead of always returning all members, and why? Would be nice to motivate the change use-case-first to evaluate whether the protocol changes make sense for what is needed by the clients, since it's pretty hard to change the proto later if it doesn't fit.
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 28 Sep 2017 20:21:37 +0000
Gerrit-HasComments: Yes
[kudu-CR] WIP [catalog manager] introduce replica type matching policy
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed Adar Dembo from this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: WIP [catalog manager] introduce replica type matching policy
......................................................................
Removed reviewer Adar Dembo.
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] [catalog manager] introduce replica selector
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica selector
......................................................................
Patch Set 3:
> (1 comment)
And another option is ReplicaMatchPolicy -- I think I'll use that one.
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 05 Oct 2017 20:10:46 +0000
Gerrit-HasComments: No
[kudu-CR] [catalog manager] introduce replica type filter
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica type filter
......................................................................
Patch Set 8:
(2 comments)
oops, just a couple more commit message nits
http://gerrit.cloudera.org:8080/#/c/8161/8//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/8161/8//COMMIT_MSG@9
PS8, Line 9: replica type filter policy
s/policy//; a replica type filter
http://gerrit.cloudera.org:8080/#/c/8161/8//COMMIT_MSG@11
PS8, Line 11: locate
include
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 8
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 30 Nov 2017 00:09:23 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] introduce replica selector
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica selector
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/8161/2/src/kudu/master/catalog_manager.h
File src/kudu/master/catalog_manager.h:
http://gerrit.cloudera.org:8080/#/c/8161/2/src/kudu/master/catalog_manager.h@692
PS2, Line 692: Status BuildLocationsForTablet(const scoped_refptr<TabletInfo>& tablet,
> warning: function 'kudu::master::CatalogManager::BuildLocationsForTablet' h
Done
http://gerrit.cloudera.org:8080/#/c/8161/2/src/kudu/master/catalog_manager.cc
File src/kudu/master/catalog_manager.cc:
http://gerrit.cloudera.org:8080/#/c/8161/2/src/kudu/master/catalog_manager.cc@4110
PS2, Line 4110: } else if (s.IsNotFound()) {
> warning: do not use 'else' after 'continue' [readability-else-after-return]
Done
http://gerrit.cloudera.org:8080/#/c/8161/2/src/kudu/master/master_service.cc
File src/kudu/master/master_service.cc:
http://gerrit.cloudera.org:8080/#/c/8161/2/src/kudu/master/master_service.cc@239
PS2, Line 239: // TODO: once we have catalog data. ACL checks would also go here, probably.
> warning: missing username/bug in TODO [google-readability-todo]
Done
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 29 Sep 2017 23:13:34 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] introduce replica type matching policy
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica type matching policy
......................................................................
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
File src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc:
http://gerrit.cloudera.org:8080/#/c/8161/7/src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc@198
PS7, Line 198: GetTabletLocation
GetTabletLocations
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 7
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Wed, 29 Nov 2017 03:36:35 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] introduce replica type filter
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Mike Percy, Kudu Jenkins, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8161
to look at the new patch set (#10).
Change subject: [catalog manager] introduce replica type filter
......................................................................
[catalog manager] introduce replica type filter
Introduced a replica type filter for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The filter is used to specify what type of tablet replicas to include
while populating the result of the GetTabletLocations RPC.
When not specified otherwise or when using earlier versions of the
GetTable[t]LocationsRequestPB protobuf messages, VOTER_REPLICA policy
is used. To locate all replicas, specify ANY_REPLICA policy in the
'replica_type_filter' field of the GetTable[t]LocationsRequestPB. The
latter is done now unconditionally for already existing non-voter
replica-related tests. This will change by a follow-up commit:
the 'regular' client operations should work with voter replicas only,
so the use cases for getting all tablet replicas will be just
the following:
* the kudu CLI tool fetching that information to report on replica
membership status for listings in 'table list --tablets' and
'ksck' sub-commands.
* various tests exercising the functionality related to
non-voter tablet replicas
Added an integration test to verify the functionality of the replica
type filter for GetTableLocations and GetTabletLocations RPC.
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
---
M src/kudu/client/meta_cache.cc
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/delete_table-itest.cc
M src/kudu/integration-tests/flex_partitioning-itest.cc
M src/kudu/integration-tests/raft_config_change-itest.cc
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/integration-tests/tablet_copy-itest.cc
M src/kudu/integration-tests/tombstoned_voting-itest.cc
M src/kudu/integration-tests/ts_itest-base.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
18 files changed, 242 insertions(+), 64 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/8161/10
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 10
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] [catalog manager] introduce replica selector
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica selector
......................................................................
[catalog manager] introduce replica selector
Introduced replica selector for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The selector is used to specify replica membership matching policy
while populating the result of the GetTabletLocations RPC.
As of now, only VOTER_REPLICA policy is used for all GetTabletLocations
invocations and only VOTER replicas are returned in the result.
As soon as NON_VOTER replicas appear in a follow-up commit, it will
be possible to use ANY_REPLICA policy with GetTabletLocation request
to receive information on NON_VOTER replicas as well.
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
---
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
10 files changed, 160 insertions(+), 47 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/8161/2
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
[kudu-CR] [catalog manager] introduce replica selector
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica selector
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/8161/3/src/kudu/master/master.proto
File src/kudu/master/master.proto:
http://gerrit.cloudera.org:8080/#/c/8161/3/src/kudu/master/master.proto@378
PS3, Line 378: ReplicaSelector
Bike-shedding, but "ReplicaSelector" (especially when seen in C++) suggests a stateful class that uses some sort of heuristic or algorithm to make a selection. The reality is that it's just an enum, so perhaps ReplicaSelectionMode would be a clearer name?
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Mon, 02 Oct 2017 21:36:53 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] introduce replica type filter
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Mike Percy, Kudu Jenkins, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8161
to look at the new patch set (#8).
Change subject: [catalog manager] introduce replica type filter
......................................................................
[catalog manager] introduce replica type filter
Introduced replica type filter policy for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The filter is used to specify what type of tablet replicas to locate
while populating the result of the GetTabletLocations RPC.
When not specified otherwise or when using earlier versions of the
GetTable[t]LocationsRequestPB protobuf messages, VOTER_REPLICA policy
is used. To locate all replicas, specify ANY_REPLICA policy in the
'replica_type_filter' field of the GetTable[t]LocationsRequestPB. The
latter is done now unconditionally for already existing non-voter
replica-related tests. This will change by a follow-up commit:
the 'regular' client operations should work with voter replicas only,
so the use cases for getting all tablet replicas will be just
the following:
* the kudu CLI tool fetching that information to report on replica
membership status for listings in 'table list --tablets' and
'ksck' sub-commands.
* various tests exercising the functionality related to
non-voter tablet replicas
Added an integration test to verify the functionality of the replica
type filter for GetTableLocations and GetTabletLocations RPC.
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
---
M src/kudu/client/meta_cache.cc
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/delete_table-itest.cc
M src/kudu/integration-tests/flex_partitioning-itest.cc
M src/kudu/integration-tests/raft_config_change-itest.cc
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/integration-tests/tablet_copy-itest.cc
M src/kudu/integration-tests/tombstoned_voting-itest.cc
M src/kudu/integration-tests/ts_itest-base.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
18 files changed, 243 insertions(+), 66 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/8161/8
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 8
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] [catalog manager] introduce replica selector
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/8161 )
Change subject: [catalog manager] introduce replica selector
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/8161/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/8161/2//COMMIT_MSG@18
PS2, Line 18: to receive information on NON_VOTER replicas as well.
> can you elaborate on the use case here? What client operations need to spec
The 'regular' client operations should work with voter replicas only, as I understand.
The use case for getting all replicas is when kudu CLI tool fetches that information to report on replica membership status for listings in 'table list --tablets' and 'ksck' sub-commands.
I hope to post corresponding patch for that later today.
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Thu, 28 Sep 2017 22:39:50 +0000
Gerrit-HasComments: Yes
[kudu-CR] [catalog manager] introduce replica type filter
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Mike Percy, Kudu Jenkins, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/8161
to look at the new patch set (#9).
Change subject: [catalog manager] introduce replica type filter
......................................................................
[catalog manager] introduce replica type filter
Introduced replica type filter policy for GetTabletLocations RPC and
various wrapper methods used in master, catalog manager, and tests.
The filter is used to specify what type of tablet replicas to locate
while populating the result of the GetTabletLocations RPC.
When not specified otherwise or when using earlier versions of the
GetTable[t]LocationsRequestPB protobuf messages, VOTER_REPLICA policy
is used. To locate all replicas, specify ANY_REPLICA policy in the
'replica_type_filter' field of the GetTable[t]LocationsRequestPB. The
latter is done now unconditionally for already existing non-voter
replica-related tests. This will change by a follow-up commit:
the 'regular' client operations should work with voter replicas only,
so the use cases for getting all tablet replicas will be just
the following:
* the kudu CLI tool fetching that information to report on replica
membership status for listings in 'table list --tablets' and
'ksck' sub-commands.
* various tests exercising the functionality related to
non-voter tablet replicas
Added an integration test to verify the functionality of the replica
type filter for GetTableLocations and GetTabletLocations RPC.
Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
---
M src/kudu/client/meta_cache.cc
M src/kudu/integration-tests/cluster_itest_util.cc
M src/kudu/integration-tests/cluster_itest_util.h
M src/kudu/integration-tests/delete_table-itest.cc
M src/kudu/integration-tests/flex_partitioning-itest.cc
M src/kudu/integration-tests/raft_config_change-itest.cc
M src/kudu/integration-tests/raft_consensus-itest.cc
M src/kudu/integration-tests/raft_consensus_nonvoter-itest.cc
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/integration-tests/tablet_copy-itest.cc
M src/kudu/integration-tests/tombstoned_voting-itest.cc
M src/kudu/integration-tests/ts_itest-base.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
M src/kudu/tools/kudu-admin-test.cc
18 files changed, 242 insertions(+), 64 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/61/8161/9
--
To view, visit http://gerrit.cloudera.org:8080/8161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I303a6d158184575a9a105c2d2bf26961ae8b3e93
Gerrit-Change-Number: 8161
Gerrit-PatchSet: 9
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>