You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Alexey Serbin (Jira)" <ji...@apache.org> on 2019/09/30 17:23:00 UTC

[jira] [Comment Edited] (KUDU-2914) Rebalance tool support moving replicas from some specific tablet servers

    [ https://issues.apache.org/jira/browse/KUDU-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941152#comment-16941152 ] 

Alexey Serbin edited comment on KUDU-2914 at 9/30/19 5:22 PM:
--------------------------------------------------------------

Some more information just for better perspective.  Let's take a look at the bigger picture in case of the 3-4-3 replica management scheme.  I think we can ignore the old 3-2-3 scheme which was used in Kudu v1.6 and earlier.

In case of the 3-4-3 scheme, if it's necessary to remove a tablet replica from a tablet server, it's enough to mark the replica with the {{REPLACE}} attribute (see {{RaftConsensus::BulkChangeConfig()}}, {{RaftPeerAttrsPB::replace}} in {{src/kudu/consensus/metadata.proto}}).  Once the {{REPLACE}} attribute is set, the logic in the master automatically creates a non-voter replacement replica elsewhere, trying to satisfy the placement policy and keeping the tablet distribution as balanced as possible.  Eventually, when a new replica is promoted to a voter, the source replica is removed.

If there were a way to mark a tablet server as not available for placement of new tablet replicas (something like already implemented with https://github.com/apache/kudu/commit/e8362ca1bb48ed0150cf134d93899d7a2f74caf3 as a part of [KUDU-2069|https://issues.apache.org/jira/browse/KUDU-2069]), it would be necessary to simply mark the server as 'decomissioned' and then mark all replicas at the tablet server with the {{REPLACE}} attribute.  After some time, Kudu master would automatically remove and redistribute all tablet replicas from the tablet server.


was (Author: aserbin):
Some more information just for better perspective.  Let's take a look at the bigger picture in case of the 3-4-3 replica management scheme.  I think we can ignore the old 3-2-3 scheme which was used in Kudu v1.6 and earlier.

In case of the 3-4-3 scheme, if it's necessary to remove a tablet replica from a tablet server, it's enough to mark the replica with the {{REPLACE}} attribute (see {{RaftConsensus::BulkChangeConfig()}}, {{RaftPeerAttrsPB::replace}} in {{src/kudu/consensus/metadata.proto}}).  Once the {{REPLACE}} attribute is set, the logic in the master automatically creates a non-voter replacement replica elsewhere, trying to satisfy the placement policy and keeping the tablet distribution as balanced as possible.  Eventually, when a new replica is promoted to a voter, the source replica is removed.

If there were a way to mark a tablet server as not available for placement of new tablet replicas (something like already implemented with https://github.com/apache/kudu/commit/e8362ca1bb48ed0150cf134d93899d7a2f74caf3 as a part of [KUDU-2069|https://issues.apache.org/jira/browse/KUDU-2069]), it would be necessary to simply mark all replicas at the tablet server with the {{REPLACE}} attribute and await until there is no tablet replicas left at the tablet server.

> Rebalance tool support moving replicas from some specific tablet servers
> ------------------------------------------------------------------------
>
>                 Key: KUDU-2914
>                 URL: https://issues.apache.org/jira/browse/KUDU-2914
>             Project: Kudu
>          Issue Type: Improvement
>          Components: CLI
>            Reporter: YifanZhang
>            Assignee: YifanZhang
>            Priority: Minor
>
> When we need to remove some tservers from a kudu cluster (maybe just for saving resources or replacing these servers with new servers), it's better to move all replicas on these tservers to other tservers in a cluster in advance, instead of waiting for all replicas kicked out and evicting new replicas. This can be achieved by rebalance tool supporting specifying 'blacklist_tservers'.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)