[kudu-CR] [consensus] KUDU-2181 Allow sending status msgs to FAILED peers

["Bankim Bhavsar (Code Review)" <ge...@cloudera.org>]

Hello Kudu Jenkins, Andrew Wong, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16830

to look at the new patch set (#5).

Change subject: [consensus] KUDU-2181 Allow sending status msgs to FAILED peers
......................................................................

[consensus] KUDU-2181 Allow sending status msgs to FAILED peers

This change adds the ability for the leader to send status only messages
to the peer even if it's in FAILED_UNRECOVERABLE state. Without this
change when the system catalog is copied externally the new master
remains in FAILED_UNRECOVERABLE state and doesn't get promoted to being
a VOTER despite the system catalog being up to date. This behavior is
currently disabled by default under the
--consensus_allow_status_msg_for_failed_peer flag.

Bulk of the change is a test that outlines the procedure to copy system
catalog for the newly added master using existing CLI tools and
the master ChangeConfig RPC.
- Flag --consensus_allow_status_msg_for_failed_peer must be turned on
for existing masters.
- Start the new master with
--master_address_add_new_master=<new-master-hostport> and
--master_addresses that contains itself and existing masters.
- Invoke ChangeConfig to add the master.
- If the new master is promoted to being a VOTER then following tablet
copy steps can be skipped.
- Shutdown the new master.
- Delete the system catalog on the new master.
- Copy the system catalog from the leader master to the new master.
- Bring up the new master.
- Verify the new master is promoted as VOTER.

Change-Id: I142c1dec442ec72c38c5be9d62cdf270e441d6e3
---
M src/kudu/consensus/consensus_queue.cc
M src/kudu/master/dynamic_multi_master-test.cc
2 files changed, 279 insertions(+), 137 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/16830/5
-- 
To view, visit http://gerrit.cloudera.org:8080/16830
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I142c1dec442ec72c38c5be9d62cdf270e441d6e3
Gerrit-Change-Number: 16830
Gerrit-PatchSet: 5
Gerrit-Owner: Bankim Bhavsar <ba...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Bankim Bhavsar <ba...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)