[jira] [Created] (CASSANDRA-5571) Reject bootstrapping endpoints that are already in the ring with different gossip data

["Rick Branson (JIRA)" <ji...@apache.org>]

Rick Branson created CASSANDRA-5571:
---------------------------------------

             Summary: Reject bootstrapping endpoints that are already in the ring with different gossip data
                 Key: CASSANDRA-5571
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5571
             Project: Cassandra
          Issue Type: Improvement
          Components: Core
            Reporter: Rick Branson
            Assignee: Rick Branson


The ring can be silently broken by improperly bootstrapping an endpoint that has existing entries in the gossip tables. In the case where a node attempts to bootstrap with the same IP address as an existing ring member, the old token metadata is dropped without warning, resulting in range shifts for the cluster.

This isn't so bad for non-vnode cases where, in general, tokens are explicitly assigned, and a bootstrap on the same token would result in no range shifts. For vnode cases, the convention is to just let nodes come up by selecting their own tokens, and a bootstrap will override the existing tokens for that endpoint.

While there are some other issues open for adding an explicit rebootstrap feature for vnode cases, given the changes in operator habits for vnode rings, it seems a bit too easy to make this happen. Even more undesirable is the fact that it's basically silent.

This is a proposal for checking for this exact case: bootstraps on endpoints with existing ring entries that have different hostIDs and/or tokens should be rejected with an error message describing what happened and how to override the safety check. It looks like the override can be supported using the existing "nodetool removenode -force".

I can work up a patch for this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira