[jira] [Commented] (CASSANDRA-5888) Need a way to start or bootstrap a node regardless of its current state

["Brandon Williams (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13739932#comment-13739932 ] 

Brandon Williams commented on CASSANDRA-5888:
---------------------------------------------

bq. 1. If a data directory is purged and the node is restarted with the same host IP and initial token, the node will successfully start but will not bootstrap, creating a silent consistency loss.

Can you explain how to recreate this? Because without explicitly setting auto_bootstrap to false I don't see how this can happen, unless you don't fully purge the data directory and leave the system tables intact.

bq. 2. If -Dcassandra.replace_token is given, the node will bootstrap again, even if its successfully bootstrapped already with the same token.

We can add a check for replace_token where it will fail if the node has bootstrapped previously.

bq. Cassandra daemon should always bootstrap on start if it hasn't yet finished bootstrapping successfully. -Dcassandra.replace_token (or host replacement equivalent with nodes) should just ALLOW replacing a token of a down node, but not force an unnecessary bootstrap or fail if the token is not present.

If the operator is telling Cassandra explicitly to replace a token and that token doesn't exist, Cassandra should alert them to their mistake rather than try to guess what they really want to do.  If external tools need to infer ring membership status to make intelligent decisions, they can do this via JMX the same way nodetool does.

                
> Need a way to start or bootstrap a node regardless of its current state
> -----------------------------------------------------------------------
>
>                 Key: CASSANDRA-5888
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5888
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Oleg Kibirev
>
> Currently, there is no straightforward way to start a cassandra node on a given host without knowing:
> 1. Weather the node's initial_token has been successfully registered through gossip 
> 2. If the node has successfully finished bootstrapping
> 3. Weather data directory has been purged by, say, replacing a dead disk
> Specifically the following cases are problematic:
> 1. If a data directory is purged and the node is restarted with the same host IP and initial token, the node will successfully start but will not bootstrap, creating a silent consistency loss.
> 2. If -Dcassandra.replace_token is given, the node will bootstrap again, even if its successfully bootstrapped already with the same token.
> The information on the correct thing to do can only come from gossip and system keyspace. Its very difficult to infer correct start arguments from a process external to cassandra and have it work 100% of the time on large scale. What if a node already gossiped its token but has not successfully finished bootstrapping - how do I know to drop replace_token and will it still re-bootstrap?
> Cassandra daemon should always bootstrap on start if it hasn't yet finished bootstrapping successfully. -Dcassandra.replace_token (or host replacement equivalent with nodes) should just ALLOW replacing a token of a down node, but not force an unnecessary bootstrap or fail if the token is not present.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira