[jira] [Commented] (CASSANDRA-10134) Always require replace_address to replace existing address

["Stefania (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096261#comment-15096261 ] 

Stefania commented on CASSANDRA-10134:
--------------------------------------

The first problem I've encountered is that if we call {{checkForEndpointCollision()}} for seeds then all seeds die in the Gossip shadow round when they are started simultaneously because none of them are able to make progress ({{ERROR: Unable to gossip with any seeds}}).


> Always require replace_address to replace existing address
> ----------------------------------------------------------
>
>                 Key: CASSANDRA-10134
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10134
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Distributed Metadata
>            Reporter: Tyler Hobbs
>            Assignee: Stefania
>             Fix For: 2.2.x, 3.0.x, 3.x
>
>
> Normally, when a node is started from a clean state with the same address as an existing down node, it will fail to start with an error like this:
> {noformat}
> ERROR [main] 2015-08-19 15:07:51,577 CassandraDaemon.java:554 - Exception encountered during startup
> java.lang.RuntimeException: A node with address /127.0.0.3 already exists, cancelling join. Use cassandra.replace_address if you want to replace this node.
> 	at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:543) ~[main/:na]
> 	at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:783) ~[main/:na]
> 	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:720) ~[main/:na]
> 	at org.apache.cassandra.service.StorageService.initServer(StorageService.java:611) ~[main/:na]
> 	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) [main/:na]
> 	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:537) [main/:na]
> 	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:626) [main/:na]
> {noformat}
> However, if {{auto_bootstrap}} is set to false or the node is in its own seed list, it will not throw this error and will start normally.  The new node then takes over the host ID of the old node (even if the tokens are different), and the only message you will see is a warning in the other nodes' logs:
> {noformat}
> logger.warn("Changing {}'s host ID from {} to {}", endpoint, storedId, hostId);
> {noformat}
> This could cause an operator to accidentally wipe out the token information for a down node without replacing it.  To fix this, we should check for an endpoint collision even if {{auto_bootstrap}} is false or the node is a seed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)