You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Axel Kämpfe (JIRA)" <ji...@apache.org> on 2015/07/23 13:24:04 UTC
[jira] [Updated] (CASSANDRA-9878) Rolling Updates 2.0 to 2.1
"unable to gossip"
[ https://issues.apache.org/jira/browse/CASSANDRA-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Axel Kämpfe updated CASSANDRA-9878:
-----------------------------------
Description:
Hi there,
we are currently testing an upgrade of our servers from Cassandra 2.0.16 to 2.1.8 on the EC2 service of Amazon
Usually, we launch a new server which gets the newest version and then joins the existing ring, bootstrapping, and after some time we kill one of the old nodes.
But with the upgrade to 2.1 the new server will not join the existing ring.
{panel:title=system.log}
INFO [HANDSHAKE-/10.] 2015-07-23 11:06:14,997 OutboundTcpConnection.java:485 - Handshaking version with /10.xx.yy.zz
INFO [HANDSHAKE-/10.] 2015-07-23 11:06:14,999 OutboundTcpConnection.java:485 - Handshaking version with /10.yy.zz.aa
INFO [HANDSHAKE-/10.] 2015-07-23 11:06:15,000 OutboundTcpConnection.java:485 - Handshaking version with /10.aa.bb.cc
ERROR [main] 2015-07-23 11:06:46,016 CassandraDaemon.java:541 - Exception encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1307) ~[apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:533) ~[apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:777) ~[apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:714) ~[apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:605) ~[apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) [apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:524) [apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:613) [apache-cassandra-2.1.8.jar:2.1.8]
WARN [StorageServiceShutdownHook] 2015-07-23 11:06:46,023 Gossiper.java:1418 - No local state or state is in silent shutdown, not announcing shutdown
INFO [StorageServiceShutdownHook] 2015-07-23 11:06:46,023 MessagingService.java:708 - Waiting for messaging service to quiesce
INFO [ACCEPT-/10.] 2015-07-23 11:06:46,045 MessagingService.java:958 - MessagingService has terminated the accept() thread
{panel}
our config uses the "RandomPartitioner", "SimpleSnitch" and the internal nodes ip for communication.
when i use the same config but ONLY 2.1.x Servers, everthing works perfectly, but as soon as we start to "mix in" a new version into an "old" ring, the new servers will not "gossip"... ( so it cannot be a firewall issue, or such)
If you need any more information from my side, please let me know.
was:
Hi there,
we are currently testing an upgrade of our servers from Cassandra 2.0.16 to 2.1.8 on the EC2 service of Amazon
Usually, we launch a new server which gets the newest version and then joins the existing ring, bootstrapping, and after some time we kill one of the old nodes.
But with the upgrade to 2.1 the new server will not join the existing ring.
{panel:title=system.log}
INFO [HANDSHAKE-/10.84.17.93] 2015-07-23 11:06:14,997 OutboundTcpConnection.java:485 - Handshaking version with /10.84.17.93
INFO [HANDSHAKE-/10.90.1.242] 2015-07-23 11:06:14,999 OutboundTcpConnection.java:485 - Handshaking version with /10.90.1.242
INFO [HANDSHAKE-/10.97.180.155] 2015-07-23 11:06:15,000 OutboundTcpConnection.java:485 - Handshaking version with /10.97.180.155
ERROR [main] 2015-07-23 11:06:46,016 CassandraDaemon.java:541 - Exception encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1307) ~[apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:533) ~[apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:777) ~[apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:714) ~[apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:605) ~[apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) [apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:524) [apache-cassandra-2.1.8.jar:2.1.8]
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:613) [apache-cassandra-2.1.8.jar:2.1.8]
WARN [StorageServiceShutdownHook] 2015-07-23 11:06:46,023 Gossiper.java:1418 - No local state or state is in silent shutdown, not announcing shutdown
INFO [StorageServiceShutdownHook] 2015-07-23 11:06:46,023 MessagingService.java:708 - Waiting for messaging service to quiesce
INFO [ACCEPT-/10.82.152.187] 2015-07-23 11:06:46,045 MessagingService.java:958 - MessagingService has terminated the accept() thread
{panel}
our config uses the "RandomPartitioner", "SimpleSnitch" and the internal nodes ip for communication.
when i use the same config but ONLY 2.1.x Servers, everthing works perfectly, but as soon as we start to "mix in" a new version into an "old" ring, the new servers will not "gossip"... ( so it cannot be a firewall issue, or such)
If you need any more information from my side, please let me know.
> Rolling Updates 2.0 to 2.1 "unable to gossip"
> ---------------------------------------------
>
> Key: CASSANDRA-9878
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9878
> Project: Cassandra
> Issue Type: Bug
> Reporter: Axel Kämpfe
>
> Hi there,
> we are currently testing an upgrade of our servers from Cassandra 2.0.16 to 2.1.8 on the EC2 service of Amazon
> Usually, we launch a new server which gets the newest version and then joins the existing ring, bootstrapping, and after some time we kill one of the old nodes.
> But with the upgrade to 2.1 the new server will not join the existing ring.
> {panel:title=system.log}
> INFO [HANDSHAKE-/10.] 2015-07-23 11:06:14,997 OutboundTcpConnection.java:485 - Handshaking version with /10.xx.yy.zz
> INFO [HANDSHAKE-/10.] 2015-07-23 11:06:14,999 OutboundTcpConnection.java:485 - Handshaking version with /10.yy.zz.aa
> INFO [HANDSHAKE-/10.] 2015-07-23 11:06:15,000 OutboundTcpConnection.java:485 - Handshaking version with /10.aa.bb.cc
> ERROR [main] 2015-07-23 11:06:46,016 CassandraDaemon.java:541 - Exception encountered during startup
> java.lang.RuntimeException: Unable to gossip with any seeds
> at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1307) ~[apache-cassandra-2.1.8.jar:2.1.8]
> at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:533) ~[apache-cassandra-2.1.8.jar:2.1.8]
> at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:777) ~[apache-cassandra-2.1.8.jar:2.1.8]
> at org.apache.cassandra.service.StorageService.initServer(StorageService.java:714) ~[apache-cassandra-2.1.8.jar:2.1.8]
> at org.apache.cassandra.service.StorageService.initServer(StorageService.java:605) ~[apache-cassandra-2.1.8.jar:2.1.8]
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) [apache-cassandra-2.1.8.jar:2.1.8]
> at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:524) [apache-cassandra-2.1.8.jar:2.1.8]
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:613) [apache-cassandra-2.1.8.jar:2.1.8]
> WARN [StorageServiceShutdownHook] 2015-07-23 11:06:46,023 Gossiper.java:1418 - No local state or state is in silent shutdown, not announcing shutdown
> INFO [StorageServiceShutdownHook] 2015-07-23 11:06:46,023 MessagingService.java:708 - Waiting for messaging service to quiesce
> INFO [ACCEPT-/10.] 2015-07-23 11:06:46,045 MessagingService.java:958 - MessagingService has terminated the accept() thread
> {panel}
> our config uses the "RandomPartitioner", "SimpleSnitch" and the internal nodes ip for communication.
> when i use the same config but ONLY 2.1.x Servers, everthing works perfectly, but as soon as we start to "mix in" a new version into an "old" ring, the new servers will not "gossip"... ( so it cannot be a firewall issue, or such)
> If you need any more information from my side, please let me know.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)