You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "HuS.Andy" <hu...@hotmail.com> on 2013/07/15 05:00:45 UTC
Setup cassandra cluster across 2 openstack cluster
Hi,
I have 4 node, they are vm, maked as node A and node B on the same host X.X.X.104, Node C and Node D on the same host X.X.X.106. X.X.X.104 and X.X.X.106 are single node Openstack cluster.
When I start Node A and Node B, with configuration snitch:Ec2MultiRegionSnitch, seeds: NodeA public IP,NodeB public IP,Node A and Node B can be a cassandra cluster for a few seconds. but then they tells the other is dead.
I get the logs:
on the previous seconds, Node A and Node B can recieve each other's GossipDigestAckMessage and GossipDigestAck2Message.but after that, Node A can Node B can't recieve each other's syn msg
I can't find the key to solve this problem, Can somebody help me out, thanks. May is the openstack error
Node A's Log:private Ip 192.168.1.7 public ip 10.10.101.69
Here Connected with Node BDEBUG [Thread-3] 2013-07-12 15:17:38,650 IncomingTcpConnection.java (line 75) Connection version 6 from /10.10.101.200DEBUG [Thread-3] 2013-07-12 15:17:38,653 IncomingTcpConnection.java (line 112) Upgrading incoming connection to be compressedDEBUG [Thread-3] 2013-07-12 15:17:38,656 IncomingTcpConnection.java (line 120) Max version for /10.10.101.200 is 6DEBUG [Thread-3] 2013-07-12 15:17:38,657 MessagingService.java (line 809) Setting version 6 for /10.10.101.200DEBUG [Thread-3] 2013-07-12 15:17:38,658 IncomingTcpConnection.java (line 129) set version for /10.10.101.200 to 6TRACE [GossipStage:1] 2013-07-12 15:17:38,662 GossipDigestSynVerbHandler.java (line 40) Received a GossipDigestSynMessage from /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,662 GossipDigestSynVerbHandler.java (line 71) Gossip syn digests are : /10.10.101.200:1373613457:9TRACE [GossipStage:1] 2013-07-12 15:17:38,663 Gossiper.java (line 945) requestAll for /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,664 GossipDigestSynVerbHandler.java (line 84) Sending a GossipDigestAckMessage to /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,664 MessagingService.java (line 602) /10.10.101.69 sending GOSSIP_DIGEST_ACK to 8@/10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,665 GossipDigestAckVerbHandler.java (line 41) Received a GossipDigestAckMessage from /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,666 Gossiper.java (line 681) local heartbeat version 20 greater than 0 for /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,670 Gossiper.java (line 695) Adding state LOAD: 37474.0TRACE [GossipStage:1] 2013-07-12 15:17:38,672 Gossiper.java (line 695) Adding state INTERNAL_IP: 192.168.1.7TRACE [GossipStage:1] 2013-07-12 15:17:38,672 Gossiper.java (line 695) Adding state RELEASE_VERSION: 1.2.6TRACE [GossipStage:1] 2013-07-12 15:17:38,673 Gossiper.java (line 695) Adding state DC: novTRACE [GossipStage:1] 2013-07-12 15:17:38,673 Gossiper.java (line 695) Adding state NET_VERSION: 6TRACE [GossipStage:1] 2013-07-12 15:17:38,674 Gossiper.java (line 695) Adding state RACK: novaTRACE [GossipStage:1] 2013-07-12 15:17:38,674 Gossiper.java (line 695) Adding state TOKENS: ^@^@^@^H<80>^@^@^@^@^@^@^@^@^@^@^@TRACE [GossipStage:1] 2013-07-12 15:17:38,675 Gossiper.java (line 695) Adding state RPC_ADDRESS: 0.0.0.0TRACE [GossipStage:1] 2013-07-12 15:17:38,675 Gossiper.java (line 695) Adding state SCHEMA: d8e69b1e-c675-361a-ab78-beba6aade405TRACE [GossipStage:1] 2013-07-12 15:17:38,676 Gossiper.java (line 695) Adding state HOST_ID: f24fd5a2-2023-4414-b8f1-88cdd294022fTRACE [GossipStage:1] 2013-07-12 15:17:38,676 Gossiper.java (line 695) Adding state STATUS: NORMAL,-9223372036854775808TRACE [GossipStage:1] 2013-07-12 15:17:38,687 GossipDigestAckVerbHandler.java (line 74) Sending a GossipDigestAck2Message to /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,687 MessagingService.java (line 602) /10.10.101.69 sending GOSSIP_DIGEST_ACK2 to 11@/10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,700 GossipDigestAckVerbHandler.java (line 41) Received a GossipDigestAckMessage from /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,701 Gossiper.java (line 681) local heartbeat version 20 greater than 0 for /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,701 Gossiper.java (line 695) Adding state LOAD: 37474.0TRACE [GossipStage:1] 2013-07-12 15:17:38,701 Gossiper.java (line 695) Adding state INTERNAL_IP: 192.168.1.7TRACE [GossipStage:1] 2013-07-12 15:17:38,701 Gossiper.java (line 695) Adding state RELEASE_VERSION: 1.2.6TRACE [GossipStage:1] 2013-07-12 15:17:38,702 Gossiper.java (line 695) Adding state DC: novTRACE [GossipStage:1] 2013-07-12 15:17:38,702 Gossiper.java (line 695) Adding state NET_VERSION: 6TRACE [GossipStage:1] 2013-07-12 15:17:38,702 Gossiper.java (line 695) Adding state RACK: novaTRACE [GossipStage:1] 2013-07-12 15:17:38,702 Gossiper.java (line 695) Adding state TOKENS: ^@^@^@^H<80>^@^@^@^@^@^@^@^@^@^@^@TRACE [GossipStage:1] 2013-07-12 15:17:38,703 Gossiper.java (line 695) Adding state RPC_ADDRESS: 0.0.0.0TRACE [GossipStage:1] 2013-07-12 15:17:38,703 Gossiper.java (line 695) Adding state SCHEMA: d8e69b1e-c675-361a-ab78-beba6aade405TRACE [GossipStage:1] 2013-07-12 15:17:38,703 Gossiper.java (line 695) Adding state HOST_ID: f24fd5a2-2023-4414-b8f1-88cdd294022fTRACE [Thread-3] 2013-07-12 15:17:38,703 IncomingTcpConnection.java (line 86) eof reading from socket; closingjava.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:134) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:82)TRACE [GossipStage:1] 2013-07-12 15:17:38,703 Gossiper.java (line 695) Adding state STATUS: NORMAL,-9223372036854775808DEBUG [Thread-3] 2013-07-12 15:17:38,705 MessagingService.java (line 816) Reseting version for /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,705 GossipDigestAckVerbHandler.java (line 74) Sending a GossipDigestAck2Message to /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,706 MessagingService.java (line 602) /10.10.101.69 sending GOSSIP_DIGEST_ACK2 to 15@/10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,706 GossipDigestAck2VerbHandler.java (line 38) Received a GossipDigestAck2Message from /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,707 FailureDetector.java (line 168) reporting /10.10.101.200 INFO [GossipStage:1] 2013-07-12 15:17:38,711 Gossiper.java (line 805) Node /10.10.101.200 is now part of the clusterTRACE [GossipStage:1] 2013-07-12 15:17:38,711 Gossiper.java (line 808) Adding endpoint state for /10.10.101.200DEBUG [GossipStage:1] 2013-07-12 15:17:38,711 MessagingService.java (line 378) Resetting pool for /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,721 Gossiper.java (line 764) marking as alive /10.10.101.200DEBUG [GossipStage:1] 2013-07-12 15:17:38,722 Gossiper.java (line 770) removing expire time for endpoint : /10.10.101.200 INFO [GossipStage:1] 2013-07-12 15:17:38,722 Gossiper.java (line 771) InetAddress /10.10.101.200 is now UPTRACE [GossipStage:1] 2013-07-12 15:17:38,723 MessagingService.java (line 826) Assuming current protocol version for /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,724 MessagingService.java (line 826) Assuming current protocol version for /10.10.101.200DEBUG [GossipStage:1] 2013-07-12 15:17:38,727 ReconnectableSnitchHelper.java (line 69) Intiated reconnect to an Internal IP /192.168.4.34 for the /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,727 Gossiper.java (line 775) Notified [org.apache.cassandra.service.StorageService@1531dde4, org.apache.cassandra.service.MigrationManager@7cdeae9a, org.apache.cassandra.locator.ReconnectableSnitchHelper@7f994081, org.apache.cassandra.service.LoadBroadcaster@265a1699]TRACE [MigrationStage:1] 2013-07-12 15:17:38,729 MessagingService.java (line 602) /10.10.101.69 sending MIGRATION_REQUEST to 16@/10.10.101.200DEBUG [WRITE-/10.10.101.200] 2013-07-12 15:17:38,730 OutboundTcpConnection.java (line 289) attempting to connect to /192.168.4.34TRACE [WRITE-/10.10.101.200] 2013-07-12 15:17:38,731 MessagingService.java (line 826) Assuming current protocol version for /192.168.4.34TRACE [GossipStage:1] 2013-07-12 15:17:38,731 SlabAllocator.java (line 103) 1 regions now allocated in org.apache.cassandra.utils.SlabAllocator@3c51d2cbTRACE [GossipStage:1] 2013-07-12 15:17:38,741 MessagingService.java (line 826) Assuming current protocol version for /10.10.101.200TRACE [GossipTasks:1] 2013-07-12 15:17:39,028 Gossiper.java (line 116) My heartbeat is now 21TRACE [GossipTasks:1] 2013-07-12 15:17:39,029 Gossiper.java (line 350) Gossip Digests are : /10.10.101.69:1373613450:21 /10.10.101.200:1373613457:9
Here start disconnect with Node BTRACE [GossipTasks:1] 2013-07-12 15:17:39,030 Gossiper.java (line 505) Sending a GossipDigestSyn to /10.10.101.200 ...TRACE [GossipTasks:1] 2013-07-12 15:17:39,030 MessagingService.java (line 602) /10.10.101.69 sending GOSSIP_DIGEST_SYN to 17@/10.10.101.200TRACE [GossipTasks:1] 2013-07-12 15:17:39,031 Gossiper.java (line 154) Performing status check ...DEBUG [WRITE-/10.10.101.200] 2013-07-12 15:17:39,032 OutboundTcpConnection.java (line 289) attempting to connect to /192.168.4.34TRACE [WRITE-/10.10.101.200] 2013-07-12 15:17:39,032 MessagingService.java (line 826) Assuming current protocol version for /192.168.4.34TRACE [GossipTasks:1] 2013-07-12 15:17:39,032 FailureDetector.java (line 189) PHI for /10.10.101.200 : 0.2814228242733071TRACE [OptionalTasks:1] 2013-07-12 15:17:39,171 MeteredFlusher.java (line 107) memtable memory usage is 239 bytes with 239 liveTRACE [GossipTasks:1] 2013-07-12 15:17:40,033 Gossiper.java (line 116) My heartbeat is now 22TRACE [GossipTasks:1] 2013-07-12 15:17:40,034 Gossiper.java (line 350) Gossip Digests are : /10.10.101.200:1373613457:9 /10.10.101.69:1373613450:22TRACE [GossipTasks:1] 2013-07-12 15:17:40,035 Gossiper.java (line 505) Sending a GossipDigestSyn to /10.10.101.200 ...TRACE [GossipTasks:1] 2013-07-12 15:17:40,035 MessagingService.java (line 602) /10.10.101.69 sending GOSSIP_DIGEST_SYN to 18@/10.10.101.200TRACE [GossipTasks:1] 2013-07-12 15:17:40,036 Gossiper.java (line 154) Performing status check ...TRACE [GossipTasks:1] 2013-07-12 15:17:40,036 FailureDetector.java (line 189) PHI for /10.10.101.200 : 1.1543547328988433TRACE [OptionalTasks:1] 2013-07-12 15:17:40,173 MeteredFlusher.java (line 107) memtable memory usage is 239 bytes with 239 liveTRACE [GossipTasks:1] 2013-07-12 15:17:41,037 Gossiper.java (line 116) My heartbeat is now 23TRACE [GossipTasks:1] 2013-07-12 15:17:41,038 Gossiper.java (line 350) Gossip Digests are : /10.10.101.69:1373613450:23 /10.10.101.200:1373613457:9TRACE [GossipTasks:1] 2013-07-12 15:17:41,038 Gossiper.java (line 505) Sending a GossipDigestSyn to /10.10.101.200 ...TRACE [GossipTasks:1] 2013-07-12 15:17:41,039 MessagingService.java (line 602) /10.10.101.69 sending GOSSIP_DIGEST_SYN to 19@/10.10.101.200TRACE [GossipTasks:1] 2013-07-12 15:17:41,039 Gossiper.java (line 154) Performing status check ...TRACE [GossipTasks:1] 2013-07-12 15:17:41,040 FailureDetector.java (line 189) PHI for /10.10.101.200 : 2.025549463596766TRACE [OptionalTasks:1] 2013-07-12 15:17:41,174 MeteredFlusher.java (line 107) memtable memory usage is 239 bytes with 239 liveTRACE [GossipTasks:1] 2013-07-12 15:17:42,040 Gossiper.java (line 116) My heartbeat is now 24TRACE [GossipTasks:1] 2013-07-12 15:17:42,041 Gossiper.java (line 350) Gossip Digests are : /10.10.101.200:1373613457:9 /10.10.101.69:1373613450:24
Node B Here Connected with Node BTRACE [main] 2013-07-12 15:17:38,361 SlabAllocator.java (line 103) 1 regions now allocated in org.apache.cassandra.utils.SlabAllocator@1aff2b36DEBUG [main] 2013-07-12 15:17:38,362 MigrationManager.java (line 326) Gossiping my schema version 59adb24e-f3cd-3e02-97f0-5b395827453f INFO [main] 2013-07-12 15:17:38,373 MessagingService.java (line 453) Starting Messaging Service on port 7000DEBUG [Thread-2] 2013-07-12 15:17:38,417 IncomingTcpConnection.java (line 75) Connection version 6 from /10.10.101.69DEBUG [Thread-2] 2013-07-12 15:17:38,424 IncomingTcpConnection.java (line 112) Upgrading incoming connection to be compressedDEBUG [Thread-2] 2013-07-12 15:17:38,428 IncomingTcpConnection.java (line 120) Max version for /10.10.101.69 is 6DEBUG [Thread-2] 2013-07-12 15:17:38,428 MessagingService.java (line 809) Setting version 6 for /10.10.101.69DEBUG [Thread-2] 2013-07-12 15:17:38,446 IncomingTcpConnection.java (line 129) set version for /10.10.101.69 to 6TRACE [GossipTasks:1] 2013-07-12 15:17:38,481 Gossiper.java (line 116) My heartbeat is now 9TRACE [GossipTasks:1] 2013-07-12 15:17:38,483 Gossiper.java (line 350) Gossip Digests are : /10.10.101.200:1373613457:9TRACE [GossipTasks:1] 2013-07-12 15:17:38,486 Gossiper.java (line 505) Sending a GossipDigestSyn to /10.10.101.69 ...TRACE [GossipTasks:1] 2013-07-12 15:17:38,487 MessagingService.java (line 602) /10.10.101.200 sending GOSSIP_DIGEST_SYN to 1@/10.10.101.69DEBUG [main] 2013-07-12 15:17:38,515 HintedHandOffManager.java (line 127) Created HHOM instance, registered MBean.TRACE [main] 2013-07-12 15:17:38,550 SSTableReader.java (line 695) Adding cache entry for KeyCacheKey(/data/cassandra/system/local/system-local-ic-1, 6c6f63616c) -> org.apache.cassandra.db.RowIndexEntry@1da50791TRACE [GossipTasks:1] 2013-07-12 15:17:38,564 Gossiper.java (line 154) Performing status check ...DEBUG [WRITE-/10.10.101.69] 2013-07-12 15:17:38,565 OutboundTcpConnection.java (line 289) attempting to connect to /10.10.101.69DEBUG [main] 2013-07-12 15:17:38,569 StorageService.java (line 606) Bootstrap variables: true false false trueTRACE [GossipStage:1] 2013-07-12 15:17:38,590 GossipDigestSynVerbHandler.java (line 40) Received a GossipDigestSynMessage from /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,591 GossipDigestSynVerbHandler.java (line 71) Gossip syn digests are : /10.10.101.69:1373613450:12TRACE [WRITE-/10.10.101.69] 2013-07-12 15:17:38,589 OutboundTcpConnection.java (line 350) Upgrading OutputStream to be compresse
TRACE [GossipStage:1] 2013-07-12 15:17:38,593 Gossiper.java (line 945) requestAll for /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,594 GossipDigestSynVerbHandler.java (line 84) Sending a GossipDigestAckMessage to /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,594 MessagingService.java (line 602) /10.10.101.200 sending GOSSIP_DIGEST_ACK to 3@/10.10.101.69TRACE [main] 2013-07-12 15:17:38,593 SliceQueryFilter.java (line 161) collecting 1 of 2147483647: cql_version:false:5@1373613456604000TRACE [main] 2013-07-12 15:17:38,596 SliceQueryFilter.java (line 161) collecting 1 of 2147483647: data_center:false:3@1373613456604000TRACE [main] 2013-07-12 15:17:38,597 SliceQueryFilter.java (line 161) collecting 1 of 2147483647: gossip_generation:false:4@1373613457323000TRACE [main] 2013-07-12 15:17:38,598 SliceQueryFilter.java (line 161) collecting 1 of 2147483647: host_id:false:16@1373613457298000TRACE [main] 2013-07-12 15:17:38,598 SliceQueryFilter.java (line 161) collecting 1 of 2147483647: partitioner:false:43@1373613456604000TRACE [main] 2013-07-12 15:17:38,599 SliceQueryFilter.java (line 161) collecting 1 of 2147483647: rack:false:4@1373613456604000
TRACE [GossipStage:1] 2013-07-12 15:17:38,614 GossipDigestSynVerbHandler.java (line 40) Received a GossipDigestSynMessage from /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,614 GossipDigestSynVerbHandler.java (line 71) Gossip syn digests are : /10.10.101.69:1373613450:20TRACE [GossipStage:1] 2013-07-12 15:17:38,615 Gossiper.java (line 945) requestAll for /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,616 GossipDigestSynVerbHandler.java (line 84) Sending a GossipDigestAckMessage to /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,616 MessagingService.java (line 602) /10.10.101.200 sending GOSSIP_DIGEST_ACK to 8@/10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,617 GossipDigestAckVerbHandler.java (line 41) Received a GossipDigestAckMessage from /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,618 Gossiper.java (line 681) local heartbeat version 9 greater than 0 for /10.10.101.200TRACE [GossipStage:1] 2013-07-12 15:17:38,622 Gossiper.java (line 695) Adding state INTERNAL_IP: 192.168.4.34TRACE [GossipStage:1] 2013-07-12 15:17:38,623 Gossiper.java (line 695) Adding state RELEASE_VERSION: 1.2.6TRACE [GossipStage:1] 2013-07-12 15:17:38,624 Gossiper.java (line 695) Adding state RACK: novaTRACE [GossipStage:1] 2013-07-12 15:17:38,624 Gossiper.java (line 695) Adding state NET_VERSION: 6TRACE [GossipStage:1] 2013-07-12 15:17:38,625 Gossiper.java (line 695) Adding state RPC_ADDRESS: 0.0.0.0TRACE [GossipStage:1] 2013-07-12 15:17:38,625 Gossiper.java (line 695) Adding state SCHEMA: 59adb24e-f3cd-3e02-97f0-5b395827453fTRACE [GossipStage:1] 2013-07-12 15:17:38,626 Gossiper.java (line 695) Adding state HOST_ID: c071b679-0970-416c-adcf-0f64ed11a592TRACE [GossipStage:1] 2013-07-12 15:17:38,627 Gossiper.java (line 695) Adding state DC: novTRACE [GossipStage:1] 2013-07-12 15:17:38,628 GossipDigestAckVerbHandler.java (line 74) Sending a GossipDigestAck2Message to /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,628 MessagingService.java (line 602) /10.10.101.200 sending GOSSIP_DIGEST_ACK2 to 9@/10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,630 GossipDigestAck2VerbHandler.java (line 38) Received a GossipDigestAck2Message from /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,631 FailureDetector.java (line 168) reporting /10.10.101.69 INFO [GossipStage:1] 2013-07-12 15:17:38,637 Gossiper.java (line 805) Node /10.10.101.69 is now part of the clusterTRACE [GossipStage:1] 2013-07-12 15:17:38,637 Gossiper.java (line 808) Adding endpoint state for /10.10.101.69DEBUG [GossipStage:1] 2013-07-12 15:17:38,638 MessagingService.java (line 378) Resetting pool for /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,640 Gossiper.java (line 764) marking as alive /10.10.101.69DEBUG [GossipStage:1] 2013-07-12 15:17:38,640 Gossiper.java (line 770) removing expire time for endpoint : /10.10.101.69 INFO [GossipStage:1] 2013-07-12 15:17:38,641 Gossiper.java (line 771) InetAddress /10.10.101.69 is now UPTRACE [Thread-2] 2013-07-12 15:17:38,651 IncomingTcpConnection.java (line 86) eof reading from socket; closingjava.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:134) at org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:82)DEBUG [Thread-2] 2013-07-12 15:17:38,658 MessagingService.java (line 816) Reseting version for /10.10.101.69TRACE [MigrationStage:1] 2013-07-12 15:17:38,660 MessagingService.java (line 602) /10.10.101.200 sending MIGRATION_REQUEST to 10@/10.10.101.69TRACE [main] 2013-07-12 15:17:38,654 IntervalTree.java (line 227) Creating IntervalNode from [[java.nio.HeapByteBuffer[pos=0 lim=9 cap=16], java.nio.HeapByteBuffer[pos=0 lim=9 cap=16]](deletedAt=1373613458651999, localDeletion=1373613458)]DEBUG [WRITE-/10.10.101.69] 2013-07-12 15:17:38,662 OutboundTcpConnection.java (line 289) attempting to connect to /192.168.1.7TRACE [WRITE-/10.10.101.69] 2013-07-12 15:17:38,665 MessagingService.java (line 826) Assuming current protocol version for /192.168.1.7DEBUG [GossipStage:1] 2013-07-12 15:17:38,661 ReconnectableSnitchHelper.java (line 69) Intiated reconnect to an Internal IP /192.168.1.7 for the /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,666 Gossiper.java (line 775) Notified [org.apache.cassandra.service.StorageService@76dc09c7, org.apache.cassandra.service.MigrationManager@680624c7, org.apache.cassandra.locator.ReconnectableSnitchHelper@6b58ba2b, org.apache.cassandra.service.LoadBroadcaster@e26941c]TRACE [main] 2013-07-12 15:17:38,668 IntervalTree.java (line 227) Creating IntervalNode from [[java.nio.HeapByteBuffer[pos=0 lim=9 cap=16], java.nio.HeapByteBuffer[pos=0 lim=9 cap=16]](deletedAt=1373613458651999, localDeletion=1373613458)]TRACE [GossipStage:1] 2013-07-12 15:17:38,783 MessagingService.java (line 826) Assuming current protocol version for /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,784 GossipDigestAck2VerbHandler.java (line 38) Received a GossipDigestAck2Message from /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,784 Gossiper.java (line 867) /10.10.101.69local generation 1373613450, remote generation 1373613450TRACE [GossipStage:1] 2013-07-12 15:17:38,785 Gossiper.java (line 887) Ignoring remote version 20 <= 20 for /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,785 GossipDigestAck2VerbHandler.java (line 38) Received a GossipDigestAck2Message from /10.10.101.69TRACE [GossipStage:1] 2013-07-12 15:17:38,785 Gossiper.java (line 867) /10.10.101.69local generation 1373613450, remote generation 1373613450TRACE [GossipStage:1] 2013-07-12 15:17:38,786 Gossiper.java (line 887) Ignoring remote version 20 <= 20 for /10.10.101.69
Here start disconnect with Node B
TRACE [GossipTasks:1] 2013-07-12 15:17:39,567 Gossiper.java (line 350) Gossip Digests are : /10.10.101.69:1373613450:20 /10.10.101.200:1373613457:12TRACE [GossipTasks:1] 2013-07-12 15:17:39,568 Gossiper.java (line 505) Sending a GossipDigestSyn to /10.10.101.69 ...TRACE [GossipTasks:1] 2013-07-12 15:17:39,568 MessagingService.java (line 602) /10.10.101.200 sending GOSSIP_DIGEST_SYN to 12@/10.10.101.69DEBUG [WRITE-/10.10.101.69] 2013-07-12 15:17:39,569 OutboundTcpConnection.java (line 289) attempting to connect to /192.168.1.7TRACE [WRITE-/10.10.101.69] 2013-07-12 15:17:39,569 MessagingService.java (line 826) Assuming current protocol version for /192.168.1.7TRACE [GossipTasks:1] 2013-07-12 15:17:39,570 Gossiper.java (line 154) Performing status check ...TRACE [GossipTasks:1] 2013-07-12 15:17:39,571 FailureDetector.java (line 189) PHI for /10.10.101.69 : 0.8156050370143069
Best Regards,Andy Hu（胡 珊）