You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Stefania (JIRA)" <ji...@apache.org> on 2015/11/02 01:57:27 UTC
[jira] [Commented] (CASSANDRA-10298) Replaced dead node stayed in
gossip forever
[ https://issues.apache.org/jira/browse/CASSANDRA-10298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984617#comment-14984617 ]
Stefania commented on CASSANDRA-10298:
--------------------------------------
+1, pending CI here:
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10298-2.1-testall/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10298-2.1-dtest/
The code that catches the exception in {{handleStateLeft()}} will be improved by CASSANDRA-10089 where we will return an empty collection in case of no tokens therefore removing the need for a try/catch. We must also remember to check for {{!tokens.isEmpty()}} in {{excise()}} but we'll deal with this in 10089.
A test to reproduce this problem would be helpful to ensure we don't break it again later on. [~dikanggu] are you able to reproduce this regularly and if so what are the exact steps?
> Replaced dead node stayed in gossip forever
> -------------------------------------------
>
> Key: CASSANDRA-10298
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10298
> Project: Cassandra
> Issue Type: Bug
> Reporter: Dikang Gu
> Assignee: Dikang Gu
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-10298.patch
>
>
> The dead node stayed in the nodetool status,
> DN 10.210.165.55 379.76 GB 256 ? null
> And in the log, it throws NPE when trying to remove it.
> {code}
> 2015-09-10_06:41:22.92453 ERROR 06:41:22 Exception in thread Thread[GossipStage:1,5,main]
> 2015-09-10_06:41:22.92454 java.lang.NullPointerException: null
> 2015-09-10_06:41:22.92455 at org.apache.cassandra.utils.UUIDGen.decompose(UUIDGen.java:100)
> 2015-09-10_06:41:22.92455 at org.apache.cassandra.db.HintedHandOffManager.deleteHintsForEndpoint(HintedHandOffManager.java:201)
> 2015-09-10_06:41:22.92455 at org.apache.cassandra.service.StorageService.excise(StorageService.java:1886)
> 2015-09-10_06:41:22.92455 at org.apache.cassandra.service.StorageService.excise(StorageService.java:1902)
> 2015-09-10_06:41:22.92456 at org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1805)
> 2015-09-10_06:41:22.92457 at org.apache.cassandra.service.StorageService.onChange(StorageService.java:1473)
> 2015-09-10_06:41:22.92457 at org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2099)
> 2015-09-10_06:41:22.92457 at org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1009)
> 2015-09-10_06:41:22.92458 at org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1085)
> 2015-09-10_06:41:22.92458 at org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
> 2015-09-10_06:41:22.92458 at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
> 2015-09-10_06:41:22.92459 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_45]
> 2015-09-10_06:41:22.92460 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_45]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)