[jira] [Comment Edited] (CASSANDRA-6335) Hints broken for nodes that change broadcast address

["Rick Branson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820415#comment-13820415 ] 

Rick Branson edited comment on CASSANDRA-6335 at 11/12/13 8:41 PM:
-------------------------------------------------------------------

It looks like it stalls the hint delivery thread pool indefinitely. Restarting the node frees it up and it starts delivering hints again. I'm rolling this cluster to workaround the issue.

EDIT: disregard the "stalling" issue... there were just a large # of hints that needed to be delivered to a specific node (because it was down for a bit for some maintenance) clogging up the pool and confusing the issue.


was (Author: rbranson):
It looks like it stalls the hint delivery thread pool indefinitely. Restarting the node frees it up and it starts delivering hints again. I'm rolling this cluster to workaround the issue.

> Hints broken for nodes that change broadcast address
> ----------------------------------------------------
>
>                 Key: CASSANDRA-6335
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6335
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Rick Branson
>
> When a node changes it's broadcast address, the transition process works properly, but hints that are destined for it can't be delivered because of the address change. It produces an exception:
> java.lang.AssertionError: Missing host ID for 10.1.60.22
>         at org.apache.cassandra.service.StorageProxy.writeHintForMutation(StorageProxy.java:598)
>         at org.apache.cassandra.service.StorageProxy$5.runMayThrow(StorageProxy.java:567)
>         at org.apache.cassandra.service.StorageProxy$HintRunnable.run(StorageProxy.java:1679)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)



--
This message was sent by Atlassian JIRA
(v6.1#6144)