[jira] Commented: (CASSANDRA-644) Provide way to remove nodes from gossip entirely

["Jaakko Laine (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793178#action_12793178 ] 

Jaakko Laine commented on CASSANDRA-644:
----------------------------------------

> connecting to a _different_ cluster, but the old one notices it is back and starts gossiping to it again!

Actually this cannot happen since gossip SYN carries cluster ID and only messages from the same cluster are handled.

I think removing fat clients automatically is OK. We could run a check periodically and remove all nodes from gossiper that have been silent for certain period of time, and do not have token associated with it. I'm not sure in what kind of situations this kind of clients are used, but if a node does not participate in storage and has been dead for an hour, there should be no harm in removing it. It will be back in gossip anyway immediately if it reappears. Cleaning up fat clients automatically would prevent gossip data from growing "rapidly" if there are many clients coming and going from different IP addresses.

As for "proper" storage nodes: These are likely to be more stable than clients, so probably it is best to have removetoken and/or other manual commands to take care of this.


> Provide way to remove nodes from gossip entirely
> ------------------------------------------------
>
>                 Key: CASSANDRA-644
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-644
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Tools
>            Reporter: Jonathan Ellis
>            Assignee: Jaakko Laine
>            Priority: Minor
>             Fix For: 0.9
>
>
> As reported in CASSANDRA-634, "Now that we're gossiping about dead nodes as well, gossip digest continues to grow without boundary when nodes come and go. This information will never disappear as it will be propagated to new nodes no matter how old and obsolete it is. To counter this, we need some mechanism to (1) either remove dead node from endpointstateinfo or (2) at some point stop to gossip about it, or both."
> This is also seen when using "fat clients" that participate in the gossip ring; if a client leaves and does not come back it stays in the gossip forever.  (This can be confusing if the client does start up again, connecting to a _different_ cluster, but the old one notices it is back and starts gossiping to it again!)
> I would prefer to leave management of these things explicit; 3 days is long enough that the fat client problem in particular needs another solution, and if it needs another solution then that can become the only solution. :)
> So I would be in favor of removeToken clearing out gossip entries, and also adding a command to remove an endpoint from the gossip ring that does not have a token associated with it (like fat clients).  A command to ask "what are all the known gossip hosts" would also be useful, since nodeprobe ring only includes nodes w/ tokens.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.