You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brandon Williams (Jira)" <ji...@apache.org> on 2021/09/03 17:07:00 UTC
[jira] [Comment Edited] (CASSANDRA-16718) Changing listen_address
with prefer_local may lead to issues
[ https://issues.apache.org/jira/browse/CASSANDRA-16718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17409614#comment-17409614 ]
Brandon Williams edited comment on CASSANDRA-16718 at 9/3/21, 5:06 PM:
-----------------------------------------------------------------------
This issue boils down to CASSANDRA-10134 loading the ring state, which includes preferred_ip. OTC then queries this directly if it exists and uses it before any changes can be learned. Given this, I'm not sure it even makes sense to store the preferred_ip, since if we try to use it eagerly we'll never be able to learn of the change to it, as this issue exemplifies. I think the best plan is just to remove this optimization and do the reconnection dance every time, which still shouldn't be super-often. WDYT, [~beobal]?
In the meantime, operators may add {{-Dcassandra.load_ring_state=false}} if that's an acceptable workaround.
was (Author: brandon.williams):
This issue boils down to CASSANDRA-10134 loading the ring state, which includes preferred_ip. OTC then queries this directly if it exists and uses it before any changes can be learned. Given this, I'm not sure it even makes sense to store the preferred_ip, since if we try to use it eagerly we'll never be able to learn of the change to it, as this issue exemplifies. I think the best plan is just to remove this optimization and do the reconnection dance every time, which still shouldn't be super-often. WDYT, [~beobal]?
In the meantime, operators may add `-Dcassandra.load_ring_state=false` if that's an acceptable workaround.
> Changing listen_address with prefer_local may lead to issues
> ------------------------------------------------------------
>
> Key: CASSANDRA-16718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16718
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
> Reporter: Jan Karlsson
> Assignee: Brandon Williams
> Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>
> Many container based solution function by assigning new listen_addresses when nodes are stopped. Changing the listen_address is usually as simple as turning off the node and changing the yaml file.
> However, if prefer_local is enabled, I observed that nodes were unable to join the cluster and fail with 'Unable to gossip with any seeds'.
> Trace shows that the changing node will try to communicate with the existing node but the response is never received. I assume it is because the existing node attempts to communicate with the local address during the shadow round.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org