You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Sebastian Estevez (JIRA)" <ji...@apache.org> on 2015/03/04 05:55:05 UTC

[jira] [Comment Edited] (CASSANDRA-7122) Replacement nodes have null entries in system.peers

    [ https://issues.apache.org/jira/browse/CASSANDRA-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14319146#comment-14319146 ] 

Sebastian Estevez edited comment on CASSANDRA-7122 at 3/4/15 4:54 AM:
----------------------------------------------------------------------

Hello, seeing a similar issue on a 2.0.11 cluster(2.0.11). Is it possible that this might still be an issue?

Here is the table content for system.peers (with the tokens column omitted):
{code}
 peer          | data_center | host_id                              | preferred_ip | rack  | release_version | rpc_address | schema_version                       | workload
---------------+-------------+--------------------------------------+--------------+-------+-----------------+-------------+--------------------------------------+-----------
  127.0.0.56 |   Cassandra | 7fc420af-d284-48aa-ba58-2b71508995ab |         null | rack1 |       2.0.11.83 |     0.0.0.0 | 9dd12e7e-07cf-395b-9d9c-05028bdedd04 | Cassandra
  127.0.0.248 |   Cassandra |                                 null |         null |  null |            null |        null |                                 null |      null
 127.0.0.246 |   Cassandra |                                 null |         null |  null |            null |        null |                                 null |      null
   127.0.0.63 |   Cassandra |                                 null |         null |  null |            null |        null |                                 null |      null
  127.0.0.104 |   Cassandra |                                 null |         null |  null |            null |        null |                                 null |      null
   127.0.0.41 |   Cassandra | b6a9ea10-cf58-452f-9003-87c1e183c888 |         null | rack1 |       2.0.11.83 |     0.0.0.0 | 9dd12e7e-07cf-395b-9d9c-05028bdedd04 | Cassandra
  127.0.0.48 |   Cassandra | 5f33e073-5c71-4ba1-9d9f-7d2b396bd916 |         null | rack1 |       2.0.11.83 |     0.0.0.0 | 9dd12e7e-07cf-395b-9d9c-05028bdedd04 | Cassandra
  127.0.0.58 |   Cassandra | 0f6c891a-b8e5-486c-a3f2-04c62d368925 |         null | rack1 |       2.0.11.82 |     0.0.0.0 | 8223f2ad-fddd-3c7f-a0f3-837c598ca96b |      null
   127.0.0.66 |   Cassandra | 8b8f0c8b-bfb5-4256-8da9-c10296f6c4e7 |         null | rack1 |       2.0.11.83 |     0.0.0.0 | 9dd12e7e-07cf-395b-9d9c-05028bdedd04 | Cassandra
   127.0.0.72 |   Cassandra |                                 null |         null |  null |            null |        null |                                 null |      null
{code}

Out of the 10 rows, the 5 with null host_id were the nodes removed on Monday. 


was (Author: sebastian.estevez@datastax.com):
Hello, seeing a similar issue on a 2.0.11 cluster(DSE 4.6). Is it possible that this might still be an issue?

Here is the table content for system.peers (with the tokens column omitted):
{code}
 peer          | data_center | host_id                              | preferred_ip | rack  | release_version | rpc_address | schema_version                       | workload
---------------+-------------+--------------------------------------+--------------+-------+-----------------+-------------+--------------------------------------+-----------
  127.0.0.56 |   Cassandra | 7fc420af-d284-48aa-ba58-2b71508995ab |         null | rack1 |       2.0.11.83 |     0.0.0.0 | 9dd12e7e-07cf-395b-9d9c-05028bdedd04 | Cassandra
  127.0.0.248 |   Cassandra |                                 null |         null |  null |            null |        null |                                 null |      null
 127.0.0.246 |   Cassandra |                                 null |         null |  null |            null |        null |                                 null |      null
   127.0.0.63 |   Cassandra |                                 null |         null |  null |            null |        null |                                 null |      null
  127.0.0.104 |   Cassandra |                                 null |         null |  null |            null |        null |                                 null |      null
   127.0.0.41 |   Cassandra | b6a9ea10-cf58-452f-9003-87c1e183c888 |         null | rack1 |       2.0.11.83 |     0.0.0.0 | 9dd12e7e-07cf-395b-9d9c-05028bdedd04 | Cassandra
  127.0.0.48 |   Cassandra | 5f33e073-5c71-4ba1-9d9f-7d2b396bd916 |         null | rack1 |       2.0.11.83 |     0.0.0.0 | 9dd12e7e-07cf-395b-9d9c-05028bdedd04 | Cassandra
  127.0.0.58 |   Cassandra | 0f6c891a-b8e5-486c-a3f2-04c62d368925 |         null | rack1 |       2.0.11.82 |     0.0.0.0 | 8223f2ad-fddd-3c7f-a0f3-837c598ca96b |      null
   127.0.0.66 |   Cassandra | 8b8f0c8b-bfb5-4256-8da9-c10296f6c4e7 |         null | rack1 |       2.0.11.83 |     0.0.0.0 | 9dd12e7e-07cf-395b-9d9c-05028bdedd04 | Cassandra
   127.0.0.72 |   Cassandra |                                 null |         null |  null |            null |        null |                                 null |      null
{code}

Out of the 10 rows, the 5 with null host_id were the nodes removed on Monday. 

> Replacement nodes have null entries in system.peers
> ---------------------------------------------------
>
>                 Key: CASSANDRA-7122
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7122
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Richard Low
>            Assignee: Brandon Williams
>             Fix For: 1.2.17, 2.0.9, 2.1 rc1
>
>         Attachments: 7122.txt
>
>
> If a node is replaced with -Dcassandra.replace_address, the new node has mostly null entries in system.peers:
> {code}
> > select * from system.peers;
>  peer      | data_center | host_id | rack | release_version | rpc_address | schema_version | tokens
> -----------+-------------+---------+------+-----------------+-------------+----------------+--------------------------
>  127.0.0.3 |        null |    null | null |            null |        null |           null | {'-3074457345618258602'}
> {code}
> To reproduce, simply kill a node and replace it. The entries are correctly populated if the replacement node is restarted but they are never populated if it isn't.
> I can think of at least two bad consequences of this:
> 1. Drivers like Datastax java-driver use the peers table to find the rpc_address and location info of a node. If the entires are null it assumes rpc_address=ip and the node is in the local DC.
> 2. When using GossipingPropertyFileSnitch and node won't persist the DC/rack of another node so may not be able to locate it during restarts.
> I reproduced in 1.2.15 but from inspection it looks to be present in 1.2.16 and 2.0.7.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)