You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Haithem Jarraya <a-...@expedia.com> on 2013/08/27 13:29:45 UTC

DSE 3.1.1 null rpc_address in system.peers

Hi All,

We have just installed 3 nodes in QA with DSE version 3.1, I can see in JMX that the version installed is 3.1.1 to be precise. When our application start we see this error message from the java-datastax driver 1.0.2 :

 No rpc_address found for host /X.X.X.3 in node3/X.X.X.3's peers system table. That should not happen but using address /X.X.X.3 instead

when I cqlsh to node3

cqlsh node3 and run SELECT * FROM system.peers; I get null values in the columns relative to node3.

[cqlsh 3.1.6 | Cassandra 1.2.6.1 | CQL spec 3.0.0 | Thrift protocol 19.36.0]
Use HELP for help.
cqlsh> select * from system.peers;

 peer          | data_center | host_id                              | rack | release_version | rpc_address   | schema_version                       | tokens                   | workload
---------------+-------------+--------------------------------------+------+-----------------+---------------+--------------------------------------+--------------------------+----------
 X.X.X.2 |         DC1 | 914206cd-5f89-48ac-aaf9-680be1cd5f6e | RAC1 |         1.2.6.1 | X.X.X.1 | 19457474-58fd-3a52-aeac-912385fea487 | {'-5882736283116946676'} |     null
 X.X.X.3 |         DC1 |                                 null | null |            null |          null |                                 null |                     null |     null
 X.X.X.1 |         DC1 | 8b28f39b-afb8-4927-985c-82119919d018 | RAC1 |         1.2.6.1 | X.X.X.2 | 19457474-58fd-3a52-aeac-912385fea487 |  {'8794980417373438532'} |     null

When I point cqlsh to node2 I get null values in columns relative to node2
cqlsh> select * from system.peers;

 peer          | data_center | host_id                              | rack | release_version | rpc_address   | schema_version                       | tokens                   | workload
---------------+-------------+--------------------------------------+------+-----------------+---------------+--------------------------------------+--------------------------+----------
  X.X.X.2 |         DC1 |                                 null | null |            null |          null |                                 null |                     null |     null
  X.X.X.3 |         DC1 | c3273360-27c2-4042-a962-5416e7829409 | RAC1 |         1.2.6.1 |  X.X.X.3 | 19457474-58fd-3a52-aeac-912385fea487 | {'-7564491331177403445'} |     null
  X.X.X.1 |         DC1 | 8b28f39b-afb8-4927-985c-82119919d018 | RAC1 |         1.2.6.1 |  X.X.X.1 | 19457474-58fd-3a52-aeac-912385fea487 |  {'8794980417373438532'} |     null

I run ccm locally using 3 nodes and I found that when I query the connected node, it does not show up in the peers tables.
./cqlsh 127.0.0.2                                                                                                                                                                                              [12:06:06]
Connected to staging at 127.0.0.2:9160.
[cqlsh 3.1.6 | Cassandra 1.2.6-SNAPSHOT | CQL spec 3.0.0 | Thrift protocol 19.36.0]
Use HELP for help.
cqlsh> select * from system.peers;

 peer      | data_center | host_id                              | rack  | release_version | rpc_address | schema_version                       | tokens
-----------+-------------+--------------------------------------+-------+-----------------+-------------+--------------------------------------+--------------------------
 127.0.0.3 | datacenter1 | bcd0b964-f68c-4686-86e8-77ba63c7aa74 | rack1 |  1.2.6-SNAPSHOT |   127.0.0.3 | b296f03b-6e6e-3b9a-91b5-a5cf1545351c |  {'3074457345618258602'}
 127.0.0.1 | datacenter1 | 019658d6-9ee7-46c1-978f-0441f8bad917 | rack1 |  1.2.6-SNAPSHOT |   127.0.0.1 | b296f03b-6e6e-3b9a-91b5-a5cf1545351c | {'-9223372036854775808'}

I am wondering what is going wrong in our QA installation, why it is showing the connected nodes where is not supposed to?
And what is the work around this? I am wondering if I delete the null entries from the system.peers table would fix the problem, or It might affect other things?

Many Thanks,


Haithem