You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Michael Theroux <mt...@yahoo.com> on 2015/06/16 01:17:53 UTC
Nodetool ring and "Replicas" after 1.2 upgrade
Hello,
We (finally) have just upgraded from Cassandra 1.1 to Cassandra 1.2.19. Everything appears to be up and running normally, however, we have noticed unusual output from nodetool ring. There is a new (to us) field "Replicas" in the nodetool output, and this field, seemingly at random, is changing from 2 to 3 and back to 2.
We are using the byte ordered partitioner (we hash our own keys), and have a replication factor of 3. We are also on AWS and utilize the Ec2snitch on a single Datacenter.
Other calls appear to be normal. "nodetool getEndpoints" returns the proper endpoints when querying various keys, nodetool ring and status return that all nodes appear healthy.
Anyone have any hints on what maybe happening, or if this is a problem we should be concerned with?
Thanks,-Mike
Re: Nodetool ring and "Replicas" after 1.2 upgrade
Posted by Michael Theroux <mt...@yahoo.com>.
After looking at the cassandra code a little, I believe this is not really an issue.
After the upgrade to 1.2, we still see the issue described in this bug I filed:
https://issues.apache.org/jira/browse/CASSANDRA-5264
The "Replicas" is calculated by adding up the effective ownership of all the nodes, and chopping off the remainder. So if your effective ownership is 299.99%, it appears the code will will report the number of replicas as 2. This might become reliably 3 after I complete running repairs after upgrade.
Thanks for you time,-Mike
From: Alain RODRIGUEZ <ar...@gmail.com>
To: user@cassandra.apache.org; Michael Theroux <mt...@yahoo.com>
Sent: Tuesday, June 16, 2015 4:43 PM
Subject: Re: Nodetool ring and "Replicas" after 1.2 upgrade
Hi Michael,
I barely can access internet right now and was not able to check outputs on my computer, yet first thing that come to my mind is that since 1.2.x (and vnodes) I use rather nodetool status instead. What is the "nodetool status" output ?
Also did you try to specify the keyspace ? Since RF is a per keyspace value, maybe this would help.
Other than that, I don't have any idea. I don't remember anything similar, but it was a while ago. I have to ask... Why staying so much behind the current "stable" / "production ready" version ?
C*heers,
Alain
2015-06-16 14:57 GMT+02:00 Michael Theroux <mt...@yahoo.com>:
Thanks Jason,
No errors in the log. Also the nodes do have a consistent schema for the keyspace (although this was a problem during the upgrade that we resolved using the procedure specified here: https://wiki.apache.org/cassandra/FAQ#schema_disagreement).
-Mike
From: Jason Wee <pe...@gmail.com>
To: user@cassandra.apache.org; Michael Theroux <mt...@yahoo.com>
Sent: Tuesday, June 16, 2015 12:07 AM
Subject: Re: Nodetool ring and "Replicas" after 1.2 upgrade
maybe check the system.log to see if there is any exception and/or error? check as well if they are having consistent schema for the keyspace?
hth
jason
On Tue, Jun 16, 2015 at 7:17 AM, Michael Theroux <mt...@yahoo.com> wrote:
Hello,
We (finally) have just upgraded from Cassandra 1.1 to Cassandra 1.2.19. Everything appears to be up and running normally, however, we have noticed unusual output from nodetool ring. There is a new (to us) field "Replicas" in the nodetool output, and this field, seemingly at random, is changing from 2 to 3 and back to 2.
We are using the byte ordered partitioner (we hash our own keys), and have a replication factor of 3. We are also on AWS and utilize the Ec2snitch on a single Datacenter.
Other calls appear to be normal. "nodetool getEndpoints" returns the proper endpoints when querying various keys, nodetool ring and status return that all nodes appear healthy.
Anyone have any hints on what maybe happening, or if this is a problem we should be concerned with?
Thanks,-Mike
Re: Nodetool ring and "Replicas" after 1.2 upgrade
Posted by Alain RODRIGUEZ <ar...@gmail.com>.
Hi Michael,
I barely can access internet right now and was not able to check outputs on
my computer, yet first thing that come to my mind is that since 1.2.x (and
vnodes) I use rather nodetool status instead. What is the "nodetool status"
output ?
Also did you try to specify the keyspace ? Since RF is a per keyspace
value, maybe this would help.
Other than that, I don't have any idea. I don't remember anything similar,
but it was a while ago. I have to ask... Why staying so much behind the
current "stable" / "production ready" version ?
C*heers,
Alain
2015-06-16 14:57 GMT+02:00 Michael Theroux <mt...@yahoo.com>:
> Thanks Jason,
>
> No errors in the log. Also the nodes do have a consistent schema for the
> keyspace (although this was a problem during the upgrade that we resolved
> using the procedure specified here:
> https://wiki.apache.org/cassandra/FAQ#schema_disagreement).
>
> -Mike
>
>
> ------------------------------
> *From:* Jason Wee <pe...@gmail.com>
> *To:* user@cassandra.apache.org; Michael Theroux <mt...@yahoo.com>
> *Sent:* Tuesday, June 16, 2015 12:07 AM
> *Subject:* Re: Nodetool ring and "Replicas" after 1.2 upgrade
>
> maybe check the system.log to see if there is any exception and/or error?
> check as well if they are having consistent schema for the keyspace?
>
> hth
>
> jason
>
>
>
> On Tue, Jun 16, 2015 at 7:17 AM, Michael Theroux <mt...@yahoo.com>
> wrote:
>
> Hello,
>
> We (finally) have just upgraded from Cassandra 1.1 to Cassandra 1.2.19.
> Everything appears to be up and running normally, however, we have noticed
> unusual output from nodetool ring. There is a new (to us) field "Replicas"
> in the nodetool output, and this field, seemingly at random, is changing
> from 2 to 3 and back to 2.
>
> We are using the byte ordered partitioner (we hash our own keys), and have
> a replication factor of 3. We are also on AWS and utilize the Ec2snitch on
> a single Datacenter.
>
> Other calls appear to be normal. "nodetool getEndpoints" returns the
> proper endpoints when querying various keys, nodetool ring and status
> return that all nodes appear healthy.
>
> Anyone have any hints on what maybe happening, or if this is a problem we
> should be concerned with?
>
> Thanks,
> -Mike
>
>
>
>
>
Re: Nodetool ring and "Replicas" after 1.2 upgrade
Posted by Michael Theroux <mt...@yahoo.com>.
Thanks Jason,
No errors in the log. Also the nodes do have a consistent schema for the keyspace (although this was a problem during the upgrade that we resolved using the procedure specified here: https://wiki.apache.org/cassandra/FAQ#schema_disagreement).
-Mike
From: Jason Wee <pe...@gmail.com>
To: user@cassandra.apache.org; Michael Theroux <mt...@yahoo.com>
Sent: Tuesday, June 16, 2015 12:07 AM
Subject: Re: Nodetool ring and "Replicas" after 1.2 upgrade
maybe check the system.log to see if there is any exception and/or error? check as well if they are having consistent schema for the keyspace?
hth
jason
On Tue, Jun 16, 2015 at 7:17 AM, Michael Theroux <mt...@yahoo.com> wrote:
Hello,
We (finally) have just upgraded from Cassandra 1.1 to Cassandra 1.2.19. Everything appears to be up and running normally, however, we have noticed unusual output from nodetool ring. There is a new (to us) field "Replicas" in the nodetool output, and this field, seemingly at random, is changing from 2 to 3 and back to 2.
We are using the byte ordered partitioner (we hash our own keys), and have a replication factor of 3. We are also on AWS and utilize the Ec2snitch on a single Datacenter.
Other calls appear to be normal. "nodetool getEndpoints" returns the proper endpoints when querying various keys, nodetool ring and status return that all nodes appear healthy.
Anyone have any hints on what maybe happening, or if this is a problem we should be concerned with?
Thanks,-Mike
Re: Nodetool ring and "Replicas" after 1.2 upgrade
Posted by Jason Wee <pe...@gmail.com>.
maybe check the system.log to see if there is any exception and/or error?
check as well if they are having consistent schema for the keyspace?
hth
jason
On Tue, Jun 16, 2015 at 7:17 AM, Michael Theroux <mt...@yahoo.com>
wrote:
> Hello,
>
> We (finally) have just upgraded from Cassandra 1.1 to Cassandra 1.2.19.
> Everything appears to be up and running normally, however, we have noticed
> unusual output from nodetool ring. There is a new (to us) field "Replicas"
> in the nodetool output, and this field, seemingly at random, is changing
> from 2 to 3 and back to 2.
>
> We are using the byte ordered partitioner (we hash our own keys), and have
> a replication factor of 3. We are also on AWS and utilize the Ec2snitch on
> a single Datacenter.
>
> Other calls appear to be normal. "nodetool getEndpoints" returns the
> proper endpoints when querying various keys, nodetool ring and status
> return that all nodes appear healthy.
>
> Anyone have any hints on what maybe happening, or if this is a problem we
> should be concerned with?
>
> Thanks,
> -Mike
>
>