You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by huyle <hu...@springpartners.com> on 2011/12/01 19:26:03 UTC
decommissioned not show being gossipped
Hi,
We have 2 nodes have been decommissioned from the cluster running 1.0.3.
However, the live nodes still making references to the decommissioned nodes
3 days after the nodes were decommissioned. Nodetool does not show the
decommissioned noes. Here are sample log entries:
INFO [GossipStage:1] 2011-12-01 18:20:37,882 Gossiper.java (line 759)
InetAddress /x.x.x.x is now dead.
INFO [GossipStage:1] 2011-12-01 18:20:37,882 StorageService.java (line
1039) Removing token 170141183460469231731687303715884105727 for /x.x.x.x
What might be causing this issue? Any chance this is related to
https://issues.apache.org/jira/browse/CASSANDRA-3243?
Thanks!
Huy
--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/decommissioned-not-show-being-gossipped-tp7051526p7051526.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.
Re: decommissioned not show being gossipped
Posted by huyle <hu...@springpartners.com>.
In our case, we need to execute "unsafeAssassinateEndpoint" from
https://issues.apache.org/jira/browse/CASSANDRA-3337. How do we go about
applying the patch attached to CASSANDRA-3337? We always used the binary
distribution; never had to compile cassandra; so we are not sure of the
proper way to apply the patch and compile new binary. We currently running
1.0.3.
Thanks!
Huy
unsafeAssassinateEndpoint
Brandon Williams wrote
>
> On Thu, Dec 1, 2011 at 1:10 PM, huyle <huyle@> wrote:
>> The clocks are very sync'ed between the nodes as they have ntp running
>> hitting our time servers.
>
> Maybe they weren't 3 days after the token left, which
> https://issues.apache.org/jira/browse/CASSANDRA-2961 requires.
>
> If a node sees the token you can removetoken it, otherwise you'll need
> https://issues.apache.org/jira/browse/CASSANDRA-3337
>
> -Brandon
>
--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/decommissioned-nodes-still-being-gossipped-tp7051526p7055647.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.
Re: decommissioned not show being gossipped
Posted by Brandon Williams <dr...@gmail.com>.
On Thu, Dec 1, 2011 at 1:10 PM, huyle <hu...@springpartners.com> wrote:
> The clocks are very sync'ed between the nodes as they have ntp running
> hitting our time servers.
Maybe they weren't 3 days after the token left, which
https://issues.apache.org/jira/browse/CASSANDRA-2961 requires.
If a node sees the token you can removetoken it, otherwise you'll need
https://issues.apache.org/jira/browse/CASSANDRA-3337
-Brandon
Re: decommissioned not show being gossipped
Posted by huyle <hu...@springpartners.com>.
The clocks are very sync'ed between the nodes as they have ntp running
hitting our time servers.
Huy
--
View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/decommissioned-nodes-still-being-gossipped-tp7051526p7051701.html
Sent from the cassandra-user@incubator.apache.org mailing list archive at Nabble.com.
Re: decommissioned not show being gossipped
Posted by Brandon Williams <dr...@gmail.com>.
On Thu, Dec 1, 2011 at 12:26 PM, huyle <hu...@springpartners.com> wrote:
> Hi,
>
> We have 2 nodes have been decommissioned from the cluster running 1.0.3.
> However, the live nodes still making references to the decommissioned nodes
> 3 days after the nodes were decommissioned. Nodetool does not show the
> decommissioned noes. Here are sample log entries:
How in sync are the clocks in the cluster?
-Brandon