Changing IPs of all nodes in a ring

[Rahul Neelakantan <ra...@rahul.be>]

All,
I need to change the IPs of all nodes in my ring in a flash cut, at the same time. Any recommendations on how to do this?

Rahul Neelakantan

Re: Changing IPs of all nodes in a ring

[Robert Coli <rc...@eventbrite.com>]

On Fri, Jul 25, 2014 at 6:30 PM, Rahul Neelakantan <ra...@rahul.be> wrote:

> Do I need to do any move tokens of the new IPs to the old token -1 and the
> removeToken of the old tokens?  I ask this because the old IPs will
> continue to show in gossip info with a status of Normal.
>

If you use auto_bootstrap:false, the old IPs should be removed from gossip,
or no longer matter. However IIRC you are using an ancient version of
Cassandra, so you might see unexpected behavior.

=Rob

Re: Changing IPs of all nodes in a ring

[Rahul Neelakantan <ra...@rahul.be>]

The new IPs are not available before switch time. So I will try the all down method you mentioned. 

Do I need to do any move tokens of the new IPs to the old token -1 and the removeToken of the old tokens?  I ask this because the old IPs will continue to show in gossip info with a status of Normal.



Rahul Neelakantan
678-451-4545

> On Jul 25, 2014, at 9:06 PM, Robert Coli <rc...@eventbrite.com> wrote:
> 
>> On Fri, Jul 25, 2014 at 4:42 PM, Rahul Neelakantan <ra...@rahul.be> wrote:
>> I am ok with taking upto 2 hours of planned downtime. The problem is all the IPs will change at the same time and the previous IPs will no longer be available. So it's either all old IPs or all new IPs. 
> 
> Are the new IPs available before switchover time? If so, you can switch to them with a rolling restart. If not :
> 
> 1) down entire cluster
> 2) change ips in cassandra.yaml, including in seed list
> 3) use auto_bootstrap:false technique, as in blog post, to bring them all back
>  
> =Rob
> 

Re: Changing IPs of all nodes in a ring

[Robert Coli <rc...@eventbrite.com>]

On Fri, Jul 25, 2014 at 4:42 PM, Rahul Neelakantan <ra...@rahul.be> wrote:

> I am ok with taking upto 2 hours of planned downtime. The problem is all
> the IPs will change at the same time and the previous IPs will no longer be
> available. So it's either all old IPs or all new IPs.
>

Are the new IPs available before switchover time? If so, you can switch to
them with a rolling restart. If not :

1) down entire cluster
2) change ips in cassandra.yaml, including in seed list
3) use auto_bootstrap:false technique, as in blog post, to bring them all
back

=Rob

Re: Changing IPs of all nodes in a ring

[Rahul Neelakantan <ra...@rahul.be>]

I am ok with taking upto 2 hours of planned downtime. The problem is all the IPs will change at the same time and the previous IPs will no longer be available. So it's either all old IPs or all new IPs. 

Rahul Neelakantan
678-451-4545

> On Jul 25, 2014, at 7:23 PM, Robert Coli <rc...@eventbrite.com> wrote:
> 
>> On Fri, Jul 25, 2014 at 3:54 PM, Rahul Neelakantan <ra...@rahul.be> wrote:
>> I need to change the IPs of all nodes in my ring in a flash cut, at the same time. Any recommendations on how to do this?
> 
> What are your uptime requirements as you do this?
> 
> Because no, there's no way to change the ip address on all Cassandra nodes in a cluster simultaneously and have it stay available.
> 
> https://engineering.eventbrite.com/changing-the-ip-address-of-a-cassandra-node-with-auto_bootstrapfalse/
> 
> Is how you do it with a rolling restart.
> 
> =Rob
> 

Re: Changing IPs of all nodes in a ring

[Robert Coli <rc...@eventbrite.com>]

On Fri, Jul 25, 2014 at 3:54 PM, Rahul Neelakantan <ra...@rahul.be> wrote:

> I need to change the IPs of all nodes in my ring in a flash cut, at the
> same time. Any recommendations on how to do this?
>

What are your uptime requirements as you do this?

Because no, there's no way to change the ip address on all Cassandra nodes
in a cluster simultaneously and have it stay available.

https://engineering.eventbrite.com/changing-the-ip-address-of-a-cassandra-node-with-auto_bootstrapfalse/

Is how you do it with a rolling restart.

=Rob