restore with snapshot

[Lee Parker <le...@socialagency.com>]

I am working on finalizing our backup and restore procedures for a cassandra
cluster running on EC2. I understand based on the wiki that in order to
replace a single node, I don't actually need to put data on that node.  I
just need to bootstrap the new node into the cluster and it will get data
from the other nodes.  However, would is speed up the process if that node
already has the data from the node it is replacing?  Also, what do I do if
the entire cluster goes down?  I am planning to snapshot the data each night
for each node.  Should I save the system keyspace snapshots?  Is it
problematic to bring the cluster back up with new ips on each node, but the
same tokens as before?

Lee Parker

Re: restore with snapshot

[Jonathan Ellis <jb...@gmail.com>]

On Mon, Apr 19, 2010 at 2:03 PM, Lee Parker <le...@socialagency.com> wrote:
> I am working on finalizing our backup and restore procedures for a cassandra
> cluster running on EC2. I understand based on the wiki that in order to
> replace a single node, I don't actually need to put data on that node.  I
> just need to bootstrap the new node into the cluster and it will get data
> from the other nodes.  However, would is speed up the process if that node
> already has the data from the node it is replacing?

No, that would speed up repair, but not bootstrap.  See the section on
failure recovery in http://wiki.apache.org/cassandra/Operations

>  Also, what do I do if
> the entire cluster goes down?  I am planning to snapshot the data each night
> for each node.  Should I save the system keyspace snapshots?

Yes, since that is where the token is stored.

>  Is it
> problematic to bring the cluster back up with new ips on each node, but the
> same tokens as before?

No.  Just make sure all the old instances are down, before bringing up
instances w/ new ips.

-Jonathan

Re: restore with snapshot

["Edward M. Goldberg" <em...@edwardmgoldberg.com>]

This is a very important thread for me also.

I have assumed up to this point in time that the Nodes IP is not part of the
equation at all.

I have assumed that a new node with the exact same data store files an
configuration but with a new IP value can replace that same node.

I will test this out ASAP.  I should not have assumes this is true.

I will report my findings.  ( yes this is a EBS SNAP ).

Edward M. Goldberg
http://myCloudWatcher.com/
e.m.g.


On Mon, Apr 19, 2010 at 12:03 PM, Lee Parker <le...@socialagency.com> wrote:

> I am working on finalizing our backup and restore procedures for a
> cassandra cluster running on EC2. I understand based on the wiki that in
> order to replace a single node, I don't actually need to put data on that
> node.  I just need to bootstrap the new node into the cluster and it will
> get data from the other nodes.  However, would is speed up the process if
> that node already has the data from the node it is replacing?  Also, what do
> I do if the entire cluster goes down?  I am planning to snapshot the data
> each night for each node.  Should I save the system keyspace snapshots?  Is
> it problematic to bring the cluster back up with new ips on each node, but
> the same tokens as before?
>
> Lee Parker
>