You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Tyler Hobbs <ty...@datastax.com> on 2012/12/01 00:21:46 UTC
Re: Data backup and restore
The nodetool snapshot command has keyspace and column family options (from
nodetool --help):
snapshot [keyspaces...] -cf [columnfamilyName] -t [snapshotName] - Take a
snapshot of the optionally specified column family of the specified
keyspaces using optional name snapshotName
On Wed, Nov 28, 2012 at 5:40 AM, Adeel Akbar
<ad...@panasiangroup.com>wrote:
> Dear All,
>
> I have Cassandra 1.1.4 cluster with 2 nodes. I need to take backup and
> restore on staging for testing purpose. I have taken snapshot with below
> mentioned command but It created snapshot on every Keyspace's column
> family. Is there any other way to take backup and restore quick.
>
> /opt/apache-cassandra-1.1.4/bin/nodetool -h localhost snapshot -t
> cassandra_bkup
>
> *Snapshot directory:*
> /var/log/cassandra/data/<KeySpace>/<subfolder/snapshot/cassandra_bkup
>
> --
>
>
> Thanks & Regards
>
> *Adeel** Akbar*
>
--
Tyler Hobbs
DataStax <http://datastax.com/>
Re: Data backup and restore
Posted by aaron morton <aa...@thelastpickle.com>.
I wrote a script to sym link the snapshots together the other day https://github.com/amorton/cass_snapshot_link
I've not really used it in anger yet. That is to say I wrote it for fun and it worked on my mac book. If you use it let me know if it works.
Cheers
A
-----------------
Aaron Morton
Freelance Cassandra Developer
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 5/12/2012, at 6:51 AM, Alain RODRIGUEZ <ar...@gmail.com> wrote:
> Hi Adeel,
>
> I am not sure this is the best solution but we did it this way:
>
> On one production server :
> - $cassandra-cli -f show_schema > schema (show_schema file contains "use <myKeyspaceName>; show schema;")
> - Then open the schema file and remove headers lines, your file must start by "create keyspace..."
>
> On one dev server
> - Copy the schema file
> - $cassandra-cli -f schema
>
> Now you have your keyspace in your dev env.
>
> if you have 1 node dev cluster and RF=total number of node in production, then simply snapshot whatever you want to restore (CF or all keyspace) then copy these files in the CF directories of your test cluster and refresh the new sstables.
>
> if RF < total number of node in production then you need to take the sstable from various nodes and take care not overriding files with the same name while copying files.
>
> I have some bash/parallel-ssh scripts to do this, but not on this computer.
>
> I would be glad learning how other people do this.
>
> Alain
>
>
> 2012/12/4 Yang <te...@gmail.com>
> my guess (from what I learnt on this forum): you probably have to manually create the schema on the new cluster. shutdown new cluster. overwrite the column family files with your backup on all nodes in the new cluster, then boot up.
>
>
> On Tue, Dec 4, 2012 at 8:19 AM, Adeel Akbar <ad...@panasiangroup.com> wrote:
> Hi Tomas,
>
> You are right and now my question is how I restore on test cluster. Do I need to create column families and then copy snapshot on each directory?
>
>
> Thanks & Regards
>
> Adeel Akbar
>
> On 12/4/2012 9:09 PM, Tomas Nunez wrote:
>> Hi
>>
>> I think he was talking about the "fragmentation" of the snapshot. In cassandra 1.0.X all ColumnFamilies are in the same directory, but in cassandra 1.1.X each ColumnFamily is in its own directory, and snapshots of each ColumnFamily are inside this directory.
>>
>> 1.0.X Snapshot directory:
>> /cassandra/data/<KeySpace>/snapshots/<cassandra_bkup_files>
>>
>> 1.1.X Snapshot directory
>> /cassandra/data/<KeySpace>/<ColumnFamily>/snapshots/<cassandra_bkup_files>
>>
>> In 1.0.X you can restore a Keyspace backup by copying just one directory. In 1.1.X it seems you need to copy one directory for each ColumnFamily, which is a little more complicated.
>>
>>
>>
>> 2012/12/1 Tyler Hobbs <ty...@datastax.com>
>> The nodetool snapshot command has keyspace and column family options (from nodetool --help):
>>
>> snapshot [keyspaces...] -cf [columnfamilyName] -t [snapshotName] - Take a snapshot of the optionally specified column family of the specified keyspaces using optional name snapshotName
>>
>>
>> On Wed, Nov 28, 2012 at 5:40 AM, Adeel Akbar <ad...@panasiangroup.com> wrote:
>> Dear All,
>>
>> I have Cassandra 1.1.4 cluster with 2 nodes. I need to take backup and restore on staging for testing purpose. I have taken snapshot with below mentioned command but It created snapshot on every Keyspace's column family. Is there any other way to take backup and restore quick.
>>
>> /opt/apache-cassandra-1.1.4/bin/nodetool -h localhost snapshot -t cassandra_bkup
>>
>> Snapshot directory:
>> /var/log/cassandra/data/<KeySpace>/<subfolder/snapshot/cassandra_bkup
>>
>> --
>>
>> Thanks & Regards
>>
>> Adeel Akbar
>>
>>
>>
>>
>> --
>> Tyler Hobbs
>> DataStax
>>
>>
>>
>>
>> --
>> <Mail Attachment.jpeg>
>> www.groupalia.com
>> Tomàs Núñez
>> IT-Sysprod
>> Tel. + 34 93 159 31 00
>> Fax. + 34 93 396 18 52
>> Llull, 95-97, 2º planta, 08005 Barcelona
>> Skype: tomas.nunez.groupalia
>> tomas.nunez@groupalia.com
>> <Mail Attachment.png> Twitter <Mail Attachment.png> Facebook <Mail Attachment.png> Linkedin
>
>
>
Re: Data backup and restore
Posted by Alain RODRIGUEZ <ar...@gmail.com>.
Hi Adeel,
I am not sure this is the best solution but we did it this way:
On one production server :
- $cassandra-cli -f show_schema > schema (show_schema file contains
"use <myKeyspaceName>; show schema;")
- Then open the schema file and remove headers lines, your file must start
by "create keyspace..."
On one dev server
- Copy the schema file
- $cassandra-cli -f schema
Now you have your keyspace in your dev env.
if you have 1 node dev cluster and RF=total number of node in production,
then simply snapshot whatever you want to restore (CF or all keyspace) then
copy these files in the CF directories of your test cluster and refresh the
new sstables.
if RF < total number of node in production then you need to take the
sstable from various nodes and take care not overriding files with the same
name while copying files.
I have some bash/parallel-ssh scripts to do this, but not on this computer.
I would be glad learning how other people do this.
Alain
2012/12/4 Yang <te...@gmail.com>
> my guess (from what I learnt on this forum): you probably have to manually
> create the schema on the new cluster. shutdown new cluster. overwrite the
> column family files with your backup on all nodes in the new cluster, then
> boot up.
>
>
> On Tue, Dec 4, 2012 at 8:19 AM, Adeel Akbar <adeel.akbar@panasiangroup.com
> > wrote:
>
>> Hi Tomas,
>>
>> You are right and now my question is how I restore on test cluster. Do I
>> need to create column families and then copy snapshot on each directory?
>>
>>
>> Thanks & Regards
>>
>> *Adeel** Akbar*
>> On 12/4/2012 9:09 PM, Tomas Nunez wrote:
>>
>> Hi
>>
>> I think he was talking about the "fragmentation" of the snapshot. In
>> cassandra 1.0.X all ColumnFamilies are in the same directory, but in
>> cassandra 1.1.X each ColumnFamily is in its own directory, and snapshots of
>> each ColumnFamily are inside this directory.
>>
>> 1.0.X Snapshot directory:
>> /cassandra/data/<KeySpace>/snapshots/<cassandra_bkup_files>
>>
>> 1.1.X Snapshot directory
>> /cassandra/data/<KeySpace>/<ColumnFamily>/snapshots/<cassandra_bkup_files>
>>
>> In 1.0.X you can restore a Keyspace backup by copying just one
>> directory. In 1.1.X it seems you need to copy one directory for each
>> ColumnFamily, which is a little more complicated.
>>
>>
>>
>> 2012/12/1 Tyler Hobbs <ty...@datastax.com>
>>
>>> The nodetool snapshot command has keyspace and column family options
>>> (from nodetool --help):
>>>
>>> snapshot [keyspaces...] -cf [columnfamilyName] -t [snapshotName] - Take
>>> a snapshot of the optionally specified column family of the specified
>>> keyspaces using optional name snapshotName
>>>
>>>
>>> On Wed, Nov 28, 2012 at 5:40 AM, Adeel Akbar <
>>> adeel.akbar@panasiangroup.com> wrote:
>>>
>>>> Dear All,
>>>>
>>>> I have Cassandra 1.1.4 cluster with 2 nodes. I need to take backup and
>>>> restore on staging for testing purpose. I have taken snapshot with below
>>>> mentioned command but It created snapshot on every Keyspace's column
>>>> family. Is there any other way to take backup and restore quick.
>>>>
>>>> /opt/apache-cassandra-1.1.4/bin/nodetool -h localhost snapshot -t
>>>> cassandra_bkup
>>>>
>>>> *Snapshot directory:*
>>>> /var/log/cassandra/data/<KeySpace>/<subfolder/snapshot/cassandra_bkup
>>>>
>>>> --
>>>>
>>>>
>>>> Thanks & Regards
>>>>
>>>> *Adeel** Akbar*
>>>>
>>>
>>>
>>>
>>> --
>>> Tyler Hobbs
>>> DataStax <http://datastax.com/>
>>>
>>>
>>
>>
>> --
>> [image: Groupalia] <http://es.groupalia.com/>
>> www.groupalia.com <http://es.groupalia.com/> Tomàs Núñez IT-Sysprod Tel.
>> + 34 93 159 31 00 Fax. + 34 93 396 18 52 Llull, 95-97, 2º planta,
>> 08005 Barcelona Skype: tomas.nunez.groupalia tomas.nunez@groupalia.com<no...@groupalia.com> [image:
>> Twitter] Twitter <http://twitter.com/#%21/groupaliaes> [image:
>> Twitter] Facebook <https://www.facebook.com/GroupaliaEspana> [image:
>> Twitter] Linkedin <http://www.linkedin.com/company/groupalia>
>>
>>
>>
>
Re: Data backup and restore
Posted by Yang <te...@gmail.com>.
my guess (from what I learnt on this forum): you probably have to manually
create the schema on the new cluster. shutdown new cluster. overwrite the
column family files with your backup on all nodes in the new cluster, then
boot up.
On Tue, Dec 4, 2012 at 8:19 AM, Adeel Akbar
<ad...@panasiangroup.com>wrote:
> Hi Tomas,
>
> You are right and now my question is how I restore on test cluster. Do I
> need to create column families and then copy snapshot on each directory?
>
>
> Thanks & Regards
>
> *Adeel** Akbar*
> On 12/4/2012 9:09 PM, Tomas Nunez wrote:
>
> Hi
>
> I think he was talking about the "fragmentation" of the snapshot. In
> cassandra 1.0.X all ColumnFamilies are in the same directory, but in
> cassandra 1.1.X each ColumnFamily is in its own directory, and snapshots of
> each ColumnFamily are inside this directory.
>
> 1.0.X Snapshot directory:
> /cassandra/data/<KeySpace>/snapshots/<cassandra_bkup_files>
>
> 1.1.X Snapshot directory
> /cassandra/data/<KeySpace>/<ColumnFamily>/snapshots/<cassandra_bkup_files>
>
> In 1.0.X you can restore a Keyspace backup by copying just one
> directory. In 1.1.X it seems you need to copy one directory for each
> ColumnFamily, which is a little more complicated.
>
>
>
> 2012/12/1 Tyler Hobbs <ty...@datastax.com>
>
>> The nodetool snapshot command has keyspace and column family options
>> (from nodetool --help):
>>
>> snapshot [keyspaces...] -cf [columnfamilyName] -t [snapshotName] - Take a
>> snapshot of the optionally specified column family of the specified
>> keyspaces using optional name snapshotName
>>
>>
>> On Wed, Nov 28, 2012 at 5:40 AM, Adeel Akbar <
>> adeel.akbar@panasiangroup.com> wrote:
>>
>>> Dear All,
>>>
>>> I have Cassandra 1.1.4 cluster with 2 nodes. I need to take backup and
>>> restore on staging for testing purpose. I have taken snapshot with below
>>> mentioned command but It created snapshot on every Keyspace's column
>>> family. Is there any other way to take backup and restore quick.
>>>
>>> /opt/apache-cassandra-1.1.4/bin/nodetool -h localhost snapshot -t
>>> cassandra_bkup
>>>
>>> *Snapshot directory:*
>>> /var/log/cassandra/data/<KeySpace>/<subfolder/snapshot/cassandra_bkup
>>>
>>> --
>>>
>>>
>>> Thanks & Regards
>>>
>>> *Adeel** Akbar*
>>>
>>
>>
>>
>> --
>> Tyler Hobbs
>> DataStax <http://datastax.com/>
>>
>>
>
>
> --
> [image: Groupalia] <http://es.groupalia.com/>
> www.groupalia.com <http://es.groupalia.com/> Tomàs Núñez IT-Sysprod Tel.
> + 34 93 159 31 00 Fax. + 34 93 396 18 52 Llull, 95-97, 2º planta,
> 08005 Barcelona Skype: tomas.nunez.groupalia tomas.nunez@groupalia.com<no...@groupalia.com> [image:
> Twitter] Twitter <http://twitter.com/#%21/groupaliaes> [image: Twitter]
> Facebook <https://www.facebook.com/GroupaliaEspana> [image: Twitter]
> Linkedin <http://www.linkedin.com/company/groupalia>
>
>
>
Re: Data backup and restore
Posted by Adeel Akbar <ad...@panasiangroup.com>.
Hi Tomas,
You are right and now my question is how I restore on test cluster. Do I
need to create column families and then copy snapshot on each directory?
Thanks & Regards
*Adeel**Akbar*
On 12/4/2012 9:09 PM, Tomas Nunez wrote:
> Hi
>
> I think he was talking about the "fragmentation" of the snapshot. In
> cassandra 1.0.X all ColumnFamilies are in the same directory, but in
> cassandra 1.1.X each ColumnFamily is in its own directory, and
> snapshots of each ColumnFamily are inside this directory.
>
> 1.0.X Snapshot directory:
> /cassandra/data/<KeySpace>/snapshots/<cassandra_bkup_files>
>
> 1.1.X Snapshot directory
> /cassandra/data/<KeySpace>/<ColumnFamily>/snapshots/<cassandra_bkup_files>
>
> In 1.0.X you can restore a Keyspace backup by copying just one
> directory. In 1.1.X it seems you need to copy one directory for each
> ColumnFamily, which is a little more complicated.
>
>
>
> 2012/12/1 Tyler Hobbs <tyler@datastax.com <ma...@datastax.com>>
>
> The nodetool snapshot command has keyspace and column family
> options (from nodetool --help):
>
> snapshot [keyspaces...] -cf [columnfamilyName] -t [snapshotName] -
> Take a snapshot of the optionally specified column family of the
> specified keyspaces using optional name snapshotName
>
>
> On Wed, Nov 28, 2012 at 5:40 AM, Adeel Akbar
> <adeel.akbar@panasiangroup.com
> <ma...@panasiangroup.com>> wrote:
>
> Dear All,
>
> I have Cassandra 1.1.4 cluster with 2 nodes. I need to take
> backup and restore on staging for testing purpose. I have
> taken snapshot with below mentioned command but It created
> snapshot on every Keyspace's column family. Is there any other
> way to take backup and restore quick.
>
> /opt/apache-cassandra-1.1.4/bin/nodetool -h localhost snapshot
> -t cassandra_bkup
>
> _*Snapshot directory:*_
> /var/log/cassandra/data/<KeySpace>/<subfolder/snapshot/cassandra_bkup
>
> --
>
>
> Thanks & Regards
>
> *Adeel**Akbar*
>
>
>
>
> --
> Tyler Hobbs
> DataStax <http://datastax.com/>
>
>
>
>
> --
> Groupalia <http://es.groupalia.com/>
> www.groupalia.com <http://es.groupalia.com/>
> Tomàs Núñez
> IT-Sysprod
> Tel. + 34 93 159 31 00
> Fax. + 34 93 396 18 52
> Llull, 95-97, 2º planta, 08005 Barcelona
> Skype: tomas.nunez.groupalia
> tomas.nunez@groupalia.com <ma...@groupalia.com>
> Twitter Twitter <http://twitter.com/#%21/groupaliaes> Twitter Facebook
> <https://www.facebook.com/GroupaliaEspana> Twitter Linkedin
> <http://www.linkedin.com/company/groupalia>
>
>
Re: Data backup and restore
Posted by Tomas Nunez <to...@groupalia.com>.
Hi
I think he was talking about the "fragmentation" of the snapshot. In
cassandra 1.0.X all ColumnFamilies are in the same directory, but in
cassandra 1.1.X each ColumnFamily is in its own directory, and snapshots of
each ColumnFamily are inside this directory.
1.0.X Snapshot directory:
/cassandra/data/<KeySpace>/snapshots/<cassandra_bkup_files>
1.1.X Snapshot directory
/cassandra/data/<KeySpace>/<ColumnFamily>/snapshots/<cassandra_bkup_files>
In 1.0.X you can restore a Keyspace backup by copying just one directory.
In 1.1.X it seems you need to copy one directory for each ColumnFamily,
which is a little more complicated.
2012/12/1 Tyler Hobbs <ty...@datastax.com>
> The nodetool snapshot command has keyspace and column family options (from
> nodetool --help):
>
> snapshot [keyspaces...] -cf [columnfamilyName] -t [snapshotName] - Take a
> snapshot of the optionally specified column family of the specified
> keyspaces using optional name snapshotName
>
>
> On Wed, Nov 28, 2012 at 5:40 AM, Adeel Akbar <
> adeel.akbar@panasiangroup.com> wrote:
>
>> Dear All,
>>
>> I have Cassandra 1.1.4 cluster with 2 nodes. I need to take backup and
>> restore on staging for testing purpose. I have taken snapshot with below
>> mentioned command but It created snapshot on every Keyspace's column
>> family. Is there any other way to take backup and restore quick.
>>
>> /opt/apache-cassandra-1.1.4/bin/nodetool -h localhost snapshot -t
>> cassandra_bkup
>>
>> *Snapshot directory:*
>> /var/log/cassandra/data/<KeySpace>/<subfolder/snapshot/cassandra_bkup
>>
>> --
>>
>>
>> Thanks & Regards
>>
>> *Adeel** Akbar*
>>
>
>
>
> --
> Tyler Hobbs
> DataStax <http://datastax.com/>
>
>
--
[image: Groupalia] <http://es.groupalia.com/>
www.groupalia.com <http://es.groupalia.com/>Tomàs NúñezIT-SysprodTel. + 34
93 159 31 00 Fax. + 34 93 396 18 52Llull, 95-97, 2º planta, 08005
BarcelonaSkype:
tomas.nunez.groupaliatomas.nunez@groupalia.com<no...@groupalia.com>[image:
Twitter] Twitter <http://twitter.com/#%21/groupaliaes> [image: Twitter]
Facebook <https://www.facebook.com/GroupaliaEspana> [image: Twitter]
Linkedin <http://www.linkedin.com/company/groupalia>