You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by S C <as...@outlook.com> on 2013/02/15 03:47:14 UTC
Question on Cassandra Snapshot
I have been looking at incremental backups and snapshots. I have done some experimentation but could not come to a conclusion. Can somebody please help me understanding it right?
/data is my data partition
With incremental_backup turned OFF in Cassandra.yaml - Are all SSTables are under /data/TestKeySpace/ColumnFamily at all times?With incremental_backup turned ON in cassandra.yaml - Are current SSTables under /data/TestKeySpace/ColumnFamily/ with a hardlink to /data/TestKeySpace/ColumnFamily/backups? Lets say I have taken snapshot and moved the /data/TestKeySpace/ColumnFamily/snapshots/<snapshot-name>/*.db to tape, at what point should I be backing up *.db files from /data/TestKeySpace/ColumnFamily/backups directory. Also, should I be deleting the *.db files whose inode matches with the files in the snapshot? Is that a correct approach? I noticed /data/TestKeySpace/ColumnFamily/snapshots/<timestamp>-ColumnFamily/ what are these <timestamp> directories?
Thanks in advance. SC
RE: Question on Cassandra Snapshot
Posted by S C <as...@outlook.com>.
Thank you Aaron.
From: aaron@thelastpickle.com
Subject: Re: Question on Cassandra Snapshot
Date: Mon, 18 Feb 2013 06:37:34 +1300
To: user@cassandra.apache.org
With incremental_backup turned OFF in Cassandra.yaml - Are all SSTables are under /data/TestKeySpace/ColumnFamily at all times?No. They are deleted when they are compacted and no internal operations are referencing them.
With incremental_backup turned ON in cassandra.yaml - Are current SSTables under /data/TestKeySpace/ColumnFamily/ with a hardlink to /data/TestKeySpace/ColumnFamily/backups? Yes, sort of. *All* SSTables ever created are in the backups directory. Not just the ones currently "live".
Lets say I have taken snapshot and moved the /data/TestKeySpace/ColumnFamily/snapshots/<snapshot-name>/*.db to tape, at what point should I be backing up *.db files from /data/TestKeySpace/ColumnFamily/backups directory. Also, should I be deleting the *.db files whose inode matches with the files in the snapshot? Is that a correct approach? Backup all files in the snapshots. There may be non .db extensions files if you use levelled compactionsWhen you are finished with the snapshot delete it. If the inode is not longer referenced from the live data dir it will be deleted.
I noticed /data/TestKeySpace/ColumnFamily/snapshots/<timestamp>-ColumnFamily/ what are these <timestamp> directories?Probably automatic snapshot from dropping KS or CF's
Cheers
-----------------Aaron MortonFreelance Cassandra DeveloperNew Zealand
@aaronmortonhttp://www.thelastpickle.com
On 16/02/2013, at 4:41 AM, S C <as...@outlook.com> wrote:I appreciate any advise or pointers on this.
Thanks in advance.
From: asf11@outlook.com
To: user@cassandra.apache.org
Subject: Question on Cassandra Snapshot
Date: Thu, 14 Feb 2013 20:47:14 -0600
I have been looking at incremental backups and snapshots. I have done some experimentation but could not come to a conclusion. Can somebody please help me understanding it right?
/data is my data partition
With incremental_backup turned OFF in Cassandra.yaml - Are all SSTables are under /data/TestKeySpace/ColumnFamily at all times?With incremental_backup turned ON in cassandra.yaml - Are current SSTables under /data/TestKeySpace/ColumnFamily/ with a hardlink to /data/TestKeySpace/ColumnFamily/backups? Lets say I have taken snapshot and moved the /data/TestKeySpace/ColumnFamily/snapshots/<snapshot-name>/*.db to tape, at what point should I be backing up *.db files from /data/TestKeySpace/ColumnFamily/backups directory. Also, should I be deleting the *.db files whose inode matches with the files in the snapshot? Is that a correct approach? I noticed /data/TestKeySpace/ColumnFamily/snapshots/<timestamp>-ColumnFamily/ what are these <timestamp> directories?
Thanks in advance. SC
Re: Question on Cassandra Snapshot
Posted by aaron morton <aa...@thelastpickle.com>.
> With incremental_backup turned OFF in Cassandra.yaml - Are all SSTables are under /data/TestKeySpace/ColumnFamily at all times?
No.
They are deleted when they are compacted and no internal operations are referencing them.
> With incremental_backup turned ON in cassandra.yaml - Are current SSTables under /data/TestKeySpace/ColumnFamily/ with a hardlink to /data/TestKeySpace/ColumnFamily/backups?
Yes, sort of.
*All* SSTables ever created are in the backups directory.
Not just the ones currently "live".
> Lets say I have taken snapshot and moved the /data/TestKeySpace/ColumnFamily/snapshots/<snapshot-name>/*.db to tape, at what point should I be backing up *.db files from /data/TestKeySpace/ColumnFamily/backups directory. Also, should I be deleting the *.db files whose inode matches with the files in the snapshot? Is that a correct approach?
Backup all files in the snapshots. There may be non .db extensions files if you use levelled compactions
When you are finished with the snapshot delete it. If the inode is not longer referenced from the live data dir it will be deleted.
> I noticed /data/TestKeySpace/ColumnFamily/snapshots/<timestamp>-ColumnFamily/ what are these <timestamp> directories?
Probably automatic snapshot from dropping KS or CF's
Cheers
-----------------
Aaron Morton
Freelance Cassandra Developer
New Zealand
@aaronmorton
http://www.thelastpickle.com
On 16/02/2013, at 4:41 AM, S C <as...@outlook.com> wrote:
> I appreciate any advise or pointers on this.
>
> Thanks in advance.
>
> From: asf11@outlook.com
> To: user@cassandra.apache.org
> Subject: Question on Cassandra Snapshot
> Date: Thu, 14 Feb 2013 20:47:14 -0600
>
> I have been looking at incremental backups and snapshots. I have done some experimentation but could not come to a conclusion. Can somebody please help me understanding it right?
>
> /data is my data partition
>
> With incremental_backup turned OFF in Cassandra.yaml - Are all SSTables are under /data/TestKeySpace/ColumnFamily at all times?
> With incremental_backup turned ON in cassandra.yaml - Are current SSTables under /data/TestKeySpace/ColumnFamily/ with a hardlink to /data/TestKeySpace/ColumnFamily/backups?
> Lets say I have taken snapshot and moved the /data/TestKeySpace/ColumnFamily/snapshots/<snapshot-name>/*.db to tape, at what point should I be backing up *.db files from /data/TestKeySpace/ColumnFamily/backups directory. Also, should I be deleting the *.db files whose inode matches with the files in the snapshot? Is that a correct approach?
> I noticed /data/TestKeySpace/ColumnFamily/snapshots/<timestamp>-ColumnFamily/ what are these <timestamp> directories?
>
> Thanks in advance.
> SC
RE: Question on Cassandra Snapshot
Posted by S C <as...@outlook.com>.
I appreciate any advise or pointers on this.
Thanks in advance.
From: asf11@outlook.com
To: user@cassandra.apache.org
Subject: Question on Cassandra Snapshot
Date: Thu, 14 Feb 2013 20:47:14 -0600
I have been looking at incremental backups and snapshots. I have done some experimentation but could not come to a conclusion. Can somebody please help me understanding it right?
/data is my data partition
With incremental_backup turned OFF in Cassandra.yaml - Are all SSTables are under /data/TestKeySpace/ColumnFamily at all times?With incremental_backup turned ON in cassandra.yaml - Are current SSTables under /data/TestKeySpace/ColumnFamily/ with a hardlink to /data/TestKeySpace/ColumnFamily/backups? Lets say I have taken snapshot and moved the /data/TestKeySpace/ColumnFamily/snapshots/<snapshot-name>/*.db to tape, at what point should I be backing up *.db files from /data/TestKeySpace/ColumnFamily/backups directory. Also, should I be deleting the *.db files whose inode matches with the files in the snapshot? Is that a correct approach? I noticed /data/TestKeySpace/ColumnFamily/snapshots/<timestamp>-ColumnFamily/ what are these <timestamp> directories?
Thanks in advance. SC