You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Michael Vaknine <mi...@citypath.com> on 2012/01/18 09:11:18 UTC
Incremental backups
Hi,
I am configured to do incremental backups on all my node on the cluster but
it is not working.
In cassandra.yaml : incremental_backups: true
When I check data folder there are some keyspaces that has folder backups
but empty and I suspect this is a folder created in the past when I had
0.7.6 version.
In a new creted Keyspace the folder does not exists.
Does someone know if I need to configure any thing besides cassandra.yaml
for this to work?
Thanks
Michael
Re: Incremental backups
Posted by aaron morton <aa...@thelastpickle.com>.
mmm, they are not included in the snapshot they are probably not used.
Have you dropped an index call 09partition on AttractionCheckins?
In [52]: "".join(chr(int(x+y, 16)) for x,y in zip("3039706172746974696f6e"[0::2], "3039706172746974696f6e"[1::2]))
Out[52]: '09partition'
The simple thing to do is restart cassandra and grep the log for the file names. If they are not there you are safe to blast away.
your upgrade process was correct.
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 19/01/2012, at 11:34 PM, Michael Vaknine wrote:
> When I upgraded I did it in 2 stages.
>
> Upgrade from 0.7.6 to 1.0.0
> Run scrub on each node.
> Run repair on the cluster
> Upgrade to 1.0.3
>
> Is it safe to run scrub again? Because it did not seem to help when I updated it to 1.0.0
> Was there a bug in the scrub process in 1.0.0?
>
> What is the best way to handle this to solve and get a valid environment?
>
> Thanks
> Michael
>
>
> From: aaron morton [mailto:aaron@thelastpickle.com]
> Sent: Thursday, January 19, 2012 12:26 PM
> To: user@cassandra.apache.org
> Subject: Re: Incremental backups
>
> Did you run a scrub as part of the upgrade process ? That will re-write all the sstables and remove the old ones.
>
> If not run a scrub now and it will re-write the data with a -hb- format in the file name.
>
> Cheers
>
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 19/01/2012, at 4:38 PM, Michael Vaknine wrote:
>
>
> I am on 1.0.3 release and it looks like very old files that remained from the upgrade process.
> How can I verify that?
>
> Michael
>
> From: aaron morton [mailto:aaron@thelastpickle.com]
> Sent: Thursday, January 19, 2012 2:22 AM
> To: user@cassandra.apache.org
> Subject: Re: Incremental backups
>
> Looks like you are on a 0.7.X release, which one exactly ? It would be a really good idea to at least be on 8.X, preferably 1.0
>
> Pre 1.0 compacted SSTables were removed during JVM GC, but compacted SSTables have a .Compacted file created so we know they are no longer needed.
>
> These SSTables look like secondary index files. It may be a bug if they are not included in the incremental backups.
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 19/01/2012, at 12:13 AM, Michael Vaknine wrote:
>
>
>
> Hi,
> Thank you for response.
> I did restart for all the nodes and now I can see files in backup folders so It seems like it is working.
> During this process I have noticed to something very strange
>
> In data/City folder there are files that are not created in the snapshot folder (it looks like old orphaned files)
> Is there any process of cassandta that will delete uneeded files I tried to run nodetool cleanup but it did not help.
>
> This is the files:
> -rw-r--r-- 1 cassandra cassandra 230281 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Data.db
> -rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Filter.db
> -rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 1321 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Data.db
> -rw-r--r-- 1 cassandra cassandra 16 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Filter.db
> -rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 2627100 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Data.db
> -rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Filter.db
> -rw-r--r-- 1 cassandra cassandra 20 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 2238358 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Data.db
> -rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Filter.db
> -rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 92 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Data.db
> -rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Filter.db
> -rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 44799 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Data.db
> -rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Filter.db
> -rw-r--r-- 1 cassandra cassandra 196 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 7647 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Data.db
> -rw-r--r-- 1 cassandra cassandra 24 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Filter.db
> -rw-r--r-- 1 cassandra cassandra 96 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Statistics.db
>
>
> Thanks
> Michael
>
> From: Alain RODRIGUEZ [mailto:arodrime@gmail.com]
> Sent: Wednesday, January 18, 2012 10:40 AM
> To: user@cassandra.apache.org
> Subject: Re: Incremental backups
>
> As this option is in the cassandra.yaml file, you might need to perform a restart of your entire cluster (a rolling restart should work).
>
> Hope this will help.
>
> Alain
>
> 2012/1/18 Michael Vaknine <mi...@citypath.com>
> Hi,
> I am configured to do incremental backups on all my node on the cluster but it is not working.
> In cassandra.yaml : incremental_backups: true
> When I check data folder there are some keyspaces that has folder backups but empty and I suspect this is a folder created in the past when I had 0.7.6 version.
> In a new creted Keyspace the folder does not exists.
> Does someone know if I need to configure any thing besides cassandra.yaml for this to work?
>
> Thanks
> Michael
RE: Incremental backups
Posted by Michael Vaknine <mi...@citypath.com>.
When I upgraded I did it in 2 stages.
Upgrade from 0.7.6 to 1.0.0
Run scrub on each node.
Run repair on the cluster
Upgrade to 1.0.3
Is it safe to run scrub again? Because it did not seem to help when I
updated it to 1.0.0
Was there a bug in the scrub process in 1.0.0?
What is the best way to handle this to solve and get a valid environment?
Thanks
Michael
From: aaron morton [mailto:aaron@thelastpickle.com]
Sent: Thursday, January 19, 2012 12:26 PM
To: user@cassandra.apache.org
Subject: Re: Incremental backups
Did you run a scrub as part of the upgrade process ? That will re-write all
the sstables and remove the old ones.
If not run a scrub now and it will re-write the data with a -hb- format in
the file name.
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 19/01/2012, at 4:38 PM, Michael Vaknine wrote:
I am on 1.0.3 release and it looks like very old files that remained from
the upgrade process.
How can I verify that?
Michael
From: aaron morton [mailto:aaron@thelastpickle.com]
Sent: Thursday, January 19, 2012 2:22 AM
To: user@cassandra.apache.org
Subject: Re: Incremental backups
Looks like you are on a 0.7.X release, which one exactly ? It would be a
really good idea to at least be on 8.X, preferably 1.0
Pre 1.0 compacted SSTables were removed during JVM GC, but compacted
SSTables have a .Compacted file created so we know they are no longer
needed.
These SSTables look like secondary index files. It may be a bug if they are
not included in the incremental backups.
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 19/01/2012, at 12:13 AM, Michael Vaknine wrote:
Hi,
Thank you for response.
I did restart for all the nodes and now I can see files in backup folders so
It seems like it is working.
During this process I have noticed to something very strange
In data/City folder there are files that are not created in the snapshot
folder (it looks like old orphaned files)
Is there any process of cassandta that will delete uneeded files I tried to
run nodetool cleanup but it did not help.
This is the files:
-rw-r--r-- 1 cassandra cassandra 230281 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Data.db
-rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Filter.db
-rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Statistics.db
-rw-r--r-- 1 cassandra cassandra 1321 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Data.db
-rw-r--r-- 1 cassandra cassandra 16 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Filter.db
-rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Statistics.db
-rw-r--r-- 1 cassandra cassandra 2627100 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Data.db
-rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Filter.db
-rw-r--r-- 1 cassandra cassandra 20 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Statistics.db
-rw-r--r-- 1 cassandra cassandra 2238358 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Data.db
-rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Filter.db
-rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Statistics.db
-rw-r--r-- 1 cassandra cassandra 92 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Data.db
-rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Filter.db
-rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Statistics.db
-rw-r--r-- 1 cassandra cassandra 44799 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Data.db
-rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Filter.db
-rw-r--r-- 1 cassandra cassandra 196 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Statistics.db
-rw-r--r-- 1 cassandra cassandra 7647 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Data.db
-rw-r--r-- 1 cassandra cassandra 24 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Filter.db
-rw-r--r-- 1 cassandra cassandra 96 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Statistics.db
Thanks
Michael
From: Alain RODRIGUEZ [mailto:arodrime@gmail.com]
Sent: Wednesday, January 18, 2012 10:40 AM
To: user@cassandra.apache.org
Subject: Re: Incremental backups
As this option is in the cassandra.yaml file, you might need to perform a
restart of your entire cluster (a rolling restart should work).
Hope this will help.
Alain
2012/1/18 Michael Vaknine <mi...@citypath.com>
Hi,
I am configured to do incremental backups on all my node on the cluster but
it is not working.
In cassandra.yaml : incremental_backups: true
When I check data folder there are some keyspaces that has folder backups
but empty and I suspect this is a folder created in the past when I had
0.7.6 version.
In a new creted Keyspace the folder does not exists.
Does someone know if I need to configure any thing besides cassandra.yaml
for this to work?
Thanks
Michael
Re: Incremental backups
Posted by aaron morton <aa...@thelastpickle.com>.
Did you run a scrub as part of the upgrade process ? That will re-write all the sstables and remove the old ones.
If not run a scrub now and it will re-write the data with a -hb- format in the file name.
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 19/01/2012, at 4:38 PM, Michael Vaknine wrote:
> I am on 1.0.3 release and it looks like very old files that remained from the upgrade process.
> How can I verify that?
>
> Michael
>
> From: aaron morton [mailto:aaron@thelastpickle.com]
> Sent: Thursday, January 19, 2012 2:22 AM
> To: user@cassandra.apache.org
> Subject: Re: Incremental backups
>
> Looks like you are on a 0.7.X release, which one exactly ? It would be a really good idea to at least be on 8.X, preferably 1.0
>
> Pre 1.0 compacted SSTables were removed during JVM GC, but compacted SSTables have a .Compacted file created so we know they are no longer needed.
>
> These SSTables look like secondary index files. It may be a bug if they are not included in the incremental backups.
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 19/01/2012, at 12:13 AM, Michael Vaknine wrote:
>
>
> Hi,
> Thank you for response.
> I did restart for all the nodes and now I can see files in backup folders so It seems like it is working.
> During this process I have noticed to something very strange
>
> In data/City folder there are files that are not created in the snapshot folder (it looks like old orphaned files)
> Is there any process of cassandta that will delete uneeded files I tried to run nodetool cleanup but it did not help.
>
> This is the files:
> -rw-r--r-- 1 cassandra cassandra 230281 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Data.db
> -rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Filter.db
> -rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 1321 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Data.db
> -rw-r--r-- 1 cassandra cassandra 16 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Filter.db
> -rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 2627100 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Data.db
> -rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Filter.db
> -rw-r--r-- 1 cassandra cassandra 20 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 2238358 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Data.db
> -rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Filter.db
> -rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 92 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Data.db
> -rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Filter.db
> -rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 44799 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Data.db
> -rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Filter.db
> -rw-r--r-- 1 cassandra cassandra 196 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 7647 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Data.db
> -rw-r--r-- 1 cassandra cassandra 24 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Filter.db
> -rw-r--r-- 1 cassandra cassandra 96 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Statistics.db
>
>
> Thanks
> Michael
>
> From: Alain RODRIGUEZ [mailto:arodrime@gmail.com]
> Sent: Wednesday, January 18, 2012 10:40 AM
> To: user@cassandra.apache.org
> Subject: Re: Incremental backups
>
> As this option is in the cassandra.yaml file, you might need to perform a restart of your entire cluster (a rolling restart should work).
>
> Hope this will help.
>
> Alain
>
> 2012/1/18 Michael Vaknine <mi...@citypath.com>
> Hi,
> I am configured to do incremental backups on all my node on the cluster but it is not working.
> In cassandra.yaml : incremental_backups: true
> When I check data folder there are some keyspaces that has folder backups but empty and I suspect this is a folder created in the past when I had 0.7.6 version.
> In a new creted Keyspace the folder does not exists.
> Does someone know if I need to configure any thing besides cassandra.yaml for this to work?
>
> Thanks
> Michael
RE: Incremental backups
Posted by Michael Vaknine <mi...@citypath.com>.
I am on 1.0.3 release and it looks like very old files that remained from
the upgrade process.
How can I verify that?
Michael
From: aaron morton [mailto:aaron@thelastpickle.com]
Sent: Thursday, January 19, 2012 2:22 AM
To: user@cassandra.apache.org
Subject: Re: Incremental backups
Looks like you are on a 0.7.X release, which one exactly ? It would be a
really good idea to at least be on 8.X, preferably 1.0
Pre 1.0 compacted SSTables were removed during JVM GC, but compacted
SSTables have a .Compacted file created so we know they are no longer
needed.
These SSTables look like secondary index files. It may be a bug if they are
not included in the incremental backups.
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 19/01/2012, at 12:13 AM, Michael Vaknine wrote:
Hi,
Thank you for response.
I did restart for all the nodes and now I can see files in backup folders so
It seems like it is working.
During this process I have noticed to something very strange
In data/City folder there are files that are not created in the snapshot
folder (it looks like old orphaned files)
Is there any process of cassandta that will delete uneeded files I tried to
run nodetool cleanup but it did not help.
This is the files:
-rw-r--r-- 1 cassandra cassandra 230281 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Data.db
-rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Filter.db
-rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Statistics.db
-rw-r--r-- 1 cassandra cassandra 1321 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Data.db
-rw-r--r-- 1 cassandra cassandra 16 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Filter.db
-rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Statistics.db
-rw-r--r-- 1 cassandra cassandra 2627100 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Data.db
-rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Filter.db
-rw-r--r-- 1 cassandra cassandra 20 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Statistics.db
-rw-r--r-- 1 cassandra cassandra 2238358 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Data.db
-rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Filter.db
-rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Statistics.db
-rw-r--r-- 1 cassandra cassandra 92 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Data.db
-rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Filter.db
-rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Statistics.db
-rw-r--r-- 1 cassandra cassandra 44799 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Data.db
-rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Filter.db
-rw-r--r-- 1 cassandra cassandra 196 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Statistics.db
-rw-r--r-- 1 cassandra cassandra 7647 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Data.db
-rw-r--r-- 1 cassandra cassandra 24 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Filter.db
-rw-r--r-- 1 cassandra cassandra 96 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Statistics.db
Thanks
Michael
From: Alain RODRIGUEZ [mailto:arodrime@gmail.com]
Sent: Wednesday, January 18, 2012 10:40 AM
To: user@cassandra.apache.org
Subject: Re: Incremental backups
As this option is in the cassandra.yaml file, you might need to perform a
restart of your entire cluster (a rolling restart should work).
Hope this will help.
Alain
2012/1/18 Michael Vaknine <mi...@citypath.com>
Hi,
I am configured to do incremental backups on all my node on the cluster but
it is not working.
In cassandra.yaml : incremental_backups: true
When I check data folder there are some keyspaces that has folder backups
but empty and I suspect this is a folder created in the past when I had
0.7.6 version.
In a new creted Keyspace the folder does not exists.
Does someone know if I need to configure any thing besides cassandra.yaml
for this to work?
Thanks
Michael
Re: Incremental backups
Posted by aaron morton <aa...@thelastpickle.com>.
Looks like you are on a 0.7.X release, which one exactly ? It would be a really good idea to at least be on 8.X, preferably 1.0
Pre 1.0 compacted SSTables were removed during JVM GC, but compacted SSTables have a .Compacted file created so we know they are no longer needed.
These SSTables look like secondary index files. It may be a bug if they are not included in the incremental backups.
Cheers
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 19/01/2012, at 12:13 AM, Michael Vaknine wrote:
> Hi,
> Thank you for response.
> I did restart for all the nodes and now I can see files in backup folders so It seems like it is working.
> During this process I have noticed to something very strange
>
> In data/City folder there are files that are not created in the snapshot folder (it looks like old orphaned files)
> Is there any process of cassandta that will delete uneeded files I tried to run nodetool cleanup but it did not help.
>
> This is the files:
> -rw-r--r-- 1 cassandra cassandra 230281 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Data.db
> -rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Filter.db
> -rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 1321 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Data.db
> -rw-r--r-- 1 cassandra cassandra 16 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Filter.db
> -rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 2627100 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Data.db
> -rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Filter.db
> -rw-r--r-- 1 cassandra cassandra 20 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 2238358 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Data.db
> -rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Filter.db
> -rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 92 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Data.db
> -rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Filter.db
> -rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 44799 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Data.db
> -rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Filter.db
> -rw-r--r-- 1 cassandra cassandra 196 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Statistics.db
> -rw-r--r-- 1 cassandra cassandra 7647 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Data.db
> -rw-r--r-- 1 cassandra cassandra 24 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Filter.db
> -rw-r--r-- 1 cassandra cassandra 96 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Index.db
> -rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Statistics.db
>
>
> Thanks
> Michael
>
> From: Alain RODRIGUEZ [mailto:arodrime@gmail.com]
> Sent: Wednesday, January 18, 2012 10:40 AM
> To: user@cassandra.apache.org
> Subject: Re: Incremental backups
>
> As this option is in the cassandra.yaml file, you might need to perform a restart of your entire cluster (a rolling restart should work).
>
> Hope this will help.
>
> Alain
>
> 2012/1/18 Michael Vaknine <mi...@citypath.com>
> Hi,
> I am configured to do incremental backups on all my node on the cluster but it is not working.
> In cassandra.yaml : incremental_backups: true
> When I check data folder there are some keyspaces that has folder backups but empty and I suspect this is a folder created in the past when I had 0.7.6 version.
> In a new creted Keyspace the folder does not exists.
> Does someone know if I need to configure any thing besides cassandra.yaml for this to work?
>
> Thanks
> Michael
RE: Incremental backups
Posted by Michael Vaknine <mi...@citypath.com>.
Hi,
Thank you for response.
I did restart for all the nodes and now I can see files in backup folders so
It seems like it is working.
During this process I have noticed to something very strange
In data/City folder there are files that are not created in the snapshot
folder (it looks like old orphaned files)
Is there any process of cassandta that will delete uneeded files I tried to
run nodetool cleanup but it did not help.
This is the files:
-rw-r--r-- 1 cassandra cassandra 230281 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Data.db
-rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Filter.db
-rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:57
AttractionCheckins.3039706172746974696f6e-f-157-Statistics.db
-rw-r--r-- 1 cassandra cassandra 1321 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Data.db
-rw-r--r-- 1 cassandra cassandra 16 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Filter.db
-rw-r--r-- 1 cassandra cassandra 27 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 00:58
AttractionCheckins.3039706172746974696f6e-f-158-Statistics.db
-rw-r--r-- 1 cassandra cassandra 2627100 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Data.db
-rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Filter.db
-rw-r--r-- 1 cassandra cassandra 20 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 06:55
Attractions.3039706172746974696f6e-f-1156-Statistics.db
-rw-r--r-- 1 cassandra cassandra 2238358 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Data.db
-rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Filter.db
-rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1157-Statistics.db
-rw-r--r-- 1 cassandra cassandra 92 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Data.db
-rw-r--r-- 1 cassandra cassandra 16 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Filter.db
-rw-r--r-- 1 cassandra cassandra 20 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50
Attractions.3039706172746974696f6e-f-1158-Statistics.db
-rw-r--r-- 1 cassandra cassandra 44799 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Data.db
-rw-r--r-- 1 cassandra cassandra 1936 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Filter.db
-rw-r--r-- 1 cassandra cassandra 196 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 01:25
CityResources.3039706172746974696f6e-f-365-Statistics.db
-rw-r--r-- 1 cassandra cassandra 7647 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Data.db
-rw-r--r-- 1 cassandra cassandra 24 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Filter.db
-rw-r--r-- 1 cassandra cassandra 96 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Index.db
-rw-r--r-- 1 cassandra cassandra 4264 2011-12-06 07:50
CityResources.3039706172746974696f6e-f-366-Statistics.db
Thanks
Michael
From: Alain RODRIGUEZ [mailto:arodrime@gmail.com]
Sent: Wednesday, January 18, 2012 10:40 AM
To: user@cassandra.apache.org
Subject: Re: Incremental backups
As this option is in the cassandra.yaml file, you might need to perform a
restart of your entire cluster (a rolling restart should work).
Hope this will help.
Alain
2012/1/18 Michael Vaknine <mi...@citypath.com>
Hi,
I am configured to do incremental backups on all my node on the cluster but
it is not working.
In cassandra.yaml : incremental_backups: true
When I check data folder there are some keyspaces that has folder backups
but empty and I suspect this is a folder created in the past when I had
0.7.6 version.
In a new creted Keyspace the folder does not exists.
Does someone know if I need to configure any thing besides cassandra.yaml
for this to work?
Thanks
Michael
Re: Incremental backups
Posted by Alain RODRIGUEZ <ar...@gmail.com>.
As this option is in the cassandra.yaml file, you might need to perform a
restart of your entire cluster (a rolling restart should work).
Hope this will help.
Alain
2012/1/18 Michael Vaknine <mi...@citypath.com>
> Hi,****
>
> I am configured to do incremental backups on all my node on the cluster
> but it is not working.****
>
> In cassandra.yaml : incremental_backups: true****
>
> When I check data folder there are some keyspaces that has folder backups
> but empty and I suspect this is a folder created in the past when I had
> 0.7.6 version.****
>
> In a new creted Keyspace the folder does not exists.****
>
> Does someone know if I need to configure any thing besides cassandra.yaml
> for this to work?****
>
> ** **
>
> Thanks****
>
> Michael****
>