incremental backups not completely working?

[md...@orange.com]

  Today I configured incremental backups in a test node which already has some data on it, 
and I found that backups are not created for STTables created by a compact:

mddione@life:~/src/works/orange/Cassandra$ sudo find /var/lib/cassandra/data/one_cf
/var/lib/cassandra/data/one_cf
/var/lib/cassandra/data/one_cf/cf_1
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Statistics.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Data.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Index.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Index.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Data.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/backups
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Index.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Data.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Index.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Data.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Index.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Statistics.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Data.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Statistics.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Data.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Statistics.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Statistics.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Index.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Statistics.db

mddione@life:~/src/works/orange/Cassandra$ nodetool compact

mddione@life:~/src/works/orange/Cassandra$ sudo find /var/lib/cassandra/data/one_cf
/var/lib/cassandra/data/one_cf
/var/lib/cassandra/data/one_cf/cf_1
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Data.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Statistics.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Index.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/backups
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Index.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Data.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Index.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Data.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Index.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Statistics.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Data.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Statistics.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Data.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Statistics.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Statistics.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Filter.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-CompressionInfo.db
/var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Index.db

as you can see, neither SSTables 4 or 6 have backups; both were created by compactions.

--
Marcos Dione
SysAdmin
Astek Sud-Est
pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo
04 97 12 62 45 - mdione.ext@orange.com



_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

Re: incremental backups not completely working?

[Tyler Hobbs <ty...@datastax.com>]

Incremental backups are only triggered when new data is written to disk,
such as a memtable being flushed or data being streamed in from a repair or
move.  Compaction does not create any new data, so there's no need to back
up the result.

On Mon, Sep 3, 2012 at 8:45 AM, <md...@orange.com> wrote:

>
>   Today I configured incremental backups in a test node which already has
> some data on it,
> and I found that backups are not created for STTables created by a compact:
>
> mddione@life:~/src/works/orange/Cassandra$ sudo find
> /var/lib/cassandra/data/one_cf
> /var/lib/cassandra/data/one_cf
> /var/lib/cassandra/data/one_cf/cf_1
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Statistics.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Data.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Index.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Index.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Filter.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Data.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/backups
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Index.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Data.db
>
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Filter.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Index.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Data.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Index.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Filter.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Filter.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Statistics.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Data.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Statistics.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Data.db
>
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Statistics.db
>
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Statistics.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Filter.db
>
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Index.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-4-Filter.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-5-Statistics.db
>
> mddione@life:~/src/works/orange/Cassandra$ nodetool compact
>
> mddione@life:~/src/works/orange/Cassandra$ sudo find
> /var/lib/cassandra/data/one_cf
> /var/lib/cassandra/data/one_cf
> /var/lib/cassandra/data/one_cf/cf_1
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Data.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Statistics.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Index.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-Filter.db
> /var/lib/cassandra/data/one_cf/cf_1/one_cf-cf_1-hd-6-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/backups
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Index.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Data.db
>
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Filter.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Index.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Data.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Index.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Filter.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Filter.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Statistics.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Data.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Statistics.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Data.db
>
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-2-Statistics.db
>
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-5-Statistics.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-1-Filter.db
>
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-CompressionInfo.db
> /var/lib/cassandra/data/one_cf/cf_1/backups/one_cf-cf_1-hd-3-Index.db
>
> as you can see, neither SSTables 4 or 6 have backups; both were created by
> compactions.
>
> --
> Marcos Dione
> SysAdmin
> Astek Sud-Est
> pour FT/TGPF/OPF/PORTAIL/DOP/HEBEX @ Marco Polo
> 04 97 12 62 45 - mdione.ext@orange.com
>
>
>
>
> _________________________________________________________________________________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
> recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> France Telecom - Orange decline toute responsabilite si ce message a ete
> altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged
> information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and
> delete this message and its attachments.
> As emails may be altered, France Telecom - Orange is not liable for
> messages that have been modified, changed or falsified.
> Thank you.
>
>


-- 
Tyler Hobbs
DataStax <http://datastax.com/>