Full repair use case

[Akshit Jain <ak...@iiitd.ac.in>]

Is there any use case where we need full repair and incremental repair will
not help?
Actually i am performing incremental repair regularly is there any need to
run full repair?

Re: Full repair use case

[Alain RODRIGUEZ <ar...@gmail.com>]

Hi,

Is there any use case where we need full repair and incremental repair will
> not help?


To complete the answer from Jon above, my understanding is that even if
incremental repairs work well for you, if a node loses some repaired
information for some reason (corrupted SSTable, disk failure, ...). The
only reliable way for the node to recover this information and thus to
reduce entropy in the cluster would then be a 'full repair' (Explained in
Alex's talk below). With incremental repair, this data will be ignored as
other nodes consider data have already been repaired. They are probably
some more corner cases as incremental repairs are tricky indeed.

I think repairs have always been a tricky part of Cassandra for operators.
For me at least, and problems comes as the dataset per node grows.

On my side, I have some concerns about incremental as well:

- What about mixing incremental and full repairs, is it safe?
- Some bug report about SSTables marked being repaired when they were not.
I did not dig it but it does not sound nice. Maybe it was fixed though, not
too sure here.
- Anti-compaction generating a lot of SSTable (even worst when using LCS).
- Some more stuff, look at the JIRA and the mailing list archive. I would
definitely try to use one of the latest minor versions when using
incremental repair as some critical fixes were made to this feature.

As Jon pointed out, I think that I would probably go as well for regular
full repairs and if the size is of the dataset is quite big above 50 / 100+
GB I would definitely introduce subranges to limit over-streaming and
improve performances (well explained in Netflix talk below, with benchmarks
to support the talk). It was a topic we discussed during the Cassandra
summit 2016, even if things evolved a bit since then, the content might be
useful to you.

Alexander Dejanovski (pinned to part about incremental repair):
https://youtu.be/1Sz_K8UID6E?t=13m39s
Vinay Chella explained how they were doing this at Netflix:
https://www.youtube.com/watch?v=FrF8wQuXXks

C*heers,

-----------------------
Alain Rodriguez - @arodream - alain@thelastpickle.com
France / Spain

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com


2017-11-22 7:40 GMT+00:00 Jonathan Haddad <jo...@jonhaddad.com>:

> I wouldn’t recommend using incremental repair at all at this time due to
> some bugs that can cause massive overstreaming.
>
> Our advice at TLP is to do subrange repair, and we maintain Reaper to help
> with that: http://cassandra-reaper.io
>
> Jon
>
> On Wed, Nov 22, 2017 at 2:18 AM Akshit Jain <ak...@iiitd.ac.in>
> wrote:
>
>> Is there any use case where we need full repair and incremental repair
>> will not help?
>> Actually i am performing incremental repair regularly is there any need
>> to run full repair?
>>
>

Re: Full repair use case

[Jonathan Haddad <jo...@jonhaddad.com>]

I wouldn’t recommend using incremental repair at all at this time due to
some bugs that can cause massive overstreaming.

Our advice at TLP is to do subrange repair, and we maintain Reaper to help
with that: http://cassandra-reaper.io

Jon
On Wed, Nov 22, 2017 at 2:18 AM Akshit Jain <ak...@iiitd.ac.in> wrote:

> Is there any use case where we need full repair and incremental repair
> will not help?
> Actually i am performing incremental repair regularly is there any need to
> run full repair?
>