[jira] [Commented] (CASSANDRA-9143) Improving consistency of repairAt field across replicas

["Blake Eggleston (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-9143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435649#comment-15435649 ] 

Blake Eggleston commented on CASSANDRA-9143:
--------------------------------------------

Since sstables compacted since the beginning of a repair are excluded from anticompaction, normal compaction is enough to create large inconsistencies of the data each node considers repaired. This will cause repaired data to be considered unrepaired, which will cause a lot of unnecessary streaming on the next repair.

At a high level, the least complicated solution I’ve come up with is:

Add a ‘pending repair’ bucket to the existing repaired and unrepaired sstable buckets. We do the anticompaction up front, but put the anticompacted data into the pending bucket. From here, the repair proceeds normally against the pending sstables, with the streamed sstables also going into the pending buckets. Once all nodes have completed streaming, the pending sstables are moved into the repaired bucket, or back into unrepaired if there’s a failure.

This should keep each replica’s notion of what’s repaired identical, and minimize over streaming caused by large chunks of repaired data being classified as unrepaired.

> Improving consistency of repairAt field across replicas 
> --------------------------------------------------------
>
>                 Key: CASSANDRA-9143
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9143
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: sankalp kohli
>            Priority: Minor
>
> We currently send an anticompaction request to all replicas. During this, a node will split stables and mark the appropriate ones repaired. 
> The problem is that this could fail on some replicas due to many reasons leading to problems in the next repair. 
> This is what I am suggesting to improve it. 
> 1) Send anticompaction request to all replicas. This can be done at session level. 
> 2) During anticompaction, stables are split but not marked repaired. 
> 3) When we get positive ack from all replicas, coordinator will send another message called markRepaired. 
> 4) On getting this message, replicas will mark the appropriate stables as repaired. 
> This will reduce the window of failure. We can also think of "hinting" markRepaired message if required. 
> Also the stables which are streaming can be marked as repaired like it is done now. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)