[jira] [Created] (CASSANDRA-15327) Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds

["Leon Zaruvinsky (Jira)" <ji...@apache.org>]

Leon Zaruvinsky created CASSANDRA-15327:
-------------------------------------------

             Summary: Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds
                 Key: CASSANDRA-15327
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-15327
             Project: Cassandra
          Issue Type: Bug
          Components: Consistency/Bootstrap and Decommission
            Reporter: Leon Zaruvinsky


Hey,

We've come across a scenario in production (noticed on Cassandra 2.2.14) where data that is deleted from Cassandra at consistency {{ALL}} can be resurrected.  I've added a reproduction in a comment.

If a {{delete}} is issued during a range movement (i.e. bootstrap, decommission, move), and {{gc_grace_seconds}} is surpassed before the stream is finished, then the tombstones from the {{delete}} can be purged from the recipient node before the data is streamed. Once the move is complete, the data now exists on the recipient node without a tombstone.

 

We noticed this because our bootstrapping time occasionally exceeds our configured gc_grace_seconds, so we lose the consistency guarantee.  As an operator, it would be great to not have to worry about this edge case.

I've attached a patch that we have tested and successfully used in production, and haven't noticed any ill effects.  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org