[jira] [Created] (CASSANDRA-14592) Reconcile should not be dependent on nowInSec

["Benedict (JIRA)" <ji...@apache.org>]

Benedict created CASSANDRA-14592:
------------------------------------

             Summary: Reconcile should not be dependent on nowInSec
                 Key: CASSANDRA-14592
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14592
             Project: Cassandra
          Issue Type: Bug
            Reporter: Benedict
            Assignee: Benedict
             Fix For: 4.0


To have the arrival time of a mutation on a replica determine the reconciliation priority seems to provide for unintuitive database behaviour.  It seems we should formalise our reconciliation logic in a manner that does not depend on this, and modify our internal APIs to prevent this dependency.
 
Take the following example, where both writes have the same timestamp:
 
Write X with a value A, TTL of 1s
Write Y with a value B, no TTL
 
If X and Y arrive on replicas in < 1s, X and Y are both live, so record Y wins the reconciliation.  The value B appears in the database.
However, if X and Y arrive on replicas in > 1s, X is now (effectively) a tombstone.  This wins the reconciliation race, and NO value is the result.
 
Note that the weirdness of this is more pronounced than it might first appear.  If write X gets stuck in hints for a period on the coordinator to one replica, the value B appears in the database until the hint is replayed.  So now we’re in a very uncertain state - will hints get replayed or not?  If they do, the value B will disappear; if they don’t it won’t.  This is despite a QUORUM of replicas ACKing both writes, and a QUORUM of readers being engaged on read; the database still changes state to the user suddenly at some arbitrary future point in time.
 
It seems to me that a simple solution to this, is to permit TTL’d data to always win a reconciliation against non-TTL’d data (of same timestamp), so that we are consistent across TTLs being transformed into tombstones.
 
4.0 seems like a good opportunity to fix this behaviour, and mention in CHANGES.txt.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org