[jira] [Commented] (CASSANDRA-11500) Obsolete MV entry may not be properly deleted

["Paulo Motta (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-11500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119461#comment-16119461 ] 

Paulo Motta commented on CASSANDRA-11500:
-----------------------------------------

The virtual cell proposal is pretty clever and looks like it would solve most outstanding issues, but I'm a bit concerned about adding new structures to the storage engine to deal with materialized-specific issues.

While I agree we should do if it's our only choice, we should explore alternatives which reuse existing structures if possible to avoid introducing feature-specific stuff into the storage engine.

Looking back at the original scenario which motivated this ticket (CASSANDRA-11500):
{noformat}
CREATE TABLE t (k int PRIMARY KEY, a int, b int);
CREATE MATERIALIZED VIEW mv AS SELECT * FROM t WHERE k IS NOT NULL AND a IS NOT NULL PRIMARY KEY (k, a);

INSERT INTO t(k, a, b) VALUES (1, 1, 1) USING TIMESTAMP 0;
UPDATE t USING TIMESTAMP 4 SET b = 2 WHERE k = 1;
UPDATE t USING TIMESTAMP 2 SET a = 2 WHERE k = 1;

SELECT * FROM mv WHERE k = 1; // This currently return 2 entries, the old (invalid) and the new one
{noformat}

It seems to me that the problem here is applying the definition for standard tables that if a single column is live, then the whole row is live, which does not need to be the case for MV where we can guarantee a view entry will always contain row-level liveness info.

We could solve this  by introducing a "strict " flag to the row liveness info, which has the following semantic:
- A strict row is only live iff it's row level liveness info is live, regardless of the liveness of its columns

Materialized views rows would have this flag set and perform deletions with its max primary key timestamp (instead of max timestamp of all keys), and this would solve the issue above by ensuring the row {{(1, 1)@(liveness@0, deleted@2)=(b=2@4)}} would not be live.

In addition to solving the original problem we would not create the second problem in the ticket description of updates to the view primary key with a smaller timestamp to be shadowed by a shadowable tombstone using the max timestamp of a non-PK column. In this approach the shadowing tombstone mechanism would still be orthogonal to the strict liveness and working as it is today.

This mechanism alone would not solve all other problems but at least the ones described in this ticket description with minimal change in the storage engine, let's now go through the other issues to see how we could solve them:

*View row expires too fast with unselected base column (CASSANDRA-13127)*