[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)

["Carl Yeksigian (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14575204#comment-14575204 ] 

Carl Yeksigian commented on CASSANDRA-6477:
-------------------------------------------

Update on this ticket:

The consistency issue was caused by not using the batch log for the materialized view replica mutations. This meant that we weren't guaranteed that the materialized view would get all of the updates, including tombstones, which is why there were additional entries in the MV.

Adding this fixed the consistency issue, but lengthened the time to execute a materialized view mutation, which exacerbated another issue: lock contention for the partition. Since we need to make sure that the MV and the base table are not updated until we have finished figuring out the new mutations; trying to acquire that lock also backs up the mutation stage and causes the batchlog mutations to fail to run. Much like counters has its own stage, we've move MV mutations to their own stage, as well as batchlog mutations. We also made the tasks reschedule themselves in the stage if they are waiting for a lock and cannot acquire it, so that the stage doesn't deadlock.

These changes have improved MV so that we can overwrite a single key many times and get a consistent result back at the end of the run. We have a [dtest|https://github.com/carlyeks/cassandra-dtest/tree/materialized-views] which tests that scenario.

Collections included in the view are properly handled now as well, so the range tombstones are created for the view properly; all of the types have now been verified by a unit test.

The next steps are to rebase this on top of 8099 for 3.0.

Many thanks to [~tjake] for all of his help looking into these problems.


> Materialized Views (was: Global Indexes)
> ----------------------------------------
>
>                 Key: CASSANDRA-6477
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: API, Core
>            Reporter: Jonathan Ellis
>            Assignee: Carl Yeksigian
>              Labels: cql
>             Fix For: 3.0 beta 1
>
>
> Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing.  However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)