[jira] [Commented] (CASSANDRA-12783) Break up large MV mutations to prevent OOMs

["Kurt Greaves (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16256404#comment-16256404 ] 

Kurt Greaves commented on CASSANDRA-12783:
------------------------------------------

So an update here. Seeing as this is going into 4 it's only getting a limited amount of my attention at the moment so I thought I'd put up my most recent version here and where it's up to (so I don't forget, and if anyone has any insights).

|[trunk|https://github.com/apache/cassandra/compare/trunk...kgreav:12783-trunk]|[dtest|https://github.com/apache/cassandra-dtest/compare/master...kgreav:12783]|

So at the moment it more or less works however the dtest fails occasionally due to the corrupt value length, even though it shouldn't happen. You can even keep the test dir, go run the query again, and get the correct results/no corrupt values.

There's still a few TODO's, and needs some performance testing for comparison + to see if it's worth chunking mutations in {{store}}. LegacyBatchlogMigrator needs a few more fixes to support the new version (it's a copy of the old 3.0 version).

Also I think ideally the {{BatchlogManager.markBatchesActive()}} should probably use the batchlog to mark the batches active so that we can guarantee if they are marked they will eventually all be marked, but at the moment that's not possible because the batchlog uses a GCGS of 0. It's probably safe to set a GCGS higher than 0, it will just end with us reading some tombstones when we want to replay the batchlog - but really it's not a great solution. Special casing local batchlog writes is also another option (as there's no point consulting GCGS for a local batchlog write) however we'd need to also store the table the batch was destined for as part of the batchlog - which seems a bit over the top just for the sake of MV's.

I also considered using an unlogged batch, because for a local write it should be enough, however I couldn't figure out any nice and clean way to issue an UNLOGGED batch, and according to CASSANDRA-9283 a LOGGED batch should essentially be unlogged if it's for a single partition, but I haven't yet found these "optimisations" that guarantee this and even still using a logged batch doesn't work because GCGS=0.

> Break up large MV mutations to prevent OOMs
> -------------------------------------------
>
>                 Key: CASSANDRA-12783
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12783
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Local Write-Read Paths, Materialized Views
>            Reporter: Carl Yeksigian
>            Assignee: Kurt Greaves
>             Fix For: 4.x
>
>
> We only use the code path added in CASSANDRA-12268 for the view builder because otherwise we would break the contract of the batchlog, where some mutations may be written and pushed out before the whole batch log has been saved.
> We would need to ensure that all of the updates make it to the batchlog before allowing the batchlog manager to try to replay them, but also before we start pushing out updates to the paired replicas.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org