[jira] [Commented] (CASSANDRA-6079) Memtables flush is delayed when having a lot of batchlog activity, making node OOM

["Oleg Anastasyev (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-6079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13774708#comment-13774708 ] 

Oleg Anastasyev commented on CASSANDRA-6079:
--------------------------------------------

>From the other hand, if all compaction pool threads are occupied by large CFs, batchlog manager stops replaying batches for a long time, which may not be acceptable.
Rescanning tombstones looks more acceptable to me than delaying of mutations. 

Ideally, batchlog should ensure compaction really happens before waiting. Either running it in its own executor or making some communication with CompactionManager. And if compaction cannot be done at the moment - just go and rescan tobmstones, trading efficiency for consistency.
                
> Memtables flush is delayed when having a lot of batchlog activity, making node OOM
> ----------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-6079
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6079
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Oleg Anastasyev
>            Assignee: Oleg Anastasyev
>            Priority: Minor
>             Fix For: 1.2.11, 2.0.2
>
>         Attachments: NoWaitBatchlogCompaction.diff
>
>
> Both MeteredFlusher and BatchlogManager share the same OptionalTasks thread. So, when batchlog manager processes its tasks no flushes can occur. Even more, batchlog manager waits for batchlog CF compaction to finish.
> On a lot of batchlog activity this prevents memtables from flush for a long time, making the node OOM.
> Fixed this by moving batchlog to its own thread and not waiting for batchlog compaction to finish.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira