You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Ekaterina Dimitrova (Jira)" <ji...@apache.org> on 2020/12/02 23:00:00 UTC
[jira] [Updated] (CASSANDRA-16261) Prevent unbounded number of
flushing tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ekaterina Dimitrova updated CASSANDRA-16261:
--------------------------------------------
Description:
The cleaner thread is not prevented from queueing an unbounded number of flushing tasks for memtables that are almost empty.
This patch adds a mechanism to track the number of pending flushing
tasks in the memtable cleaner. Above the maximum number (2x the flushing
threads by default), only memtables using at least MCT memory will be
flushed, where MCT stands for Memory Cleanup Threshold.
This patch also fixes a possible problem tracking the memory marked as
"reclaiming" in the memtable allocators and pool. Writes that complete
only after a memtable has been scheduled for flushing, did not report
their memory as reclaiming. Normally this should be a small value of no
consequence, but if the flushing tasks are blocked for a long period,
and there is a sufficient number of writes, or these writes use
a sufficiently large quantity of memory, this would cause the memtable
cleaning algorithm to schedule repeated flushing tasks because the used
memory is always > reclaiming memory + MCT.
was:The cleaner thread is not prevented from queueing an unbounded number of flushing tasks for memtables that are almost empty.
> Prevent unbounded number of flushing tasks
> ------------------------------------------
>
> Key: CASSANDRA-16261
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16261
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Reporter: Ekaterina Dimitrova
> Assignee: Ekaterina Dimitrova
> Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta4
>
>
> The cleaner thread is not prevented from queueing an unbounded number of flushing tasks for memtables that are almost empty.
> This patch adds a mechanism to track the number of pending flushing
> tasks in the memtable cleaner. Above the maximum number (2x the flushing
> threads by default), only memtables using at least MCT memory will be
> flushed, where MCT stands for Memory Cleanup Threshold.
> This patch also fixes a possible problem tracking the memory marked as
> "reclaiming" in the memtable allocators and pool. Writes that complete
> only after a memtable has been scheduled for flushing, did not report
> their memory as reclaiming. Normally this should be a small value of no
> consequence, but if the flushing tasks are blocked for a long period,
> and there is a sufficient number of writes, or these writes use
> a sufficiently large quantity of memory, this would cause the memtable
> cleaning algorithm to schedule repeated flushing tasks because the used
> memory is always > reclaiming memory + MCT.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org