[jira] [Commented] (KUDU-1954) Improve maintenance manager behavior in heavy write workload

["wangningito (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KUDU-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113291#comment-17113291 ] 

wangningito commented on KUDU-1954:
-----------------------------------

Hey Todd, I'm wondering if it is possible to split the flush ops thread out from maintenace_manager_num_threads, I sometimes found the memrowset memories exceed the flush_threshold_mb a lot but the maintenance threads were all hold by the long compaction. If the tablet is very big, the compaction may takes minutes.  
Maybe flush could keeps working in order to reduce the memory usage. 

> Improve maintenance manager behavior in heavy write workload
> ------------------------------------------------------------
>
>                 Key: KUDU-1954
>                 URL: https://issues.apache.org/jira/browse/KUDU-1954
>             Project: Kudu
>          Issue Type: Improvement
>          Components: perf, tserver
>    Affects Versions: 1.3.0
>            Reporter: Todd Lipcon
>            Priority: Major
>         Attachments: mm-trace.png
>
>
> During the investigation in [this doc|https://docs.google.com/document/d/1U1IXS1XD2erZyq8_qG81A1gZaCeHcq2i0unea_eEf5c/edit] I found a few maintenance-manager-related issues during heavy writes:
> - we don't schedule flushes until we are already in "backpressure" realm, so we spent most of our time doing backpressure
> - even if we configure N maintenance threads, we typically are only using ~50% of those threads due to the scheduling granularity
> - when we do hit the "memory-pressure flush" threshold, all threads quickly switch to flushing, which then brings us far beneath the threshold
> - long running compactions can temporarily starve flushes
> - high volume of writes can starve compactions



--
This message was sent by Atlassian Jira
(v8.3.4#803005)