[jira] [Comment Edited] (KUDU-1587) Memory-based backpressure is insufficient on seek-bound workloads

["Alexey Serbin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KUDU-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178138#comment-17178138 ] 

Alexey Serbin edited comment on KUDU-1587 at 8/15/20, 2:55 AM:
---------------------------------------------------------------

I implemented the requested functionality with the following changelists:
* [a test scenario to simulate apply queue "overload"|https://gerrit.cloudera.org/#/c/16312/]
* [tracking the state of the apply queue|https://gerrit.cloudera.org/#/c/16332/]
* [controlling the admission of write requests with CoDel-like approach|http://gerrit.cloudera.org:8080/16343]

These follow the approach suggested by [~tlipcon] in the comment above, but with small variation: while deciding whether to reject an incoming write request, the amount of already rejected requests isn't taken into account.  Instead, the criterion is to look for how long the queue has been in the 'overloaded' state: the longer the queue stays overloaded, the greater the probability of rejections.


was (Author: aserbin):
I implemented the requested functionality with the following changelists:
* [a test scenario to simulate apply queue "overload"|https://gerrit.cloudera.org/#/c/16312/]
* [tracking the state of the apply queue|https://gerrit.cloudera.org/#/c/16332/]
* [controlling the admission of write requests with CoDel-like approach|http://gerrit.cloudera.org:8080/16343]


> Memory-based backpressure is insufficient on seek-bound workloads
> -----------------------------------------------------------------
>
>                 Key: KUDU-1587
>                 URL: https://issues.apache.org/jira/browse/KUDU-1587
>             Project: Kudu
>          Issue Type: Bug
>          Components: tserver
>    Affects Versions: 0.10.0
>            Reporter: Todd Lipcon
>            Assignee: Alexey Serbin
>            Priority: Critical
>              Labels: roadmap-candidate
>         Attachments: graph.png, queue-time.png
>
>
> I pushed a uniform random insert workload from a bunch of clients to the point that the vast majority of bloom filters no longer fit in buffer cache, and the compaction had fallen way behind. Thus, every inserted row turns into 40+ seeks (due to non-compact data) and takes 400-500ms. In this kind of workload, the current backpressure (based on memory usage) is insufficient to prevent ridiculously long queues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)