[jira] [Commented] (CASSANDRA-9318) Bound the number of in-flight requests at the coordinator

["Sergio Bossa (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402137#comment-15402137 ] 

Sergio Bossa commented on CASSANDRA-9318:
-----------------------------------------

Patch rebased to trunk, new links:

| [trunk patch|https://github.com/apache/cassandra/compare/trunk...sbtourist:CASSANDRA-9318-trunk?expand=1] | [testall|https://cassci.datastax.com/view/Dev/view/sbtourist/job/sbtourist-CASSANDRA-9318-trunk-testall/] | [dtest|https://cassci.datastax.com/view/Dev/view/sbtourist/job/sbtourist-CASSANDRA-9318-trunk-dtest/] | [dtest (back-pressure enabled)|https://cassci.datastax.com/view/Dev/view/sbtourist/job/sbtourist-CASSANDRA-9318-bp-true-trunk-dtest/] |

[~Stefania], 

feel free to have another quick look at the new patch (no relevant changes by the way).

[~eduard.tudenhoefner], 

in terms of testing, I've already done some on my own as reported in one of my earlier comments, but I am sure you'll have more to add; the objective of our testing should be:
1) Verify there's no performance impact *at all* when back-pressure is disabled.
2) Verify there's no relevant performance impact when back-pressure is enabled and the cluster is well-behaving.
3) Verify back-pressure works correctly when the cluster is overloaded, and dropped mutations are reduced (how much reduced, it depends on the actual back-pressure configuration, and I hope the {{cassandra.yaml}} comments will help with that, otherwise let me know what's unclear and I'll improve them).

All tests should be run with at least 4 nodes and RF=3, the attached ByteMan rule can be used to simulate "slow replica" scenarios and kick-in back-pressure.

> Bound the number of in-flight requests at the coordinator
> ---------------------------------------------------------
>
>                 Key: CASSANDRA-9318
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9318
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Local Write-Read Paths, Streaming and Messaging
>            Reporter: Ariel Weisberg
>            Assignee: Sergio Bossa
>         Attachments: 9318-3.0-nits-trailing-spaces.patch, backpressure.png, limit.btm, no_backpressure.png
>
>
> It's possible to somewhat bound the amount of load accepted into the cluster by bounding the number of in-flight requests and request bytes.
> An implementation might do something like track the number of outstanding bytes and requests and if it reaches a high watermark disable read on client connections until it goes back below some low watermark.
> Need to make sure that disabling read on the client connection won't introduce other issues.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)