[jira] [Commented] (TEPHRA-269) Protect the Transaction Manager against misconfigured clients using rate limits

["Andreas Neumann (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEPHRA-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423226#comment-16423226 ] 

Andreas Neumann commented on TEPHRA-269:
----------------------------------------

Nice design! A couple of comments:
 * would it make sense to blacklist a misbehaving client, and require to explicitly un-blacklist it before it can start transactions again? If you unthrottle it after its bucket is empty, it will just start misbehaving again. 
 * if we have a way to generate a report, would it make sense to include other metrics such as number or percentage of invalid transactions compared to all of its transactions? Th.at would help ops to identify systemically misbehaving clients
 * An alternative to the _long nextStartTxnMillis(String clientId);_ could be _void startPermitted(clientId) throws RateLImitExceededException_, and that exception can include the number of outstanding operations as well as the required delay. 
 * In the LeakyBucket implementation, the nextOperationMillis state variable appears unnecessary, as it is derived from outstandingOps.
 * When cleaning up the LeakyBuckets, I don't think the delta is required before expiring an entry. Or why is it needed?
 * The getRateLimits() client API should return a Java object and not a String. Even if you use Json format to serialize this over the Thrift wire, that should not be exposed to the client. 

> Protect the Transaction Manager against misconfigured clients using rate limits
> -------------------------------------------------------------------------------
>
>                 Key: TEPHRA-269
>                 URL: https://issues.apache.org/jira/browse/TEPHRA-269
>             Project: Tephra
>          Issue Type: New Feature
>          Components: core
>            Reporter: Poorna Chandra
>            Assignee: Poorna Chandra
>            Priority: Major
>             Fix For: 0.14.0-incubating
>
>         Attachments: Ensure QoS in Transaction Manager by rate limiting client requests.pdf
>
>
> We have seen cases where misconfigured clients can overwhelm the system by making expensive requests (like invalidating transactions). By the time admins figure out there is a misconfigured client and take corrective action, thousands of invalid transactions can be created in a very short period of time. It would be useful to have a way to limit the number of invalid transactions that a client can create in given time.
> I'll send out a design document for this soon.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)