[GitHub] [beam] kennknowles opened a new issue, #18483: Client-side throttling for Datastore connector

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #18483:
URL: https://github.com/apache/beam/issues/18483

   The Datastore connector currently has exponential backoff on errors, which is good. But it does not do any other throttling of its write load in response to errors; once a request succeeds, it resumes writing as quickly as it can.
   
   Write loads will be more stable and more likely to compete if the client throttles itself in the event that it receives high rates of errors from the Datastore service; specifically 
   https://landing.google.com/sre/book/chapters/handling-overload.html#client-side-throttling-a7sYUg is a technique that Google has had success with on other services.
   
   We (Datastore) have a patch in progress to add this behaviour to the connector.
   
   
   Imported from Jira [BEAM-2556](https://issues.apache.org/jira/browse/BEAM-2556). Original Jira may contain additional context.
   Reported by: cphipps.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org