[GitHub] [beam] aromanenko-dev commented on a change in pull request #16288: [BEAM-13494] Avoid draining Kinesis API limits & CPU when reading from empty or low throughput shard

[GitBox <gi...@apache.org>]

aromanenko-dev commented on a change in pull request #16288:
URL: https://github.com/apache/beam/pull/16288#discussion_r773120367



##########
File path: sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/RateLimitPolicyFactory.java
##########
@@ -67,4 +84,55 @@ public void onSuccess(List<KinesisRecord> records) throws InterruptedException {
       Thread.sleep(delay.get().getMillis());
     }
   }
+
+  /**
+   * Default rate limiter that throttles reading from a shard using an exponential backoff if the
+   * response is empty or if the consumer is throttled by AWS.
+   */
+  class DefaultRateLimiter implements RateLimitPolicy {

Review comment:
       Ii would be great to add a test for this class.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org