[GitHub] [pinot] navina commented on a diff in pull request #9863: Add rate limit to Kinesis requests

[GitBox <gi...@apache.org>]

navina commented on code in PR #9863:
URL: https://github.com/apache/pinot/pull/9863#discussion_r1032994002


##########
pinot-plugins/pinot-stream-ingestion/pinot-kinesis/src/main/java/org/apache/pinot/plugin/stream/kinesis/KinesisConsumer.java:
##########
@@ -52,16 +52,19 @@
  */
 public class KinesisConsumer extends KinesisConnectionHandler implements PartitionGroupConsumer {
   private static final Logger LOGGER = LoggerFactory.getLogger(KinesisConsumer.class);
+  public static final long SLEEP_TIME_BETWEEN_REQUESTS = 1000L;
   private final String _streamTopicName;
   private final int _numMaxRecordsToFetch;
   private final ExecutorService _executorService;
   private final ShardIteratorType _shardIteratorType;
+  private int _rpsLimit;

Review Comment:
   I don't think we should allow unlimited. It opens up misconfiguration. Instead we can just ignore with a warning and reset to the default value.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org