You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "jhung-lyft (via GitHub)" <gi...@apache.org> on 2026/04/01 18:04:36 UTC
[PR] [FLINK-37918][Connectors/Kinesis] Add GetRecords fetch interval for non-empty records to prevent throttling [flink-connector-aws]
jhung-lyft opened a new pull request, #241:
URL: https://github.com/apache/flink-connector-aws/pull/241
## Summary
The FLIP-27 `KinesisStreamsSource` has no delay between `GetRecords` calls when records are returned, causing per-shard rate limit violations (5 calls/sec/shard). This restores the equivalent of the legacy `SHARD_GETRECORDS_INTERVAL_MILLIS` (default 250ms) using the existing `scheduledFetchTimes` mechanism from FLINK-36947.
### Changes
- Add new config option `source.shard.get-records.interval` (default 250ms) in `KinesisSourceConfigOptions`
- Apply the interval in `KinesisShardSplitReaderBase.scheduleNextFetch()` for non-empty record batches
## Test plan
- [ ] Verify existing unit tests pass
- [ ] Validate against a live Kinesis stream that GetRecords calls stay within the 5 calls/sec/shard limit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37918][Connectors/Kinesis] Add GetRecords fetch interval for non-empty records to prevent throttling [flink-connector-aws]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #241:
URL: https://github.com/apache/flink-connector-aws/pull/241#issuecomment-4171993325
Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37918][Connectors/Kinesis] Add GetRecords fetch interval for non-empty records to prevent throttling [flink-connector-aws]
Posted by "jhung-lyft (via GitHub)" <gi...@apache.org>.
jhung-lyft closed pull request #241: [FLINK-37918][Connectors/Kinesis] Add GetRecords fetch interval for non-empty records to prevent throttling
URL: https://github.com/apache/flink-connector-aws/pull/241
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org