Re: [I] Use proper retrying client in Kafka source [hudi]

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on issue #15424:
URL: https://github.com/apache/hudi/issues/15424#issuecomment-4795367160

   This appears to have been addressed in commit 6485fa5c8a79 ([HUDI-4625] Clean up KafkaOffsetGen: introduce retrying KafkaConsumer, #11664, merged Jul 2024).
   
   The sleep-based `do { ... TimeUnit.SECONDS.sleep(10); ... }` loop in `KafkaOffsetGen` has been replaced by `HoodieRetryingKafkaConsumer`, which extends `KafkaConsumer` and wraps `partitionsFor`, `beginningOffsets`, `endOffsets`, `offsetsForTimes`, `listTopics`, and `committed` with `RetryHelper` driven by `hoodie.streamer.source.kafka.{initial.retry.interval.ms,max.retry.interval.ms,max.retry.count}`. `KafkaOffsetGen.fetchPartitionInfos` now does a single `consumer.partitionsFor(topic)` call against the retrying consumer, and the legacy `hoodie.streamer.source.kafka.fetch_partition.time_out` config is logged as deprecated/no-op.
   
   Can this issue be closed as resolved by HUDI-4625?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org