[GitHub] [incubator-hudi] bhasudha commented on issue #1653: [SUPPORT]: Hudi Deltastreamer OffsetoutofRange Exception reading from Kafka topic (12 partitions)

[GitBox <gi...@apache.org>]

bhasudha commented on issue #1653:
URL: https://github.com/apache/incubator-hudi/issues/1653#issuecomment-632405067


   @prashanthpdesai  quick questions.
   
   Where do you checkpoint the offsets between mini batches? and how do you configure that to deltastreamer? 
   
   Do you have the offset that you used to run this batch ()which failed with out of range) ? 
   If yes can you check if that msg offset is indeed present in the kafka topic (to avoid the possibility of your kafka retention policy deleting those msgs) . You can get smallest offset available for a topic partition by running this kafka command line
   `bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list <broker-ip:9092> --topic <topic-name> --time -2
   `


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org