You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "WarFox (via GitHub)" <gi...@apache.org> on 2023/03/23 15:00:05 UTC

[GitHub] [hudi] WarFox commented on issue #8258: [SUPPORT] Deltastreamer errors ingesting kafka-streams topics (transactional / exactly_once producers)

WarFox commented on issue #8258:
URL: https://github.com/apache/hudi/issues/8258#issuecomment-1481350315

   More observations on this
   
   - It is observed that the problem occurs when transaction marker is at the last offset for consumption
   - The ingestion is successful when we have non-transactional messages after the transaction marker offset
   
   As seen in the logs attached above, we suspect that the underlying Spark KafkaRDD library is confused about the transaction marker when it is the last record in the topic


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org