You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/14 03:31:18 UTC

[GitHub] [hudi] boneanxs commented on pull request #6098: [HUDI-4389] Make HoodieStreamingSink idempotent

boneanxs commented on PR #6098:
URL: https://github.com/apache/hudi/pull/6098#issuecomment-1278431648

   We found if 2 spark structure streaming jobs write data to the same HUDI table(enable multi-writer), the error could throw making one writer skip batches...
   
   ```java
   2022-10-12 14:33:25 INFO  HoodieActiveTimeline:170 Loaded instants upto : Option{val=[20221012143212859__commit__COMPLETED]}
   2022-10-12 14:33:25 WARN  HoodieStreamingSink:108 Skipping already completed batch 0 in query 498e6b86-7401-44d8-9b02-2b16fade25b8
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org