You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "jtmzheng (via GitHub)" <gi...@apache.org> on 2023/02/06 17:22:16 UTC

[GitHub] [hudi] jtmzheng commented on issue #7829: [SUPPORT] Using monotonically_increasing_id to generate record key causing duplicates on upsert

jtmzheng commented on issue #7829:
URL: https://github.com/apache/hudi/issues/7829#issuecomment-1419449439

   @danny0405 https://apache-hudi.slack.com/archives/C4D716NPQ/p1675378366882569?thread_ts=1675301744.998269&cid=C4D716NPQ 
   
   > issue is, our upsert partitioner/insert partitioner is based on hash of record key. and hence if on re-trigger due to spark task retries, records could move from one insert partition to another and hence could result in data loss or duplication.
   
   It sounds like `insert` would have the same issue based on that description.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org