You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 10:33:56 UTC
[I] Multi-writer unique key enforcement for OCC/NBCC [hudi]
hudi-bot opened a new issue, #16593:
URL: https://github.com/apache/hudi/issues/16593
When multiple writers try to insert records of the same record key that does not exsit in the table, both writers can succeed leading to duplicates in the table. In reality, this might not happen in common use cases (e.g., streaming CDC logs to table through Kafka). Still, if we'd like to strictly guarantee key uniqueness for concurrent inserts, we may need the Hudi index to be aware of multiple writers (right now the index only reads committed data), so that we can efficiently identify the same record key across multiple writers.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8142
- Type: Task
- Epic: https://issues.apache.org/jira/browse/HUDI-6640
- Fix version(s):
- 1.1.0
---
## Comments
23/Sep/24 18:55;yihua;Deferring this to Hudi 1.1.0 since the multi-writer unique key enforcement for inserts require fundamental redesign on concurrency control and indexing.;;;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multi-writer unique key enforcement for OCC/NBCC [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on issue #16593:
URL: https://github.com/apache/hudi/issues/16593#issuecomment-3593851260
This issue belongs to a closed JIRA Epic: https://issues.apache.org/jira/browse/HUDI-6640
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Multi-writer unique key enforcement for OCC/NBCC [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on issue #16593:
URL: https://github.com/apache/hudi/issues/16593#issuecomment-5434687972
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8142).
**Findings: still open.**
No commit on `master` references HUDI-8142, so nothing has landed under this ticket's key.
The description is substantive and self-describing, so this is workable as written - what is missing is an owner rather than a specification.
Keeping this open.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org