You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 05:05:12 UTC
[I] Make a simple method "merge" in HoodieMerge instead of "preCombine" and "combineAndGetUpdateValue" [hudi]
hudi-bot opened a new issue, #15265:
URL: https://github.com/apache/hudi/issues/15265
Historically, this has been 2 different methods with (potentially) different semantics:
* {{preCombine}} is de-duplicating the input batch (before inserting it into the table
* {{combineAndGet}} is used to merge persisted version with the incoming (that could have been previously de-duplicated
I also don't see a reason for us to get hung up on this historical context and we should try to unify these historically (potentially) divergent methods into 1 providing a single avenue of merging records either inside a batch (when de-duping) or when combining persisted one with the incoming.
The merge api should be [associative operation|https://en.wikipedia.org/wiki/Associative_property]: {{f(a, f(b, c)) = f(f(a, b), c)}}
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-4347
- Type: Improvement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make a simple method "merge" in HoodieMerge instead of "preCombine" and "combineAndGetUpdateValue" [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #15265: Make a simple method "merge" in HoodieMerge instead of "preCombine" and "combineAndGetUpdateValue"
URL: https://github.com/apache/hudi/issues/15265
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make a simple method "merge" in HoodieMerge instead of "preCombine" and "combineAndGetUpdateValue" [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #15265:
URL: https://github.com/apache/hudi/issues/15265#issuecomment-4795343372
This appears to have been addressed. The `HoodieRecordMerger` interface in `hudi-common/src/main/java/org/apache/hudi/common/model/HoodieRecordMerger.java` now exposes a single unified `merge(older, newer, recordContext, props)` method. Its Javadoc explicitly states: *"This method converges combineAndGetUpdateValue and precombine from HoodiePayload. It'd be associative operation: f(a, f(b, c)) = f(f(a, b), c)"* — matching this issue's request exactly.
The original unification landed in [HUDI-3378 / RFC-46](https://github.com/apache/hudi/commit/a5bda3ab0c0f) (PR #7345), with subsequent refinements under RFC-101 (PRs #13742, #13865, #13904 — e.g. removal of the legacy outer merge / shouldFlush). Could this issue be closed as resolved? Legacy `HoodieRecordPayload#preCombine` / `combineAndGetUpdateValue` still exist for payload-class backwards compatibility under the `PAYLOAD_BASED_MERGE_STRATEGY_UUID`, but the new merger API is the canonical path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org