Re: [I] Late arriving records and global index with partition path update set to true [hudi]

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on issue #14882:
URL: https://github.com/apache/hudi/issues/14882#issuecomment-4795276398

   Triage note: this scenario appears to be handled in current `master`.
   
   When `hoodie.{bloom,simple,record}.index.update.partition.path=true` and an incoming record's resolved partition differs from the existing record's partition, `HoodieIndexUtils.mergeForPartitionUpdatesAndDeletionsIfNeeded` merges incoming with existing via `BufferedRecordMerger.finalMerge(existing, incoming)`, which honors precombine/event-time ordering. If the incoming has a lower ordering value, the merge result equals the existing record, `inferPartitionPath` returns the existing partition, and the record is routed back to the existing location as a no-op update — no movement to a new partition occurs.
   
   This is also covered by `TestGlobalIndexEnableUpdatePartitions.testPartitionChanges` (parameterized across COW/MoR × {GLOBAL_SIMPLE, GLOBAL_BLOOM, RECORD_INDEX}), which explicitly asserts "5th batch: late update all to p4; discarded" (added in PR #8736 / HUDI-5724). Late-arriving global-index deletes have analogous coverage in `TestGlobalRecordLevelIndex.testRLIForDeletesWithHoodieIsDeletedColumn`.
   
   If you can reproduce the original misbehavior on a recent release (>= 0.14.x) with event-time ordering configured, please share the writer config + repro and we can reopen. Otherwise this can likely be closed as fixed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org