You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 08:18:39 UTC
[I] RLI and update partition path follow up [hudi]
hudi-bot opened a new issue, #16069:
URL: https://github.com/apache/hudi/issues/16069
We found an issue w/ RLI and update partition path cases.
We put in an initial fix to unblock broken master here [https://github.com/apache/hudi/pull/9114]
but there are some follow ups
* Avoid reduce by Key for update partition path = false case
* Fix parallelism for reduce By Key
* Fix RLI MOR and update partition path test case. TestGlobalIndexEnableUpdatePartitions.testUpdatePartitionsThenDelete
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-6473
- Type: Bug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] RLI and update partition path follow up [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #16069: RLI and update partition path follow up
URL: https://github.com/apache/hudi/issues/16069
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] RLI and update partition path follow up [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #16069:
URL: https://github.com/apache/hudi/issues/16069#issuecomment-4795527595
This appears resolved on current master and can likely be closed.
- The `reduceByKey` path that motivated bullets 1 and 2 has been removed. RLI record generation now lives in `RecordIndexMapper.generateRecords()` (`hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/RecordIndexMapper.java`) and emits one record per write-status delegate with no `reduceByKey` — duplicates from update-partition-path are avoided by construction (no-op updates where `currentLocation == newLocation` are skipped, so only the delete-in-old-partition + insert-in-new-partition records flow through, each with distinct keys per WriteStatus).
- Bullet 3 (RLI + MOR coverage of `testUpdatePartitionsThenDelete`) is in place: `TestGlobalIndexEnableUpdatePartitions.getTableTypeAndIndexType()` now includes `(MERGE_ON_READ, RECORD_INDEX)` and `testUpdatePartitionsThenDelete` is parameterized over it.
If the maintainers agree, this can be closed. Reopen if there's a remaining sub-item I missed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org