You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "cshuo (via GitHub)" <gi...@apache.org> on 2026/10/26 02:20:40 UTC
[PR] fix: handle empty partitioned record index lookups [hudi]
cshuo opened a new pull request, #20436:
URL: https://github.com/apache/hudi/pull/20436
### Describe the issue this Pull Request addresses
Closes https://github.com/apache/hudi/issues/20435.
Partitioned record-index point lookups throw `ArithmeticException` when querying an empty partition and `NoSuchElementException` when the input key collection is empty. Both lookups should return no locations.
### Summary and Changelog
- Return an empty result when no file slices match the requested data partition or the lookup key set is empty, before selecting a shard.
- Preserve the existing metadata index availability checks and shard read failure propagation.
- Add regression cases to the existing `TestHoodieBackedTableMetadataDataCleanup` class for empty lookups, unavailable metadata indexes, shard selection, and read failures.
### Impact
Partitioned RLI point lookups handle empty partitions and empty key collections without throwing. No API or storage format changes.
### Risk Level
Low. The change adds two early returns to the partition-scoped lookup path. Focused validation passed: 24 tests, zero failures/errors, and zero Checkstyle violations.
```bash
mvn -B -pl hudi-common -am package -Dtest=TestHoodieBackedTableMetadataDataCleanup,TestHoodieBackedTableMetadataBuildPredicate -Dsurefire.failIfNoSpecifiedTests=false -Dsurefire.rerunFailingTestsCount=0 -DskipITs -DskipFTs
```
### Documentation Update
None.
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: handle empty partitioned record index lookups [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #20436:
URL: https://github.com/apache/hudi/pull/20436#issuecomment-6407056744
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/20436?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `5 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 80.82%. Comparing base ([`67b3946`](https://app.codecov.io/gh/apache/hudi/commit/67b39465f405280bf785f95818388d8347884704?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`fd0e43d`](https://app.codecov.io/gh/apache/hudi/commit/fd0e43d9e563d4daeee4dd7a0e2cfefd4632ba3b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/20436?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...pache/hudi/metadata/HoodieBackedTableMetadata.java](https://app.codecov.io/gh/apache/hudi/pull/20436?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fmetadata%2FHoodieBackedTableMetadata.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvbWV0YWRhdGEvSG9vZGllQmFja2VkVGFibGVNZXRhZGF0YS5qYXZh) | 0.00% | [5 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/20436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #20436 +/- ##
============================================
- Coverage 80.82% 80.82% -0.01%
Complexity 35908 35908
============================================
Files 2575 2575
Lines 146762 146766 +4
Branches 17984 17986 +2
============================================
- Hits 118625 118624 -1
- Misses 20002 20007 +5
Partials 8135 8135
```
| [Components](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `84.17% <0.00%> (-0.02%)` | :arrow_down: |
| [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.99% <ø> (ø)` | |
| [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `86.16% <ø> (ø)` | |
| [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `74.20% <ø> (ø)` | |
| [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `79.20% <ø> (ø)` | |
| [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.80% <ø> (ø)` | |
| [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `71.18% <ø> (ø)` | |
| [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `77.29% <ø> (ø)` | |
| [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.48% <ø> (ø)` | |
| [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.41% <ø> (ø)` | |
| [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.01% <ø> (ø)` | |
| [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/20436/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `57.90% <ø> (ø)` | |
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/20436/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/20436/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.94% <0.00%> (-0.02%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/20436?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...pache/hudi/metadata/HoodieBackedTableMetadata.java](https://app.codecov.io/gh/apache/hudi/pull/20436?src=pr&el=tree&filepath=hudi-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fmetadata%2FHoodieBackedTableMetadata.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvbWV0YWRhdGEvSG9vZGllQmFja2VkVGFibGVNZXRhZGF0YS5qYXZh) | `93.87% <0.00%> (-1.23%)` | :arrow_down: |
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: handle empty partitioned record index lookups [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #20436:
URL: https://github.com/apache/hudi/pull/20436#discussion_r4373834306
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -280,9 +280,17 @@ private HoodieData<HoodieRecord<HoodieMetadataPayload>> lookupIndexRecords(Hoodi
List<FileSlice> fileSlicesForDataPartition = fileSlices.stream()
.filter(fileSlice -> HoodieTableMetadataUtil.getDataTablePartitionNameFromFileGroupName(fileSlice.getFileId()).equals(dataTablePartition.get()))
.collect(Collectors.toList());
- // all keys will be from the same shard index so just calculate the first key and reduce partitionFileSlices to 1
+ // The whole-index slice check has already passed; a new data partition may still have no shards.
+ if (fileSlicesForDataPartition.isEmpty()) {
Review Comment:
`TestRecordLevelIndex.testPartitionedRecordLevelIndexRollback` still expects the `partition0` lookup after the rollback to throw `ArithmeticException`, so all four of its cases now fail and `test-spark-java17-java-tests-part2` is red. It should `assertTrue(readRecordIndex(metadata, holder.bulkRecordKeys, HOption.of("partition0")).isEmpty)` instead, and the now-dead `ArithmeticException` catch in `getRecordIndexEntries` can go too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org