You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 12:45:51 UTC
[I] Remove Reliance on Relations in the IncrementalFileIndex [hudi]
hudi-bot opened a new issue, #17104:
URL: https://github.com/apache/hudi/issues/17104
Right now, we use methods on an mor incremental relation in the IncrementalFileIndex. We should create shared util methods instead of using the relation. This will help us for when we get rid of the relations
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-9623
- Type: Improvement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Remove Reliance on Relations in the IncrementalFileIndex [hudi]
Posted by "rangareddy (via GitHub)" <gi...@apache.org>.
rangareddy commented on issue #17104:
URL: https://github.com/apache/hudi/issues/17104#issuecomment-5391459526
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-9623).
**Findings: confirmed, and the coupling is structural rather than incidental.**
The relation is a **constructor parameter** of the file index, not just something called from a method. `hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieIncrementalFileIndex.scala`:
```scala
:41 mergeOnReadIncrementalRelation: MergeOnReadIncrementalRelation)
...
:47 val fileSlices = mergeOnReadIncrementalRelation.listFileSplits(partitionFilters, dataFilters).toSeq.flatMap(
:57 val fileSlices = mergeOnReadIncrementalRelation.listFileSplits(Seq.empty, Seq.empty)
:75 mergeOnReadIncrementalRelation.getRequiredFilters
```
So the index cannot be constructed at all without a relation - which is precisely the coupling that has to go before the relations can be removed.
Context that raises the priority: the legacy read paths are already being retired. PR #14060 (`refactor(spark): Remove glob paths and deprecate read paths support`, merged 2025-11-25) states read-path removal is a follow-up. That puts this refactor on the critical path for that work rather than making it optional cleanup, and the three call sites above are a small, well-defined extraction (`listFileSplits` and `getRequiredFilters` into shared utils).
Related: #17129 (HUDI-9677) reports a config that is only honoured by the legacy `IncrementalRelationV1`, which is the same retirement story from the config side.
Keeping this open.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org