You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 04:41:42 UTC
[I] Partition specific pointed lookup/reading strategy for metadata table [hudi]
hudi-bot opened a new issue, #15007:
URL: https://github.com/apache/hudi/issues/15007
Today inline reading can only be turned on for the entire metadata table. Mean all partitions either have this feature enabled or not. But, for smaller partitions like "files" inline is not preferable as it turns off external spillable map caching of records. Where as for other partitions like bloom_filters, inline reading is preferred. We need Partition specific inline reading strategy for metadata table.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3317
- Type: Improvement
- Epic: https://issues.apache.org/jira/browse/HUDI-1822
- Fix version(s):
- 1.1.0
---
## Comments
25/Jan/22 02:57;manojg;Related to
* https://issues.apache.org/jira/browse/HUDI-3300
* https://issues.apache.org/jira/browse/HUDI-3301;;;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Partition specific pointed lookup/reading strategy for metadata table [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #15007: Partition specific pointed lookup/reading strategy for metadata table
URL: https://github.com/apache/hudi/issues/15007
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Partition specific pointed lookup/reading strategy for metadata table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on issue #15007:
URL: https://github.com/apache/hudi/issues/15007#issuecomment-3593826256
This issue belongs to a closed JIRA Epic: https://issues.apache.org/jira/browse/HUDI-1822
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Partition specific pointed lookup/reading strategy for metadata table [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #15007:
URL: https://github.com/apache/hudi/issues/15007#issuecomment-4795299798
This appears to already be implemented on master. `HoodieBackedTableMetadata.isFullScanAllowedForPartition()` ([HoodieBackedTableMetadata.java#L733-L746](https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java#L733-L746)) selects per-partition strategy: `files` uses eager full-scan (so the external spillable map caches records across lookups), while `column_stats`, `bloom_filters`, and any other partition fall back to point-lookups via the log-record reader — matching the request here. The related fix landed under HUDI-3301 (commit f8028a4 / PR #6782, `Fixing HoodieMetadataLogRecordReader to avoid flushing cache for every lookup`). If there's a remaining sub-case (e.g. additional partitions like `record_index`, `secondary_index`, `partition_stats`) where the current default isn't right, please reopen with that specifics; otherwise this can likely be closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org