You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 04:56:22 UTC
[I] Improve log merging performance for Metadata Table [hudi]
hudi-bot opened a new issue, #15168:
URL: https://github.com/apache/hudi/issues/15168
For Metadata Table as a MOR table, there is a separate log record reader "HoodieMetadataMergedLogRecordReader". We want to see if we can further improve the merging latency on different file systems including S3.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-4034
- Type: Task
- Epic: https://issues.apache.org/jira/browse/HUDI-3249
---
## Comments
15/Sep/22 19:07;alexey.kudinkin;[~guoyihua] can you please clarify what are exact steps we're planning to do here?
Let's also clarify whether this is still needed after HUDI-4465;;;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Improve log merging performance for Metadata Table [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #15168: Improve log merging performance for Metadata Table
URL: https://github.com/apache/hudi/issues/15168
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Improve log merging performance for Metadata Table [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #15168:
URL: https://github.com/apache/hudi/issues/15168#issuecomment-4795329265
This issue tracks an open-ended performance improvement to `HoodieMetadataMergedLogRecordReader`. That class (and its successor `HoodieMetadataLogRecordReader`) no longer exists on master — commit [21e9a0edc2f3](https://github.com/apache/hudi/commit/21e9a0edc2f3) ([HUDI-9592], #13544) deleted the dedicated metadata log record reader and migrated metadata table reads through the unified `HoodieFileGroupReader`, which adds reusable base-file readers and record-buffer loaders for repeated lookups (see `HoodieBackedTableMetadata#readSliceWithFilter`). Closing this as obsolete unless someone has fresh S3 benchmarks showing the new path is still a bottleneck — please reopen with numbers if so.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org