You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "XiaoHongbo-Hope (via GitHub)" <gi...@apache.org> on 2026/08/09 03:40:06 UTC
[PR] [python] Avoid redundant file size lookup for Blob reads [paimon]
XiaoHongbo-Hope opened a new pull request, #9125:
URL: https://github.com/apache/paimon/pull/9125
## Purpose
Avoid the extra file-size lookup performed when `SplitRead` creates a `FormatBlobReader`. `DataFileMeta` already contains the file size, so object-store reads do not need an additional metadata request before reading the Blob footer and index.
## Changes
- Add an optional `file_size` argument to `FormatBlobReader`.
- Reuse `DataFileMeta.file_size` in both normal and Data Evolution raw Blob read paths.
- Preserve the existing lookup when `FormatBlobReader` is constructed without a valid file size.
## Tests
- Provided file size avoids `FileIO.get_file_size`.
- Missing or zero file size falls back to one lookup.
- Both `SplitRead` Blob paths forward `DataFileMeta.file_size`.
- Blob tests: 186 passed, 1 unrelated Ray/pandas compatibility test deselected, 44 subtests passed.
- Deferred Blob tests: 15 passed.
- Flake8 and `git diff --check` passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [python] Avoid an extra file-size I/O for Blob reads [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #9125:
URL: https://github.com/apache/paimon/pull/9125#issuecomment-5230000059
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [python] Avoid an extra file-size I/O for Blob reads [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #9125:
URL: https://github.com/apache/paimon/pull/9125
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org