You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2024/06/14 00:52:21 UTC
[PR] Docs: clarify when the reader will read from object store when using cached metadata [datafusion]
alamb opened a new pull request, #10909:
URL: https://github.com/apache/datafusion/pull/10909
## Which issue does this PR close?
Part of
## Rationale for this change
While working on https://github.com/apache/datafusion/pull/10701 it was quite unclear to me why the parquet reader was doing a second object store request even when I had passed it pre-existing `ParquetMetadata`
It turns out the it was because the cached `ParquetMetadata` didn't have the page index strutures loaded, and so the parquet exec will load them on demand if required.
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
## Are these changes tested?
1. Add documentation
Note: I documented this in arrow-rs too: https://github.com/apache/arrow-rs/pull/5887
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Docs: clarify when the parquet reader will read from object store when using cached metadata [datafusion]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead merged PR #10909:
URL: https://github.com/apache/datafusion/pull/10909
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org