You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "nathanb9 (via GitHub)" <gi...@apache.org> on 2026/03/29 21:58:18 UTC
[PR] ParquetPushDecoder API to clear all buffered ranges [arrow-rs]
nathanb9 opened a new pull request, #9624:
URL: https://github.com/apache/arrow-rs/pull/9624
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ParquetPushDecoder API to clear all buffered ranges [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #9624:
URL: https://github.com/apache/arrow-rs/pull/9624
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ParquetPushDecoder API to clear all buffered ranges [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9624:
URL: https://github.com/apache/arrow-rs/pull/9624#issuecomment-4199399008
> Probably should also add an analogous one for ParquetMetaDataPushDecoder? since it could also be used to speculatively push. Ill make a PR for that too if you guys can review that
Thanks @nathanb9 --yes I agree that sounds like a good idea to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ParquetPushDecoder API to clear all buffered ranges [arrow-rs]
Posted by "nathanb9 (via GitHub)" <gi...@apache.org>.
nathanb9 commented on code in PR #9624:
URL: https://github.com/apache/arrow-rs/pull/9624#discussion_r3041663492
##########
parquet/src/arrow/push_decoder/reader_builder/mod.rs:
##########
@@ -212,6 +212,11 @@ impl RowGroupReaderBuilder {
self.buffers.buffered_bytes()
}
+ /// Release any staged ranges currently buffered for future decode work.
+ pub fn release_all_ranges(&mut self) {
Review Comment:
Think `clear_all_ranges` makes sense and its consistent with naming in `push_buffers.rs`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ParquetPushDecoder API to clear all buffered ranges [arrow-rs]
Posted by "nathanb9 (via GitHub)" <gi...@apache.org>.
nathanb9 commented on PR #9624:
URL: https://github.com/apache/arrow-rs/pull/9624#issuecomment-4195349910
Thanks @alamb @AndreaBozzo. Probably should also add an analogous one for `ParquetMetaDataPushDecoder`? since it could also be used to speculatively push. Ill make a PR for that too if you guys can review that
Also, if users find clever ways of getting benefits by speculatively pushing might eventually want to have a smarter version of this clear api or more granular type of clear. Maybe can experiment with this in datafusion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ParquetPushDecoder API to clear all buffered ranges [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #9624:
URL: https://github.com/apache/arrow-rs/pull/9624#discussion_r3041345642
##########
parquet/src/arrow/push_decoder/reader_builder/mod.rs:
##########
@@ -212,6 +212,11 @@ impl RowGroupReaderBuilder {
self.buffers.buffered_bytes()
}
+ /// Release any staged ranges currently buffered for future decode work.
+ pub fn release_all_ranges(&mut self) {
Review Comment:
I wonder if we should keep the name consistent? Either name this `clear_all_ranges` or perhaps rename `clear_all_ranges` to `release_all_ranges` 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ParquetPushDecoder API to clear all buffered ranges [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9624:
URL: https://github.com/apache/arrow-rs/pull/9624#issuecomment-4200246427
I pushed a new commit to this PR to fix CI and merged up from main
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ParquetPushDecoder API to clear all buffered ranges [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #9624:
URL: https://github.com/apache/arrow-rs/pull/9624#issuecomment-4202234140
Thanks again @nathanb9 and @AndreaBozzo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org