You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "sumerman (via GitHub)" <gi...@apache.org> on 2023/09/06 11:58:06 UTC
[GitHub] [arrow-rs] sumerman opened a new pull request, #4784: Make coalesce_ranges and collect_bytes available for crate users
sumerman opened a new pull request, #4784:
URL: https://github.com/apache/arrow-rs/pull/4784
# Which issue does this PR close?
I didn't create an issue
# Rationale for this change
Both `coalesce_ranges` and `collect_bytes` are useful for `AsyncFileReader` implementations that don't quite fit `object_store` interface, but address the same fundamental problem — fetching data from a remote location.
# What changes are included in this PR?
`coalesce_ranges`, `collect_bytes` and `OBJECT_STORE_COALESCE_DEFAULT` are made `pub use`
# Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!---
If there are any breaking changes to public APIs, please add the `breaking change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] sumerman commented on pull request #4784: Make coalesce_ranges and collect_bytes available for crate users
Posted by "sumerman (via GitHub)" <gi...@apache.org>.
sumerman commented on PR #4784:
URL: https://github.com/apache/arrow-rs/pull/4784#issuecomment-1710002462
Another change, that I believe would be useful for non-`object_store` uses, is switching to `std::result::Result` type for `coalesce_ranges` and allowing errors that aren't `object_store::Error`. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] sumerman commented on pull request #4784: Make coalesce_ranges and collect_bytes available for crate users
Posted by "sumerman (via GitHub)" <gi...@apache.org>.
sumerman commented on PR #4784:
URL: https://github.com/apache/arrow-rs/pull/4784#issuecomment-1710020450
> Adding the error as a type parameter doesn't seem like it would be particularly controversial / complex.
👍🏼 I'll send it as another PR
>I am somewhat curious what you are intending to use this for though?
We have an `AsyncFileReader` implementation that, firstly relies on in-process metadata caching, secondly manages its own tokio runtime (we have to be careful with the runtime as we run inside PostgreSQL). So, we can either sidestep `object_store` or we would have to build both an `object_store` impl that is compliant with our runtime needs _and_ `AsyncFileReader` that can use the metadata cache, but delegates the rest to the `object_store`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #4784: Make coalesce_ranges and collect_bytes available for crate users
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on PR #4784:
URL: https://github.com/apache/arrow-rs/pull/4784#issuecomment-1710005228
Adding the error as a type parameter doesn't seem like it would be particularly controversial / complex. I am somewhat curious what you are intending to use this for though?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold merged pull request #4784: Make coalesce_ranges and collect_bytes available for crate users
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold merged PR #4784:
URL: https://github.com/apache/arrow-rs/pull/4784
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #4784: Make coalesce_ranges and collect_bytes available for crate users
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on PR #4784:
URL: https://github.com/apache/arrow-rs/pull/4784#issuecomment-1710027943
Makes sense, I do hope to eventually flesh out a first-party caching story, but I don't have anything to show for it as this time 😅
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] sumerman commented on pull request #4784: Make coalesce_ranges and collect_bytes available for crate users
Posted by "sumerman (via GitHub)" <gi...@apache.org>.
sumerman commented on PR #4784:
URL: https://github.com/apache/arrow-rs/pull/4784#issuecomment-1709975917
> #4786 should resolve the clippy failures
Thank you! I've rebased my PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org