You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/05/16 14:36:36 UTC
[GitHub] [arrow-datafusion] andygrove opened a new issue, #2546: [Ballista] Custom object stores only work for CSV files
andygrove opened a new issue, #2546:
URL: https://github.com/apache/arrow-datafusion/issues/2546
**Describe the bug**
The implementation of `read_csv` creates `ListingTableConfig` using the URI rather than the path of the file (so `test://employee.csv` rather than `employee.csv`) and this is required to make `roundtrip_logical_plan_custom_ctx` pass.
There are no tests for custom contexts for other file types and those paths use path instead of URI, so don't seem to support custom object stores.
There is some code duplicated between all of these read methods that could probably be refactored into common helper functions.
**To Reproduce**
Try implementing a version of `roundtrip_logical_plan_custom_ctx` for each file type.
**Expected behavior**
Functionality should be consistent between file types.
**Additional context**
None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] thinkharderdev commented on issue #2546: [Ballista] Fix regression in `roundtrip_logical_plan_custom_ctx` test
Posted by GitBox <gi...@apache.org>.
thinkharderdev commented on issue #2546:
URL: https://github.com/apache/arrow-datafusion/issues/2546#issuecomment-1128717802
Yeah, the issue here is that the serde logic uses the file scheme to resolve the `ObjectStore`. So we were relying on the `test` scheme to resolve the `TestObjectStore`. Why would we not serialize the entire file URI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] andygrove commented on issue #2546: [Ballista] Fix regression in `roundtrip_logical_plan_custom_ctx` test
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #2546:
URL: https://github.com/apache/arrow-datafusion/issues/2546#issuecomment-1127911455
@thinkharderdev @mingmwang I could use some help with this if you have time to look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org