You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/05/19 23:21:46 UTC
[GitHub] [arrow-ballista] andygrove opened a new issue, #22: Ballista context should get file metadata from scheduler, not from local disk
andygrove opened a new issue, #22:
URL: https://github.com/apache/arrow-ballista/issues/22
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
I have a Ballista cluster running, and each scheduler and executor has access to TPC-H data locally.
I am running the benchmark client on my desktop, and I do not have access to the data locally.
Query planning fails with "file not found" because `BallistaContext::read_parquet` is looking for the file on the local file system when it should be getting the file metadata from a scheduler in the cluster.
**Describe the solution you'd like**
The context should send a gRPC request to the scheduler to get the necessary metadata.
**Describe alternatives you've considered**
None
**Additional context**
None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-ballista] avantgardnerio commented on issue #22: Ballista context should get file metadata from scheduler, not from local disk
Posted by GitBox <gi...@apache.org>.
avantgardnerio commented on issue #22:
URL: https://github.com/apache/arrow-ballista/issues/22#issuecomment-1234553112
> Later I'll create a PR for this.
@yahoNanJing this intersects work I'm currently working on, so anything you could share would be helpful!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org