You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/11/18 16:22:26 UTC
[GitHub] [arrow] jorisvandenbossche opened a new pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs
jorisvandenbossche opened a new pull request #8704:
URL: https://github.com/apache/arrow/pull/8704
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on pull request #8704:
URL: https://github.com/apache/arrow/pull/8704#issuecomment-736823311
I added a test, this should be good now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on pull request #8704:
URL: https://github.com/apache/arrow/pull/8704#issuecomment-730220775
@github-actions crossbow submit -g integration
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #8704:
URL: https://github.com/apache/arrow/pull/8704#issuecomment-737046252
Thank you @jorisvandenbossche !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8704:
URL: https://github.com/apache/arrow/pull/8704#issuecomment-729801389
https://issues.apache.org/jira/browse/ARROW-10644
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #8704:
URL: https://github.com/apache/arrow/pull/8704#issuecomment-732232240
@jorisvandenbossche Is the CI satisfactory?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8704:
URL: https://github.com/apache/arrow/pull/8704#issuecomment-730224215
Revision: 5805abb1c1ae4753cae6631928b43758685f628a
Submitted crossbow builds: [ursa-labs/crossbow @ actions-725](https://github.com/ursa-labs/crossbow/branches/all?query=actions-725)
|Task|Status|
|----|------|
|test-conda-python-3.6-pandas-0.23|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.6-pandas-0.23)|
|test-conda-python-3.7-dask-latest|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.7-dask-latest)|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-kartothek-latest|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.7-kartothek-latest)|
|test-conda-python-3.7-kartothek-master|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.7-kartothek-master)|
|test-conda-python-3.7-pandas-latest|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.7-pandas-latest)|
|test-conda-python-3.7-pandas-master|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.7-pandas-master)|
|test-conda-python-3.7-spark-branch-3.0|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.7-spark-branch-3.0)|
|test-conda-python-3.7-turbodbc-latest|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.7-turbodbc-latest)|
|test-conda-python-3.7-turbodbc-master|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.7-turbodbc-master)|
|test-conda-python-3.8-dask-master|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.8-dask-master)|
|test-conda-python-3.8-jpype|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.8-jpype)|
|test-conda-python-3.8-pandas-latest|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.8-pandas-latest)|
|test-conda-python-3.8-spark-master|[](https://github.com/ursa-labs/crossbow/actions?query=branch:actions-725-github-test-conda-python-3.8-spark-master)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on pull request #8704:
URL: https://github.com/apache/arrow/pull/8704#issuecomment-729793788
- Slight downside is that we now do a `get_file_info` twice, but only in the case of a local file path with no filesystem specified (so I think for this case this is less of a concern, but I could still go through hoops to pass it through)
- I added the ability to specify a `filesystem` as a string (the functionality @kszucs added to `pyarrow.dataset`) to the general utilities in `pyarrow.fs`, which means that other functions (eg `parquet.read_table`) also gain this functionality. I should still add a test for that (or if this is not desired, I could also hide this behind an optional keyword, but I think it is good to be consistent in what we accept across submodules).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed pull request #8704: ARROW-10644: [Python] Consolidate path/filesystem handling in pyarrow.dataset and pyarrow.fs
Posted by GitBox <gi...@apache.org>.
pitrou closed pull request #8704:
URL: https://github.com/apache/arrow/pull/8704
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org