[jira] [Resolved] (ARROW-8286) [Python] Creating dataset from pathlib results in UnionDataset instead of FileSystemDataset

["Wes McKinney (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wes McKinney resolved ARROW-8286.
---------------------------------
    Resolution: Fixed

Issue resolved by pull request 6783
[https://github.com/apache/arrow/pull/6783]

> [Python] Creating dataset from pathlib results in UnionDataset instead of FileSystemDataset
> -------------------------------------------------------------------------------------------
>
>                 Key: ARROW-8286
>                 URL: https://issues.apache.org/jira/browse/ARROW-8286
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Joris Van den Bossche
>            Assignee: Joris Van den Bossche
>            Priority: Major
>              Labels: dataset, pull-request-available
>             Fix For: 0.17.0
>
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> {code}
> import pyarrow as pa
> import pyarrow.parquet as pq
> import pyarrow.dataset as ds
> table = pa.table({'a': np.random.randn(10), 'b': range(10), 'c': ['a', 'b'] * 5})
> pq.write_table(table, "test.parquet")
> import pathlib
> ds.dataset(pathlib.Path("./test.parquet"))
> # gives UnionDataset
> ds.dataset(str(pathlib.Path("./test.parquet")))
> # correctly gives FileSystemDataset
> {code}
> and since those two dataset classes have different API, this is important to give FileSystemDataset



--
This message was sent by Atlassian Jira
(v8.3.4#803005)