[jira] [Commented] (ARROW-9139) [Python] parquet read_table should not use_legacy_dataset

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-9139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136733#comment-17136733 ] 

Joris Van den Bossche commented on ARROW-9139:
----------------------------------------------

So the features of {{read_table}} that are right now not yet supported by the new API :

* Passing a buffer instead of a file path -> ARROW-8074 (tagged for 1.0.0)
* Specifying an fsspec-compatible filesystem -> ARROW-9089 (tagged for 1.0.0)
* {{metadata}} keyword (https://github.com/apache/arrow/blob/cfe114478d0c90b0822aec1c8c24635c3d35330e/python/pyarrow/parquet.py#L1095-L1096), which right now allows to pass a parquet FileMetadata object (need to check what it actually does with this)

In addition, I also need to check if we right now already infer if there is a {{_metadata}} file in the directory, and if we use that.

> [Python] parquet read_table should not use_legacy_dataset
> ---------------------------------------------------------
>
>                 Key: ARROW-9139
>                 URL: https://issues.apache.org/jira/browse/ARROW-9139
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: Python
>            Reporter: Neal Richardson
>            Assignee: Joris Van den Bossche
>            Priority: Major
>              Labels: dataset-parquet-read, parquet
>             Fix For: 1.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)