You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Dhruv Madeka (JIRA)" <ji...@apache.org> on 2018/03/21 18:32:00 UTC
[jira] [Commented] (ARROW-2332) [Python] Provide API for reading
multiple Feather files
[ https://issues.apache.org/jira/browse/ARROW-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16408384#comment-16408384 ]
Dhruv Madeka commented on ARROW-2332:
-------------------------------------
As mentioned in the Github issue -
IMHO this requires a few steps.
* In `FeatherReader`, separate the extraction of the table from the call to convert it `to_pandas`
* Create a `FeatherDataset` class which takes a list of featherfiles and creates a table for each one
* Validate that the schemas for each of the files match, the `validate_schema` for the `ParquetDataset` seems to work. Maybe we can create an abstract dataset class and inherit from there
> [Python] Provide API for reading multiple Feather files
> -------------------------------------------------------
>
> Key: ARROW-2332
> URL: https://issues.apache.org/jira/browse/ARROW-2332
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Wes McKinney
> Priority: Major
> Fix For: 0.10.0
>
>
> See discussion in https://github.com/wesm/feather/issues/273#issuecomment-374093374
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)