You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Antoine Pitrou (Jira)" <ji...@apache.org> on 2019/09/18 15:25:00 UTC
[jira] [Resolved] (ARROW-2013) [Python] Add
AzureDataLakeFilesystem to be used with ParquetDataset and reader/writer
functions
[ https://issues.apache.org/jira/browse/ARROW-2013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Antoine Pitrou resolved ARROW-2013.
-----------------------------------
Resolution: Later
Our strategy here would be to implement this at the C++ layer and then add a Python binding.
> [Python] Add AzureDataLakeFilesystem to be used with ParquetDataset and reader/writer functions
> ------------------------------------------------------------------------------------------------
>
> Key: ARROW-2013
> URL: https://issues.apache.org/jira/browse/ARROW-2013
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Python
> Reporter: Nicholas Pezolano
> Priority: Minor
> Labels: filesystem
>
> Similar to https://issues.apache.org/jira/browse/ARROW-1213, it would be great to add AzureDLFileSystem as a supported filesystem in ParquetDataset.
> Example:
> {code:java}
> from azure.datalake.store import AzureDLFileSystem
> fs = AzureDLFileSystem(token=token, store_name=store_name)
> dataset = pq.ParquetDataset(file_list, filesystem=fs){code}
> Throws:
> {code:java}
> IOError: Unrecognized filesystem: <class 'azure.datalake.store.core.AzureDLFileSystem'>{code}
> Azures github:
> https://github.com/Azure/azure-data-lake-store-python
--
This message was sent by Atlassian Jira
(v8.3.4#803005)