[jira] [Commented] (ARROW-243) Use generic HDFS component instead of libhdfs

["Wes McKinney (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394665#comment-15394665 ] 

Wes McKinney commented on ARROW-243:
------------------------------------

My preference would be to add an option to {{HdfsConnectionConfig}} to choose the client library used -- to be loaded at runtime with dlopen -- since libhdfs and libhdfs3 are designed to be roughly ABI-compatible. One reason for this is that binary distributions of Arrow may not know which libraries may be present on the target system.

This is more work than switching between the libraries at compile time but will be more flexible. 

> Use generic HDFS component instead of libhdfs
> ---------------------------------------------
>
>                 Key: ARROW-243
>                 URL: https://issues.apache.org/jira/browse/ARROW-243
>             Project: Apache Arrow
>          Issue Type: New Feature
>            Reporter: Ryan Lewis
>
> I would like to use for example libhdfs3 from pivotal to read apache parquet files. This would be a small change to the hdfs layer of apache arrow to support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)