You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/03/16 04:16:34 UTC
[GitHub] [arrow-datafusion] yahoNanJing opened a new issue #2019: [ObjectStore] Add datafusion-objectstore-hdfs to the datafusion-contrib
yahoNanJing opened a new issue #2019:
URL: https://github.com/apache/arrow-datafusion/issues/2019
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and *why* for this feature, in addition to the *what*) -->
To avoid community diverged, it's better to add several typical object store implementation to the datafusion-contrib. Then it will also be easier to do the maintenance and improvements.
**Describe the solution you'd like**
For HDFS access client implementation, since many other products, like Arrow, Implama, leverage the *libhdfs* whose real implementation is in Java provided by the official Hadoop community, we also prefer the same way. And from our testing result, the performance is not downgraded. Therefore, we hope to introduce datafusion-objectstore-hdfs to the datafusion-contrib as a standard way to access HDFS for DataFusion.
**Additional context**
#2013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] yahoNanJing commented on issue #2019: [ObjectStore] Add datafusion-objectstore-hdfs to the datafusion-contrib
Posted by GitBox <gi...@apache.org>.
yahoNanJing commented on issue #2019:
URL: https://github.com/apache/arrow-datafusion/issues/2019#issuecomment-1068804726
Thanks @yjshen, @matthewmturner and @alamb. I've just synced the code to https://github.com/datafusion-contrib/datafusion-objectstore-hdfs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] yahoNanJing commented on issue #2019: [ObjectStore] Add datafusion-objectstore-hdfs to the datafusion-contrib
Posted by GitBox <gi...@apache.org>.
yahoNanJing commented on issue #2019:
URL: https://github.com/apache/arrow-datafusion/issues/2019#issuecomment-1068715591
Hi @yjshen and @matthewmturner, could you help review the repo https://github.com/yahoNanJing/datafusion-objectstore-hdfs and check whether it's qualified to be moved into datafusion-contrib?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] xudong963 closed issue #2019: [ObjectStore] Add datafusion-objectstore-hdfs to the datafusion-contrib
Posted by GitBox <gi...@apache.org>.
xudong963 closed issue #2019:
URL: https://github.com/apache/arrow-datafusion/issues/2019
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org