[GitHub] [arrow-datafusion] yahoNanJing commented on pull request #1223: Add support of HDFS as remote object store

[GitBox <gi...@apache.org>]

yahoNanJing commented on pull request #1223:
URL: https://github.com/apache/arrow-datafusion/pull/1223#issuecomment-962387646


   Hi @alamb and @houqp, I agree that the core of DataFusion should be as simplified as possible. However, if we position it to be used in the production environment with some large scale cluster. It's better to support some remote shared storage as part of the architecture of storage and compute separation. 
   
   If we don't include the connectors into DataFusion, it's still better to provide an official repository for them. Otherwise, it might not be good for the community unity. 
   
   By the way, for the interface of **register_store** in **ObjectStoreRegistry**, there may be several **ObjectStore** with different endpoints for the same scheme. For example, hdfs://localhost:60000 and hdfs://localhost:60001


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org