[GitHub] [arrow-datafusion] yahoNanJing commented on a change in pull request #1779: Improve object store key with considering host and port in ObjectStoreRegistry

[GitBox <gi...@apache.org>]

yahoNanJing commented on a change in pull request #1779:
URL: https://github.com/apache/arrow-datafusion/pull/1779#discussion_r801275370



##########
File path: datafusion/src/datasource/object_store/local.rs
##########
@@ -39,6 +39,11 @@ pub struct LocalFileSystem;
 #[async_trait]
 impl ObjectStore for LocalFileSystem {
     async fn list_file(&self, prefix: &str) -> Result<FileMetaStream> {
+        let prefix = if let Some((_scheme, path)) = prefix.split_once("://") {

Review comment:
       For hdfs, there's a HA mechanism to provide a kind of vip host name. The hostname is not a name for a real host, by which ping will not work. However, the dependent object store client is able to recognize that name to direct requests to the real host.
   
   I think this part should be the capability of remote object store, either by the way of HDFS or providing a vip service.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org