[GitHub] [arrow-rs] tustvold commented on issue #3283: Add support for Azure Data Lake Storage Gen2 (aka: ADLS Gen2) in Object Store library

[GitBox <gi...@apache.org>]

tustvold commented on issue #3283:
URL: https://github.com/apache/arrow-rs/issues/3283#issuecomment-1341063136

   The ObjectStore library does not support currently contain logic for parsing URLs, this is tracked by https://github.com/apache/arrow-rs/issues/2304.
   
   However, querying ADLS buckets by path should work correctly. e.g. something like
   
   ```
   let store = MicrosoftAzureBuilder::new()
       .with_account("az_account_name")
       .with_container_name("my_file_system")
       .with_<CREDENTIALS>(...)
       .build()?;
   
   let files: Vec<_> = store.list(Some(&Path::from("my/path/id1/id2/"))).await?.try_collect().await?;
   ```
   
   However, the crate can and is being used with ADLS Gen2, there is in fact code such as [here](https://github.com/apache/arrow-rs/blob/master/object_store/src/azure/client.rs#L418) specifically to handle the fact Gen2 buckets have directories. 
   
   Perhaps @roeap might be able to weigh in here, as he added the initial support for ADLS Gen2 buckets back when this repo lived elsewhere - https://github.com/influxdata/object_store_rs/pull/25
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org