[GitHub] [arrow-rs] tustvold commented on issue #3888: Consider looking into Apache OpenDAL as an alternative to object_store

["tustvold (via GitHub)" <gi...@apache.org>]

tustvold commented on issue #3888:
URL: https://github.com/apache/arrow-rs/issues/3888#issuecomment-1475816315

   Thank you for raising this, I have no particular affection for the current object_store implementation and if we're able to provide the same or better experience to users with less effort, that seems like an unambiguous win.
   
   That being said I do feel strongly that:
   
   * We should preserve the API that places object store semantics first, Spark and similar systems have real issues with Object Storage as it tries to treat them like filesystems
   * We should keep the dependency footprint down
   * Continue to support the various auth options we do currently, including the more esoteric GCP and AWS techniques
   
   Personally it seems more sensible to unify the other way round, port any missing functionality to object_store and use it as the underlying implementation of the more generic OpenDAL, but I don't feel very strongly


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org