[GitHub] [libcloud] Kami edited a comment on pull request #1631: Fix object name prefix based filtering in the local storage driver

[GitBox <gi...@apache.org>]

Kami edited a comment on pull request #1631:
URL: https://github.com/apache/libcloud/pull/1631#issuecomment-984684135


   @louis-van-der-stam Thanks for the info. I will add some micro-benchmarks when I get a chance, just to get some idea on the sorting overhead.
   
   But yeah, local storage driver was never built with performance in mind - it was designed and built for testing purposes.
   
   Having said that, per my comment [here](https://github.com/apache/libcloud/pull/1584#issuecomment-983881324), I do think there are some possible performance optimizations for some scenarios. 
   
   In short, something similar to what was originally done in #1584, but making sure all the edge cases are handled correctly and only filtering by sub-directory if the actual prefix maps to a sub-directory and not to an object.
   
   But yeah, in the end, we are limited by the OS primitives available and we will still need to do some late filtering in the Python code no matter what (depending on what you are working on and your use case, inotify based approach may offer advantages, but that's of course out of scope of Libcloud).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org