You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "keen85 (via GitHub)" <gi...@apache.org> on 2025/09/08 20:56:36 UTC

[I] LocalFileSystem.head() does not work on directories [arrow-rs-object-store]

keen85 opened a new issue, #483:
URL: https://github.com/apache/arrow-rs-object-store/issues/483

   **Describe the bug**
   Executing LocalFileSystem.head() on an existing directory on Windows results in `PermissionDenied` error.
   
   **To Reproduce**
   I encountered the bug using `obstore` - a Python wrapper for `object_store`.
   Unfortunately I don't know rust.
   This is the snippet that I used for reproducing the bug in Python & `obstore`:
   https://github.com/developmentseed/obstore/pull/557
   ```python
   def test_head(tmp_path: Path):
       store = LocalStore(tmp_path)
       # create a dummy file in a new sub directory
       store.put("freshly_created/some.txt", b"foo")
   
       # ✅ works fine
       assert store.head("freshly_created/some.txt")["path"] == "freshly_created/some.txt"
   
       # 🛑 fails on windows
       assert store.head("freshly_created")["path"] == "freshly_created"
   ```
   I hope that it suffices.
   
   **Expected behavior**
   `head()` should return metadata on the directory - i.e. at the information that the *path is a directory*.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] `LocalFileSystem.head()` does not work on directories [arrow-rs-object-store]

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #483:
URL: https://github.com/apache/arrow-rs-object-store/issues/483#issuecomment-3283907743

   In general object_store doesn't support directories, it is modelled after object stores not filesystems - see https://docs.rs/object_store/latest/object_store/#why-not-a-filesystem-interface
   
   However, it should probably return a NotFoundError for directories not a PermissionDenied


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] `LocalFileSystem.head()` does not work on directories [arrow-rs-object-store]

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #483:
URL: https://github.com/apache/arrow-rs-object-store/issues/483#issuecomment-3286402705

   Yes we explicitly workaround azure hierarchical namespaces - https://github.com/apache/arrow-rs-object-store/blob/main/src%2Fazure%2Fclient.rs#L954
   
   This ensures consistency and frankly directories just add complexity for most use-cases that use a separate catalog/database.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] `LocalFileSystem.head()` does not work on directories [arrow-rs-object-store]

Posted by "keen85 (via GitHub)" <gi...@apache.org>.
keen85 commented on issue #483:
URL: https://github.com/apache/arrow-rs-object-store/issues/483#issuecomment-3284814094

   @tustvold for some object stores, _directories are objects_. E.g. Azure Data Lake Gen2 (with [hierarchical namespaces](https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-introduction#hierarchical-directory-structure)).
   In that sense it behaves more like a Windows file system than a S3.
   
   > However, it should probably return a NotFoundError for directories not a PermissionDenied
   
   Even if the directory exists?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] `LocalFileSystem.head()` does not work on directories [arrow-rs-object-store]

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold closed issue #483: `LocalFileSystem.head()` does not work on directories
URL: https://github.com/apache/arrow-rs-object-store/issues/483


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org