You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by "Abacn (via GitHub)" <gi...@apache.org> on 2025/09/25 19:20:21 UTC

Re: [I] Better filesystem discovery mechanism in Python SDK [beam]

Abacn commented on issue #18500:
URL: https://github.com/apache/beam/issues/18500#issuecomment-3335611519

   filesystems are discovered through Beam plugin mechanism: 
   
   https://github.com/apache/beam/blob/432479c6f585b4cc6aeeddcc1d253ac523a02c95/sdks/python/apache_beam/io/filesystems.py#L135
   
   https://github.com/apache/beam/blob/432479c6f585b4cc6aeeddcc1d253ac523a02c95/sdks/python/apache_beam/utils/plugin.py#L27
   
   Beam plugin classes are imported in sdk_worker_main:
   
   https://github.com/apache/beam/blob/432479c6f585b4cc6aeeddcc1d253ac523a02c95/sdks/python/apache_beam/runners/worker/sdk_worker_main.py#L55
   
   So technically custom filesystem can be registered by setting `--beam_plugin module.to.CustomFileSystem` where CustomFileSystem directly extends from FileSystem
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org