[jira] [Updated] (BEAM-13250) GCSFileSystem is not facilitating injection of the gcs url

["Kenneth Knowles (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-13250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kenneth Knowles updated BEAM-13250:
-----------------------------------

This Jira ticket has a pull request attached to it, but is still open. Did the pull request resolve the issue? If so, could you please mark it resolved? This will help the project have a clear view of its open issues.

> GCSFileSystem is not facilitating injection of the gcs url
> ----------------------------------------------------------
>
>                 Key: BEAM-13250
>                 URL: https://issues.apache.org/jira/browse/BEAM-13250
>             Project: Beam
>          Issue Type: Improvement
>          Components: io-py-gcp
>    Affects Versions: 2.33.0
>            Reporter: Mathieu Leduc-Hamel
>            Priority: P2
>             Fix For: Not applicable
>
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> Inside `apache_beam.io.gcp.gcsfilesystem.GCSFileSystem` is directly instantiating for every operations an instance of `apache_beam.io.gcp.gcsio.GcsIO` without allowing anyone to make use of the flexibility of instantiating with a custom client storage.
> As you can see inside `GcsIO` there's an optional parameter called `storage_client` and that would give us the possibility to overwrite the storage client by example depending of the run level and potentially make use of a Gcs emulator locally when testing the infrastructure.
> Right now the only way we have to overwrite the gcs url is by monkey patching the default storage which is not most clean way of doing it.
>  
> Potential solution:
> If `GCSFileSystem` was transformed by example by adding a single class method to generate every one of thse `GcsIO`, that would give us the possibility to create a custom `GCSFileSystem` class, inheriting from it, without duplicating all of it's code and to inject our desired url depending of the run level.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)