You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "bigluck (via GitHub)" <gi...@apache.org> on 2025/01/29 09:22:22 UTC

[I] FileIO Implementation Configuration Priority Question [iceberg-python]

bigluck opened a new issue, #1589:
URL: https://github.com/apache/iceberg-python/issues/1589

   ### Apache Iceberg version
   
   None
   
   ### Please describe the bug 🐞
   
   Hey team! 👋 Hope you're doing well!
   I've been working with PyIceberg and ran into an interesting situation regarding FileIO implementation configurations.
   
   TLTR: it seems like `pyiceberg` does not allow to overwrite (locally) some configurations returned by the remote Iceberg REST api, and I would like to understand if this is intended and what is the way to overwrite some of these configurations.
   
   ### Context
   
   - The REST API server (`/config` endpoint) is configured to use `PyArrow` as the default `FileIO` implementation
   - The table endpoint returns a configuration specifying a different `FileIO` implementation (`fsspec`)
   - I want to force `PyArrow` locally since I don't have `fsspec/s3fs` installed (and prefer using `PyArrow`)
   
   Starting from [PR 9868](https://github.com/projectnessie/nessie/pull/9868), the `Nessie` Iceberg endpoint always returns `config.py-io-impl=pyiceberg.io.fsspec.FsspecFileIO` when querying a table endpoint. While there might be a separate issue in Nessie's implementation (as my server is configured to override the `py-io-impl` with `pyarrow.PyArrowFileIO`), I believe there's also a concern in PyIceberg's handling of configuration priorities.
   
   For more info:
   - [My chat with the Nessie team on zulip](https://project-nessie.zulipchat.com/#narrow/channel/371187-general/topic/How.20to.20use.20PyarrowFileIO.20on.20pyiceberg.3F/near/494545947)
   - their original [PR 9868](https://github.com/projectnessie/nessie/pull/9868)
   - the temp workaround the Nessie team is implementing not [PR 10292](https://github.com/projectnessie/nessie/pull/10292)
   
   ### Current Behavior
   
   I've created a test case that explores different configuration scenarios. Here's what I'm observing:
   
   ```python
   pythonCopycatalog = load_catalog("docs", **{"uri": "https://a.b.c.d/iceberg", "py-io-impl": "pyiceberg.io.pyarrow.PyArrowFileIO"})
   table = catalog.load_table("my_namespace.my_table")
   ```
   
   similarly to what is describe on the official documentation:
   
   https://github.com/apache/iceberg-python/blob/1adbb87627bfdfe80622d78c57b6214957520be0/mkdocs/docs/api.md?plain=1#L56-L70
   
   The table configuration from the REST API seems to take precedence over both:
   - The server's default configuration
   - Local overrides passed during catalog initialization
   
   This leads to failures when the table endpoint specifies `fsspec.FsspecFileIO` but `s3fs` isn't available locally:
   
   ```
   ValueError: Could not initialize FileIO: pyiceberg.io.fsspec.FsspecFileIO
   ```
   
   ### Real-world Impact
   
   This configuration priority issue creates practical problems in multi-system setups. Consider this scenario:
   
   - System A uses `fsspec` for writing tables into Nessie/Iceberg
   - System B needs to read the same tables using `PyArrow`
   
   With the current implementation, System B can never successfully read the tables because:
   
   - The server forces the client to use `fsspec`
   - This happens even when the client explicitly requests `PyArrow`
   - There's no way to override this behavior at the client level
   
   
   ### Question
   
   Is there a way for for PyIceberg to use a specific FileIO implementation regardless of what the table endpoint or the server returns?
   This would be particularly useful in scenarios where:
   
   - The client environment is set up for a specific implementation
   - Different `FileIO` implementations might be more efficient in certain environments
   - Required dependencies for the server-specified implementation aren't available locally.
   
   I've attached a test file that demonstrates the behavior
   
   Would love to hear your thoughts on this! Is this the intended behavior? If so, could we perhaps consider adding a way to override the table-level FileIO implementations?
   
   
   Thanks
   
   [test.txt](https://github.com/user-attachments/files/18584986/test.txt)
   
   ### Willingness to contribute
   
   - [ ] I can contribute a fix for this bug independently
   - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
   - [ ] I cannot contribute a fix for this bug at this time


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "smaheshwar-pltr (via GitHub)" <gi...@apache.org>.
smaheshwar-pltr commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-2633662660

   Thanks @bigluck @kevinjqliu for this valuable discussion.
   
   Just pointing out that IMO REST catalogs can righly be reliant on clients (generally) prioritising the IO returned in the `table_response.config`. For things like federation, I think the Catalog returning the appropriate IO and clients adhering is crucial. In PyIceberg, the codepath that this depends on is (as already mentioned) https://github.com/apache/iceberg-python/blob/90706ea8bd335dda6af08cf32dc5b4cb9ecf259c/pyiceberg/catalog/rest.py#L527 and in the Java implementation, it's https://github.com/apache/iceberg/blob/1afa0e739b62355518942629b30c845e51e724a6/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L1038.
   
   I do think that *PyIceberg having consistent behaviour with the Java implementation* here is a big advantage.
   
   @bigluck's idea of explicitly allowing clients to override that behaviour if they wish (so if stuff goes wrong it's on them) seems very reasonable to me 😄. IMHO the interface to this should indicate that this is exceptional behaviour. (Also, my sense is that we'd want to involve other implementations in this discussion via a thread but I'll let @kevinjqliu express his thoughts on this matter and that action first.)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-4585305750

   This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "bigluck (via GitHub)" <gi...@apache.org>.
bigluck commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-2622544404

   Thanks @kevinjqliu , and happy new year too :)
   
   I see your point, and I see how the server should always control the clients, but from my PoV the client should always have a way to overwrite these defaults, otherwise you can't mix different server implementation together by using a single data catalog.
   
   The current situation is kind of confusing.
   
   ```
    catalog = load_catalog( 
        "docs", 
        **{ 
            "uri": "http://127.0.0.1:8181", 
            "s3.endpoint": "http://127.0.0.1:9000", 
            "py-io-impl": "pyiceberg.io.pyarrow.PyArrowFileIO", 
            "s3.access-key-id": "admin", 
            "s3.secret-access-key": "password", 
        } 
    ) 
   ```
   
   In this example `py-io-impl` seems not to be a "client config" but more a "catalog config", but `uri` is technically a "client config" and in any case both of them are stored on the same property: `self.properties`.
   
   Once started the Catalog REST API class send a request to the `/config` endpoint by invoking the `_fetch_config()` function, and it does:
   
   ```python
           config = config_response.defaults
           config.update(self.properties)
           config.update(config_response.overrides)
           self.properties = config
   ```
   
   It takes the `defaults` from the remote server, it extend them with `self.properties` (my `uri, s3.endpoint, py-io-impl, s3.access-key-id, s3.secret-access-key`) and it overrides the final config with `overrides`.
   That's fine :)
   
   In this case if the server wants to override the `py-io-impl` it can, by setting the corresponding overrides key, otherwise the user wins against the remote config.
   
   What's weird is that these properties are not used when we to a `load_table`:
   
   ```python
           identifier_tuple = self._identifier_to_tuple_without_catalog(identifier)
           response = self._session.get(
               self.url(Endpoints.load_table, prefixed=True, **self._split_identifier_for_path(identifier_tuple))
           )
           try:
               response.raise_for_status()
           except HTTPError as exc:
               self._handle_non_200_response(exc, {404: NoSuchTableError})
   
           table_response = TableResponse(**response.json())
           return self._response_to_table(identifier_tuple, table_response)
   ```
   
   we get the table data from the remote server, we ignore the local configuration and we create a new table object:
   
   ```python
   
       def _response_to_table(self, identifier_tuple: Tuple[str, ...], table_response: TableResponse) -> Table:
           return Table(
               identifier=identifier_tuple,
               metadata_location=table_response.metadata_location,  # type: ignore
               metadata=table_response.metadata,
               io=self._load_file_io(
                   {**table_response.metadata.properties, **table_response.config}, table_response.metadata_location
               ),
               catalog=self,
               config=table_response.config,
           )
   ```
   
   This is what I think it's not ok:
   ```
   {**table_response.metadata.properties, **table_response.config}
   ```
   
   We should have a way to overwrite both the table properties and the table config, because the client knows what it can use and what it can't.
   
   I think there are ~3 options:
   
   1. `{**table_response.metadata.properties, **table_response.config, **self.properties}`
   I don't like it because we propagate everything
   
   2. When the user invoke `load_table()` she can optionally set list of properties that will overwrite the remote properties:
   ```python
   load_table('my.table', **{"py-io-impl": "pyiceberg.io.pyarrow.PyArrowFileIO"})
   ```
   
   3. Similarly to 1, but with a limited set of properties. In other words we set a configurable list of porperties that needs to be propagated to the Table object:
   ```python
   {**table_response.metadata.properties, **table_response.config, **{k: v for k, v in self.properties.items() where k in self.white_list_of_keys}}


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-3169564065

   This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-2646079224

   Created https://github.com/projectnessie/nessie/issues/10363


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "psarka (via GitHub)" <gi...@apache.org>.
psarka commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-3333569101

   I'd like to register a use-case where client overriding make sense. When files are hosted on Oracle cloud, nessie catalog is configured to use S3 compat layer and nessie service pod has static credentials exposed as AWS_ env vars. But the users of catalog data have standard OCI auth. For such users it makes sense to override fileio and use one that uses Oracle sdk, rather than s3fs.
   
   At the moment the workaround is to extend RestCatalog class and to override the private _response_to_talble method, which is obviously not ideal.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-3335527638

   @psarka : Could this help in your case? https://github.com/projectnessie/nessie/issues/10363#issuecomment-2984557537


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-2646079484

   @bigluck / @smaheshwar-pltr does the explanation make sense? Please LMK if im missing something


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-2622298632

   Hey @bigluck good to hear from you again :) happy new year! 
   
   You brought up an interesting use case and i think we should cement the right behavior. There are a couple of different sets of configs
   1. catalog initialization configs, we can also call it "client config" since its the client of IRC
   2. IRC server default (from the `/config` endpoint), this is the "catalog config"
   3. namespace configs (the namespace itself can specify properties)
   4. table configs 
   
   From the server side, the table configs take precedence over namespace configs over server configs. 
   
   Between the client and the server, I'm not sure which one should take precedence. Currently, the server configs take precedence and we override client configs during catalog initialization.
   https://github.com/apache/iceberg-python/blob/1adbb87627bfdfe80622d78c57b6214957520be0/pyiceberg/catalog/rest.py#L383-L402
   
   We should figure out if this is the right behavior. Is the server config more of a suggestion or a must-override? 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-2646412685

   > Obviously, when configuring the client, I'll need to use 3 different URLs depending on these 3 conditions, but the server will always return the same URL in the /config endpoint.
   
   So you can connect to the same server with different uri schemes. 
   For example,
   * ALB (alb://server)
   * Private link (private://server)
   * Proxy (proxy://server)
   
   And the server advertise its own address as http://server. When connecting using the REST client, the `catalog.uri` gets overwritten? 
   
   `uri` is part of [catalog properties](https://iceberg.apache.org/docs/latest/configuration/#catalog-properties) so it makes sense  that it is allowed to be overwritten by the server. 
   I think the easiest solution is to have the server not send the optional `uri` properties in the response.
   
   
   I understand the concern and need for more control over the REST client. As part of the REST spec, the client and server should both be working together and adhering to the spec. Of course, you can always implement custom logic on top of existing components. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-2646078141

   Thanks for chiming in here! I took some time to look into this. Here's what I found:
   
   ### On Catalog Configuration
   The configuration precedence is described in the REST spec for the `/config` endpoint. Based on the REST spec, here is the expected behavior for catalog configurations. 
   https://github.com/apache/iceberg/blob/d935460bbed5da0ac205a67912b70fa682ee84df/open-api/rest-catalog-open-api.yaml#L81-L98
   Mainly, 
   ```
   Catalog configuration is constructed by setting the defaults, then client-
           provided configuration, and finally overrides. The final property set is then
           used to configure the catalog.
   ```
   
   The REST catalog implementation does indeed follow this order. 
   https://github.com/apache/iceberg-python/blob/826a00628216993de50f7f0c2111b6c824b02939/pyiceberg/catalog/rest.py#L397-L400
   
   So when Nessie [added the feature](https://github.com/projectnessie/nessie/pull/9868/files#diff-35b0e3dabf006d8b221a85dbed21058b2a79856d533015afa740d565514a6eb3R267) to send `py-io-impl=pyiceberg.io.fsspec.FsspecFileIO` using the configurations endpoint (`/config`), it is sent as part of the configurations endpoint's `default` properties. This means the properties specified in `load_catalog` should override this value if it is provided. And the server can use `override` properties to trump over the `load_catalog` properties. 
   
   To summarize:
   * Client can override `/config`'s `default` parameters
   * `config`'s `override` parameters can override client parameters
   
   ### On Table Configuration
   Here's the REST spec for the `loadTable` endpoint 
   https://github.com/apache/iceberg/blob/d935460bbed5da0ac205a67912b70fa682ee84df/open-api/rest-catalog-open-api.yaml#L920-L925
   ```
           Load a table from the catalog.
   
   
           The response contains both configuration and table metadata. The configuration, if non-empty is used
           as additional configuration for the table that overrides catalog configuration. For example, this
           configuration may change the FileIO implementation to be used for the table.
   ```
   This corresponds with https://github.com/apache/iceberg-python/blob/826a00628216993de50f7f0c2111b6c824b02939/pyiceberg/catalog/rest.py#L527-L531
   
   `self._load_file_io` here is defined in the base catalog class and will include the catalog's properties 
   https://github.com/apache/iceberg-python/blob/826a00628216993de50f7f0c2111b6c824b02939/pyiceberg/catalog/__init__.py#L728
   
   Expanding the parameters 
   ```
   {**self.properties , {**table_response.metadata.properties, **table_response.config}}
   ->
   {**{**config_response.defaults, **self.properties, **config_response.overrides}, {**table_response.metadata.properties, **table_response.config}}
   -> 
   **{config_response.defaults, self.properties, config_response.overrides, table_response.metadata.properties, table_response.config}
   ```
   where `config_response` is from `/config` endpoint, `self.properties` is the properties given to `load_catalog`, and `table_response` is from `loadTable`. Note, `table_response.metadata.properties` should be the same as the table's own property. 
   
   ### Conclusion
   I think the real culprit is from [Nessie `icebergConfigDefaults` setting the `py-io-impl` config](https://github.com/projectnessie/nessie/pull/9868/files#diff-35b0e3dabf006d8b221a85dbed21058b2a79856d533015afa740d565514a6eb3R267). `icebergConfigDefaults` is called by [`configureIcebergTable`](https://github.com/projectnessie/nessie/blob/9ae0f670a8a67aa573612cde4f547aac0b61f112/catalog/files/impl/src/main/java/org/projectnessie/catalog/files/s3/S3ObjectIO.java#L174-L190) which is called by [`icebergConfigPerTable`](https://github.com/projectnessie/nessie/blob/main/catalog/service/rest/src/main/java/org/projectnessie/catalog/service/rest/IcebergConfigurer.java#L216-L255) and then finally called by [loadTable](https://github.com/projectnessie/nessie/blob/main/catalog/service/rest/src/main/java/org/projectnessie/catalog/service/rest/IcebergApiV1TableResource.java#L276-L298) 
   
   This means every `loadTable` response from Nessie will include the config to override `py-io-impl`.
   And based on the hierarchy above, this will always override any parameters set by the config endpoint, the catalog itself, and the table's property. 
   
   In conclusion, pyiceberg REST client is working as intended. Nessie should not send `py-io-impl` config override as part of the loadTable response. 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "bigluck (via GitHub)" <gi...@apache.org>.
bigluck commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-2646117181

   I understand, and in principle I agree that pyiceberg is following the Iceberg Catalog API specifications.
   
   On the other hand, I believe it's incorrect that the client cannot have any final say over the server settings.
   I have a second case that better explains why I think it's incorrect.. we have the same Catalog API and it is accessible from different URLs:
   
   1. ALB
   2. Private link when the same catalog is privately shared with another VPC
   3. A proxy that, besides the hostname, also has a different `base_url`
   
   Obviously, when configuring the client, I'll need to use 3 different URLs depending on these 3 conditions, but the server will always return the same URL in the `/config` endpoint.
   
   As things stand now, I'm forced to manually override the value of `catalog.uri` after initializing the client, otherwise the client wouldn't be able to read any tables.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] `py-io-impl` config propagation [iceberg-python]

Posted by "psarka (via GitHub)" <gi...@apache.org>.
psarka commented on issue #1589:
URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-3337046398

   @dimas-b Thanks for the reference! I'm a bit burned out from trying to get things to work on OCI, but I will give it a try when I recover 😅 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org