You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "BenCradock (via GitHub)" <gi...@apache.org> on 2026/09/01 01:52:15 UTC
[I] Allow disabling remote signing in REST Catalog [iceberg-python]
BenCradock opened a new issue, #3888:
URL: https://github.com/apache/iceberg-python/issues/3888
### Feature Request / Improvement
When using a REST Catalog (e.g. Lakekeeper), pyiceberg gets remote signing details from the catalog.
However, sometimes it would be useful to _not_ use remote signing, in the case where the process performing the operations has access to the S3 endpoint through secret access keys. This is because remote signing requires a network round trip (plus any authorisation on server side).
It would be great if we could disable remote signing by respecting the `s3.signer` catalog property if it is empty.
The FileIO object gets created in `pyiceberg/catalog/rest/__init__.py`:
```python
return Table(
identifier=identifier_tuple,
metadata_location=table_response.metadata_location, # type: ignore
metadata=table_response.metadata,
io=self._load_file_io(
{**table_response.metadata.properties, **table_response.config}, table_response.metadata_location
),
catalog=self,
config=table_response.config,
)
```
This doesn't use the catalog properties that have been passed to the REST catalog init as far as I can tell.
I might be wrong, but I don't think [Support storage-credentials in REST catalog LoadTableResult- #3042
](https://github.com/apache/iceberg-python/pull/3042) fixes this, because it still uses the response from the catalog.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Allow disabling remote signing in REST Catalog [iceberg-python]
Posted by "emecii (via GitHub)" <gi...@apache.org>.
emecii commented on issue #3888:
URL: https://github.com/apache/iceberg-python/issues/3888#issuecomment-5566246322
I traced the current flow: `RestCatalog._load_file_io()` already merges `self.properties`, but the per-table response config is merged later, so a client-provided `s3.signer=""` cannot currently override a server `S3V4RestSigner` value.\n\nWould a narrow rule be acceptable: when the client explicitly supplies an empty `s3.signer`, apply that value after the table response solely as an opt-out; otherwise preserve today’s response-wins precedence? I would cover both `Table` and `StagedTable` construction paths and leave all other properties unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Allow disabling remote signing in REST Catalog [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #3888:
URL: https://github.com/apache/iceberg-python/issues/3888#issuecomment-6348303273
Looping in @c-thiel WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Allow disabling remote signing in REST Catalog [iceberg-python]
Posted by "Ankitraj-builds (via GitHub)" <gi...@apache.org>.
Ankitraj-builds commented on issue #3888:
URL: https://github.com/apache/iceberg-python/issues/3888#issuecomment-6322935570
I've opened #4134 for this. It follows the approach discussed above: setting s3.signer to an empty string in the catalog config turns off remote signing, even if the server config sends a signer. Nothing changes if s3.signer is not set.
Happy to change it if a maintainer prefers a different behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Allow disabling remote signing in REST Catalog [iceberg-python]
Posted by "BenCradock (via GitHub)" <gi...@apache.org>.
BenCradock commented on issue #3888:
URL: https://github.com/apache/iceberg-python/issues/3888#issuecomment-5578591444
Yes that makes sense to me - happy for it to be only on an explicit empty `s3.signer`. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Allow disabling remote signing in REST Catalog [iceberg-python]
Posted by "Ankitraj-builds (via GitHub)" <gi...@apache.org>.
Ankitraj-builds commented on issue #3888:
URL: https://github.com/apache/iceberg-python/issues/3888#issuecomment-6322694683
I'd like to work on this if no one else is.
Since #3956 the FileIO property order for RestCatalog is table metadata < catalog config < server config < vended credentials, so a client-side empty s3.signer is still overridden by the server's signer. I'd follow the approach suggested above: only an explicitly set empty s3.signer from the catalog config is applied after the server config, as an opt-out, for both the Table and StagedTable paths. Everything else keeps the current order. I'd add tests in tests/catalog/test_rest.py for both paths.
Could a maintainer confirm this behavior is wanted before I open a PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org