You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "manisin (via GitHub)" <gi...@apache.org> on 2026/02/10 00:24:34 UTC
[PR] Add spec support for credential refresh on staged tables [iceberg]
manisin opened a new pull request, #15280:
URL: https://github.com/apache/iceberg/pull/15280
## Summary
Extends the REST Catalog OpenAPI spec to support credential refresh for staged tables by introducing a staging session identifier, following the same pattern as `planId` for scan planning credential refresh. This addresses an earlier issue raised in community: https://github.com/apache/iceberg/issues/13554
- `createTable` (POST): When `stage-create` is true, the `LoadTableResult` response may now include an optional `staging-session` field — an opaque session identifier for subsequent credential refresh.
- `loadTable` (GET): Accepts an optional `stagingSession` query parameter to load a staged table and refresh credentials when used with `X-Iceberg-Access-Delegation: vended-credentials`.
- `loadCredentials` (GET): Accepts an optional `stagingSession` query parameter to refresh credentials for a staged table. Mutually exclusive with `planId` (returns 400 if both are provided).
- New reusable `staging-session` query parameter defined in `components/parameters`.
- New `staging-session` field added to the `LoadTableResult` schema.
### Behavior
When `stagingSession` is provided:
- If the session matches a staged table for the given table name, the server returns the response for the staged table.
- If the session matches a committed table, the server returns the response for the committed table.
- If the session does not match any staged or committed table, the server returns a 404 error.
When `stagingSession` is omitted, standard committed table lookup by name is performed.
### Backward Compatibility
These changes are fully backward compatible:
- **Existing servers** will ignore the `stagingSession` parameter, maintaining current behavior.
- **Existing clients** will not send `stagingSession` and will function as they do today.
- **ListTables** does not expose staged tables — only clients that know the session identifier can access them.
### Design
This follows the same pattern as the existing scan planning credential refresh (`planId` query parameter), where an ID returned from an initial request facilitates stateful credential refreshes for long-running asynchronous operations.
Detailed design: https://docs.google.com/document/d/1R1K6X7qYqvIFkPG3m1neV5Mvy8rwWJvhSFr8DgJgQ-E/edit?tab=t.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Add spec support for credential refresh on staged tables [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #15280:
URL: https://github.com/apache/iceberg/pull/15280#issuecomment-4051300870
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST Spec: support credential refresh on staged tables [iceberg]
Posted by "stevenzwu (via GitHub)" <gi...@apache.org>.
stevenzwu commented on code in PR #15280:
URL: https://github.com/apache/iceberg/pull/15280#discussion_r3042984414
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1227,7 +1234,17 @@ paths:
type: string
description: The plan ID that has been used for server-side scan planning
- $ref: '#/components/parameters/referenced-by'
- description: Load vended credentials for a table from the catalog.
+ - $ref: '#/components/parameters/storage-refresh-token'
+ description:
+ Load vended credentials for a table from the catalog.
+
+
+ An optional `storageRefreshToken` query parameter can be provided to refresh credentials
+ for a staged table. See the `storageRefreshToken` parameter definition for detailed behavior.
+
+
+ The `planId` and `storageRefreshToken` parameters are mutually exclusive. If both are
+ provided, the server must return a 400 Bad Request error.
Review Comment:
The OpenAPI spec conventions use `MUST` (uppercase).
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1979,6 +1998,29 @@ components:
schema:
type: string
+ storage-refresh-token:
+ name: storageRefreshToken
+ in: query
+ description: >
+ Opaque token returned on a `StorageCredential` from a previous response (e.g., staged table
+ creation, loadTable, or loadCredentials). When provided, the server uses this token to resolve
+ the credential context (e.g., a staged table's storage location) and return fresh credentials.
+
+
+ If the token resolves to a staged or committed table matching the table name in the request
Review Comment:
If the table is already committed, the token shouldn't be needed—loadCredentials already works by table name. The description conflates two cases. Is the token specifically for staged tables? if the table has since been committed, the server MAY resolve the token to the committed table (or the client can simply omit the token)?
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3470,6 +3512,13 @@ components:
type: object
additionalProperties:
type: string
+ storage-refresh-token:
+ type: string
+ description: >
+ Opaque token that clients pass back to the server to refresh this credential.
+ Returned on the loadCredentials or loadTable endpoint via the storageRefreshToken
Review Comment:
> Returned on the loadCredentials or loadTable endpoint via the storageRefreshToken query parameter.
This sentence seems problematic to me. The refresh token is returned on the `loadTable` or `loadCredentials` response. But it seems odd to say `via the storageRefreshToken query parameter`. It is passed back to the server via the query param for `loadCredentials` endpoint.
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1227,7 +1234,17 @@ paths:
type: string
description: The plan ID that has been used for server-side scan planning
- $ref: '#/components/parameters/referenced-by'
- description: Load vended credentials for a table from the catalog.
+ - $ref: '#/components/parameters/storage-refresh-token'
Review Comment:
A `createTable` response can return multiple `StorageCredential` objects (one per prefix). Each credential in the PR gets its own `storage-refresh-token`. But the `loadCredentials` endpoint accepts only one `storageRefreshToken` query parameter.
The spec doesn't clarify:
- Should clients use the token from the first credential? The longest-matching-prefix credential?
- Can different credentials in the same response have different refresh tokens?
- If the server returns multiple credentials with different tokens, does a single refresh call return all of them?
This **1:N** mismatch needs explicit guidance. If the token is really per-table (not per-credential), it might belong on the `LoadTableResult` or `LoadCredentialsResponse` rather than on individual `StorageCredential` objects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST Spec: support credential refresh on staged tables [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #15280:
URL: https://github.com/apache/iceberg/pull/15280#discussion_r3086590698
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1979,6 +1998,29 @@ components:
schema:
type: string
+ storage-refresh-token:
+ name: storageRefreshToken
+ in: query
+ description: >
+ Opaque token returned on a `StorageCredential` from a previous response (e.g., staged table
Review Comment:
as mentioned further above I don't think the mapping between refresh token and a single storage credential should be 1:1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST Spec: support credential refresh on staged tables [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #15280:
URL: https://github.com/apache/iceberg/pull/15280#issuecomment-4523570780
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST Spec: support credential refresh on staged tables [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #15280: REST Spec: support credential refresh on staged tables
URL: https://github.com/apache/iceberg/pull/15280
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST Spec: support credential refresh on staged tables [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #15280:
URL: https://github.com/apache/iceberg/pull/15280#issuecomment-4464768631
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST Spec: support credential refresh on staged tables [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #15280:
URL: https://github.com/apache/iceberg/pull/15280#discussion_r3086581455
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1227,7 +1234,17 @@ paths:
type: string
description: The plan ID that has been used for server-side scan planning
- $ref: '#/components/parameters/referenced-by'
- description: Load vended credentials for a table from the catalog.
+ - $ref: '#/components/parameters/storage-refresh-token'
+ description:
+ Load vended credentials for a table from the catalog.
+
+
+ An optional `storageRefreshToken` query parameter can be provided to refresh credentials
+ for a staged table. See the `storageRefreshToken` parameter definition for detailed behavior.
+
+
+ The `planId` and `storageRefreshToken` parameters are mutually exclusive. If both are
Review Comment:
why are they mutually exclusive? A client will most likely send both to maintain backwards compatibility depending on whether it's talking to an older server that only supports the `planId` or to a newer server. Newer servers would then still most likely support `planId` if they supported it previously.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] REST Spec: support credential refresh on staged tables [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #15280:
URL: https://github.com/apache/iceberg/pull/15280#discussion_r3086555218
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1227,7 +1234,17 @@ paths:
type: string
description: The plan ID that has been used for server-side scan planning
- $ref: '#/components/parameters/referenced-by'
- description: Load vended credentials for a table from the catalog.
+ - $ref: '#/components/parameters/storage-refresh-token'
Review Comment:
I don't think we actually want to have one refresh token for each individual storage credential. To me the refresh token is just passing some additional context around so that the server knows that a client is trying to refresh credentials within the context of X. So that means we would have a 1:N mapping between refresh token and one or more storage credentials (after all, the responses of different endpoint are capable of returning more than one storage credential for a given table)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org