You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "sungwy (via GitHub)" <gi...@apache.org> on 2026/09/12 19:16:51 UTC
[PR] SPEC: add `data-access` to `signRequest `, and `presigned-urls` to `data-access` [iceberg]
sungwy opened a new pull request, #18080:
URL: https://github.com/apache/iceberg/pull/18080
Adds a new enum value `presigned-urls` to `data-access` or `X-Iceberg-Access-Delegation` header.
This new value can be used as the existing capability negotiation mechanism by the client to signal to the server that it supports `presigned-urls`.
In addition, add `data-access` to `signRequest` operation as a selection mechanism for the client to request a specific mode of signing (`presigned-urls` vs `remote-signing` aka signed header mode) when using the `signRequest` operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4382122793
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,34 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs a request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` reports the mechanism that produced the signature in
+ `signing-mechanism`. With `remote-signing`, `headers` carries the signature. With
+ `remote-presigning`, `uri` carries the signature.
+
+
+ On this operation `X-Iceberg-Access-Delegation` lists the access delegation mechanisms
+ the client supports for this request: `remote-signing`, `remote-presigning`, or both.
+ Other values, such as `vended-credentials`, are not supported on this operation and do
+ not count as listed mechanisms. The server SHOULD answer with one of the listed
+ mechanisms, and chooses when both are listed. The server SHOULD respond with 406, rather
+ than answer with another mechanism, when the header lists no mechanism the server
+ implements. For example, `vended-credentials` lists no mechanism, and
+ `vended-credentials,remote-signing` lists only `remote-signing`.
Review Comment:
@danielcweeks - This section of the description illustrates how the protocol handles invalid mechanisms like `vended-credentials` when it is included in `X-Iceberg-Access-Delegation` on this specific `operationId`.
I understand your point regarding the execution of pre-signed url and remote-signed-url, but how is that relevant here? Just want to make sure I didn't miss your point
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4222060936
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,31 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms, reported in `signing-mechanism`. With
+ `remote-signing`, `headers` carries the signature. With `remote-presigning`, `uri`
+ carries the signature.
+
+
+ `X-Iceberg-Access-Delegation` on this operation is not a capability negotiation.
+ When used, the values are the forms the client can consume for this request, and the
+ server SHOULD answer in one of them. A server that implements none of the listed
+ modes SHOULD respond with 406 rather than answer in another form.
+
+
+ Only `remote-signing` and `remote-presigning` are evaluated on this operation. A
+ request that lists both accepts either form and the server chooses. When no mode is
+ present the request is treated as `remote-signing`.
Review Comment:
adopted this feedback
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `presigned-urls` to `data-access` [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4023079638
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -2168,6 +2188,7 @@ components:
enum:
- vended-credentials
- remote-signing
+ - presigned-urls
Review Comment:
Minor: I feel like `remote-presigning` would be more consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `presigned-urls` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4026983127
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
Review Comment:
Thanks @danielcweeks - I had used an empty `headers` as the way to tell the pre-signed forms apart, because I think it'll be useful for the client to verify that the response is in the mode it wants for the specific use case.
I agree with you that we can simply require that the headers are sent in the request to be more consistent with the protocol. And we could consider giving up on the client being able to reliably distinguish which mode of signing it had received from the REST server.
I'll add this as a future discussion point for the FILE access governance discussions at the FILE sync.
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
+
+
+ `X-Iceberg-Access-Delegation` on this operation is not a capability negotiation.
+ When used, the client selects the form of the result for this request, and the
+ server SHOULD answer in that form. A server that does not implement the requested
+ mode SHOULD respond with 406 rather than answer in another form.
+
+
+ Only `remote-signing` and `presigned-urls` are evaluated on this operation. A request
+ that selects both modes is malformed, 400. When no mode is present the request is
+ treated as `remote-signing`.
Review Comment:
Agree, I can see a client doing that. If a client doesn't want to ask for both, it'll just send one mode in the header.
I agree it's simpler to keep this consistent with the original semantic of X-Iceberg-Access-Delegation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `presigned-urls` to `data-access` [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4028000115
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
Review Comment:
I think the request as provided should be executable regardless. The implementation should be able to distinguish a presigned from remote signed request. This distinction is trivial and we shouldn't rely on side effects (like empty headers) as a signalling mechanism.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `presigned-urls` to `data-access` [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4028000115
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
Review Comment:
I think the request as provided should be executable regardless. The implementation should be able to distinguish a presigned from remote signed request. This distinction is trivial and we should rely on side effects (like empty headers) as a signalling mechanism.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `presigned-urls` to `data-access` [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#issuecomment-5710355945
@sungwy and @singhpk234 we need to align this and #17332
I think this PR has the right property names, but is missing optional fields in the response.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub) via issues" <is...@iceberg.apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4518004626
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1410,34 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs a request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` reports the mechanism that produced the signature in
+ `signing-mechanism`. With `remote-signing`, `headers` carries the signature. With
+ `remote-presigning`, `uri` carries the signature.
+
+
+ On this operation `X-Iceberg-Access-Delegation` lists the access delegation mechanisms
+ the client supports for this request: `remote-signing`, `remote-presigning`, or both.
+ Other values, such as `vended-credentials`, are not supported on this operation and do
+ not count as listed mechanisms. The server SHOULD answer with one of the listed
+ mechanisms, and chooses when both are listed. The server SHOULD respond with 406, rather
+ than answer with another mechanism, when the header lists no mechanism the server
+ implements. For example, `vended-credentials` lists no mechanism, and
+ `vended-credentials,remote-signing` lists only `remote-signing`.
Review Comment:
Yes that's right - the current text states that in my opinion:
```
The server SHOULD respond with 406, rather
than answer with another mechanism, when the header lists no mechanism the server
implements. For example, `vended-credentials` lists no mechanism, and
`vended-credentials,remote-signing` lists only `remote-signing`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub) via issues" <is...@iceberg.apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4518699964
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1410,34 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
Review Comment:
Hi @adutra, sorry for the confusion, and thanks for catching this. The pre-signing client in #18110 wrongly sends the storage location (`s3://bucket/key`), which leaves the http url conversion to the server. That was my initial assumption, but as you said, it makes sense to keep the semantic consistent with existing remote-signing: the client builds the request URL, and the server only signs the request, for both mechanisms this endpoint supports.
I have a branch in progress that does this, which I'll force-push to #18110's branch once it's ready: https://github.com/sungwy/iceberg/compare/main...presign-from-client-v2. It builds on #17457's HTTP read path, so the first commit in that is William's change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4221240559
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,31 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms, reported in `signing-mechanism`. With
+ `remote-signing`, `headers` carries the signature. With `remote-presigning`, `uri`
+ carries the signature.
Review Comment:
Yea I kept this as I think it is informational, and I think it's helpful to distinguish why anyone server and client would want to support two different modes with the same shape of the payload.
The point that @danielcweeks raised earlier was about the requirement to have an empty header for `remote-presigning`. I agree with you folks on that and I've already removed that requirement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub) via issues" <is...@iceberg.apache.org>.
sungwy commented on PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#issuecomment-6665885128
@singhpk234 @danielcweeks - do you have any additional feedback on this PR? Shall we put it up for VOTE?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "adutra (via GitHub) via issues" <is...@iceberg.apache.org>.
adutra commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4515304259
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1410,34 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
Review Comment:
Since we are reusing the same endpoint to sign and pre-sign, I think it could be good to clarify what is the expected input URL scheme for each delegation mode.
With `remote-signing` the expectation is that the incoming URL is an HTTP(S) URL, as computed by the client SDK from a virtual storage location. IOW, we expect `https://bucket.s3.us-east-1.amazonaws.com/key` and not `s3://bucket/key`. This is why we have `uri` and also `method`.
But I'm seeing evidence that we might be designing `remote-presigning` to work with S3 input URLs instead: [a separate signer client was proposed](https://github.com/apache/iceberg/pull/18110#discussion_r4023283291) and it is suggested that this endpoint [should be used to presign FILE type urls as well](https://github.com/apache/iceberg/pull/17332#discussion_r3992139348).
So, it seems to me that with presign, we are expecting the flow to be stg like: a FILE column stores `s3://bucket/images/123.jpg`; at read time, the client asks the catalog to pre-sign that location; the result is stg like `https://…?X-Amz-Signature=…` which is passed to the "inference service" and then thrown away.
But if the goal is indeed to provide an `s3://...` URL and get a pre-signed `https://...` URL back, I'd advocate that this endpoint is not suitable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub) via issues" <is...@iceberg.apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4517970232
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -5847,6 +5872,11 @@ components:
description: The storage provider for which the request is to be signed. The provider should correspond to
the scheme used for a storage native URI. For example `s3` for AWS S3 paths. For backwards compatibility,
if this is not specified, the provider is assumed to be `s3`.
+ plan-id:
+ type: string
+ description: >
+ Optional identifier of the server side scan plan, supplied for audit and correlation. A server
+ MUST NOT fail a request because the referenced plan is unknown, expired, or cancelled.
Review Comment:
I carried this bit over from the `plan-id` related semantics of @williamhyun 's original [PR: 17332](https://github.com/apache/iceberg/pull/17332/changes#diff-02549ca620d020dc9ead80088cc14e311e12a69651fa8d394cd41a4308debb2eR1425-R1427), I see your point though - I agree that it makes sense to not be prescriptive here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4086379292
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
Review Comment:
Yes, that's what I meant in my comment @danielcweeks - I'm using this as an example to support the existing protocol (leaving it to the catalog to send the header regardless of mode)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4086379292
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
Review Comment:
Yes, that's what I meant in my comment @danielcweeks - I'm using this as an example supports the current protocol
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `presigned-urls` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4040124519
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
Review Comment:
@danielcweeks I thought about this more, and I also see use cases where we may want to keep the headers in the response when using presigned URL with FILE: https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
```
Q: Why am I getting an AccessDenied error with HeadersNotSigned: if-range?
When Range is included in X-Amz-SignedHeaders, Amazon S3 requires that If-Range also be signed if present in the request. Add If-Range to X-Amz-SignedHeaders when you generate the presigned URL.
```
If a FILE value has an offset, I think we want to keep the protocol open to being able to support presigned urls that are bound to a `Range`. I agree fully that keeping the spec generalized in the current form across both modes makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4361418141
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,34 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs a request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` reports the mechanism that produced the signature in
+ `signing-mechanism`. With `remote-signing`, `headers` carries the signature. With
+ `remote-presigning`, `uri` carries the signature.
+
+
+ On this operation `X-Iceberg-Access-Delegation` lists the access delegation mechanisms
+ the client supports for this request: `remote-signing`, `remote-presigning`, or both.
+ Other values, such as `vended-credentials`, are not supported on this operation and do
+ not count as listed mechanisms. The server SHOULD answer with one of the listed
+ mechanisms, and chooses when both are listed. The server SHOULD respond with 406, rather
+ than answer with another mechanism, when the header lists no mechanism the server
+ implements. For example, `vended-credentials` lists no mechanism, and
+ `vended-credentials,remote-signing` lists only `remote-signing`.
Review Comment:
I'm not sure this is necessary and it would be good to get confirmation on this point. I believe that pre-signed url and a remote signed url will be equally executable by the client. The response is just taken and executed as provided, I think it should work (it's possible the mechanics of the SDK prevent this, but conceptually, they should be interchangeable).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4086379292
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
Review Comment:
Yes, that's what I meant in my comment @danielcweeks - I'm using this as an example to support the existing protocol (always forwarding header regardless of mode)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `presigned-urls` to `data-access` [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4023094837
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
Review Comment:
> and `headers` is empty.
This is incorrect. The headers should be included with the request. We shouldn't ignore them just because the url is signed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "danielcweeks (via GitHub) via issues" <is...@iceberg.apache.org>.
danielcweeks commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4499163881
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -5847,6 +5872,11 @@ components:
description: The storage provider for which the request is to be signed. The provider should correspond to
the scheme used for a storage native URI. For example `s3` for AWS S3 paths. For backwards compatibility,
if this is not specified, the provider is assumed to be `s3`.
+ plan-id:
+ type: string
+ description: >
+ Optional identifier of the server side scan plan, supplied for audit and correlation. A server
+ MUST NOT fail a request because the referenced plan is unknown, expired, or cancelled.
Review Comment:
Agree with @singhpk234 here. We shouldn't be directing implementations that they must not fail requests (in general).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "adutra (via GitHub) via issues" <is...@iceberg.apache.org>.
adutra commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4515304259
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1410,34 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
Review Comment:
Since we are reusing the same endpoint to sign and pre-sign, I think it could be good to clarify what is the expected input URL scheme for each delegation mode.
With `remote-signing` the expectation is that the incoming URL is an HTTP(S) URL, as computed by the client SDK from a virtual storage location. IOW, we expect `https://bucket.s3.us-east-1.amazonaws.com/key` and not `s3://bucket/key`. This is why we have `uri` and also `method`.
But I'm seeing evidence that we might be designing `remote-presigning` to work with S3 input URLs instead: [a separate signer client was proposed](https://github.com/apache/iceberg/pull/18110#discussion_r4023283291) and it is suggested that this endpoint [should be used to presign FILE type urls as well](https://github.com/apache/iceberg/pull/17332#discussion_r3992139348).
So, it seems to me that with presign, we are expecting the flow to be stg like: a FILE column stores `s3://bucket/images/123.jpg`; at read time, the client asks the catalog to pre-sign that location; the result is stg like `https://…?X-Amz-Signature=…` which is passed to the "inference service" and then thrown away.
But if the goal is indeed to provide an `s3://...` URL and get a pre-signed `https://...` URL back, I'd advocate that this endpoint is not suitable, because it's not what it does today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4382122793
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,34 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs a request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` reports the mechanism that produced the signature in
+ `signing-mechanism`. With `remote-signing`, `headers` carries the signature. With
+ `remote-presigning`, `uri` carries the signature.
+
+
+ On this operation `X-Iceberg-Access-Delegation` lists the access delegation mechanisms
+ the client supports for this request: `remote-signing`, `remote-presigning`, or both.
+ Other values, such as `vended-credentials`, are not supported on this operation and do
+ not count as listed mechanisms. The server SHOULD answer with one of the listed
+ mechanisms, and chooses when both are listed. The server SHOULD respond with 406, rather
+ than answer with another mechanism, when the header lists no mechanism the server
+ implements. For example, `vended-credentials` lists no mechanism, and
+ `vended-credentials,remote-signing` lists only `remote-signing`.
Review Comment:
@danielcweeks - This section of the description illustrates how the protocol handles invalid mechanisms like `vended-credentials` when it is included in `X-Iceberg-Access-Delegation`.
I understand your point regarding the execution of pre-signed url and remote-signed-url, but how is that relevant here? Just want to make sure I didn't miss your point
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "singhpk234 (via GitHub)" <gi...@apache.org>.
singhpk234 commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4201008113
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,31 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
Review Comment:
```suggestion
Signs a request to object storage on the client's behalf.
```
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,31 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms, reported in `signing-mechanism`. With
+ `remote-signing`, `headers` carries the signature. With `remote-presigning`, `uri`
+ carries the signature.
+
+
+ `X-Iceberg-Access-Delegation` on this operation is not a capability negotiation.
+ When used, the values are the forms the client can consume for this request, and the
+ server SHOULD answer in one of them. A server that implements none of the listed
+ modes SHOULD respond with 406 rather than answer in another form.
+
+
+ Only `remote-signing` and `remote-presigning` are evaluated on this operation. A
+ request that lists both accepts either form and the server chooses. When no mode is
+ present the request is treated as `remote-signing`.
Review Comment:
do we wanna add this is for the backward compatibility ?
what happens some one sends in this header vended-creds ? what should we do, earlier this was not possible
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,31 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms, reported in `signing-mechanism`. With
+ `remote-signing`, `headers` carries the signature. With `remote-presigning`, `uri`
+ carries the signature.
Review Comment:
should we enforce that ? i wonder if we should just say here the we return both headers and uri, i like what @danielcweeks suggested we can in our reference impl pass in both header and the uri completely
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -5841,6 +5480,11 @@ components:
description: The storage provider for which the request is to be signed. The provider should correspond to
the scheme used for a storage native URI. For example `s3` for AWS S3 paths. For backwards compatibility,
if this is not specified, the provider is assumed to be `s3`.
+ plan-id:
+ type: string
+ description: >
+ Optional identifier of the scan plan, supplied for audit and correlation. A server
Review Comment:
```suggestion
Optional identifier of the server side scan plan, supplied for audit and correlation. A server
```
wonder do we need to add more details on what this is for ?
for example in creds endpoint we dont do this, we can just call here, wdyt ? https://github.com/apache/iceberg/blob/4a84ba420e51242ee6bb16c71b849d91e81d7d71/open-api/rest-catalog-open-api.yaml#L1379
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4382122793
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,34 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs a request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` reports the mechanism that produced the signature in
+ `signing-mechanism`. With `remote-signing`, `headers` carries the signature. With
+ `remote-presigning`, `uri` carries the signature.
+
+
+ On this operation `X-Iceberg-Access-Delegation` lists the access delegation mechanisms
+ the client supports for this request: `remote-signing`, `remote-presigning`, or both.
+ Other values, such as `vended-credentials`, are not supported on this operation and do
+ not count as listed mechanisms. The server SHOULD answer with one of the listed
+ mechanisms, and chooses when both are listed. The server SHOULD respond with 406, rather
+ than answer with another mechanism, when the header lists no mechanism the server
+ implements. For example, `vended-credentials` lists no mechanism, and
+ `vended-credentials,remote-signing` lists only `remote-signing`.
Review Comment:
@danielcweeks - This section of the description illustrates how the protocol handles invalid mechanisms like `vended-credentials` when it is included in `X-Iceberg-Access-Delegation` on this specific `operationId`.
Agreed on the execution model for pre-signed and remote-signed URLs. I’m not seeing yet how that changes this section, though. Could you expand on the connection? I may be missing something.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4382562405
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,34 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs a request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` reports the mechanism that produced the signature in
+ `signing-mechanism`. With `remote-signing`, `headers` carries the signature. With
+ `remote-presigning`, `uri` carries the signature.
+
+
+ On this operation `X-Iceberg-Access-Delegation` lists the access delegation mechanisms
+ the client supports for this request: `remote-signing`, `remote-presigning`, or both.
+ Other values, such as `vended-credentials`, are not supported on this operation and do
+ not count as listed mechanisms. The server SHOULD answer with one of the listed
+ mechanisms, and chooses when both are listed. The server SHOULD respond with 406, rather
+ than answer with another mechanism, when the header lists no mechanism the server
+ implements. For example, `vended-credentials` lists no mechanism, and
+ `vended-credentials,remote-signing` lists only `remote-signing`.
Review Comment:
Are you suggesting that we don't have a separate enum value for `remote-presigning` vs `remote-signing`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4075653613
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
Review Comment:
@sungwy I don't see how this example would affect the protocol. The catalog service is responsible for generating the appropriate request (including signed headers). Regardless of whether the request is remote signed or pre-signed, the expectations should be that the returned headers should be included in the request and the URI should be used as the path of the request.
Ensuring this is a valid operation is up to the catalog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `presigned-urls` to `data-access` [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4023245265
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1400,35 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs one request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` takes one of two forms. With `remote-signing`, `uri` is the
+ requested URI and `headers` carries the signature. With `presigned-urls`, `uri`
+ carries the signature and `headers` is empty.
+
+
+ `X-Iceberg-Access-Delegation` on this operation is not a capability negotiation.
+ When used, the client selects the form of the result for this request, and the
+ server SHOULD answer in that form. A server that does not implement the requested
+ mode SHOULD respond with 406 rather than answer in another form.
+
+
+ Only `remote-signing` and `presigned-urls` are evaluated on this operation. A request
+ that selects both modes is malformed, 400. When no mode is present the request is
+ treated as `remote-signing`.
Review Comment:
I don't agree with this. If both modes are provided, it means that the client will support and use either one. The header was intended to allow for multiple modes, so that should be supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] SPEC: add `data-access` to `signRequest `, and `remote-presigning` to `data-access` [iceberg]
Posted by "singhpk234 (via GitHub) via issues" <is...@iceberg.apache.org>.
singhpk234 commented on code in PR #18080:
URL: https://github.com/apache/iceberg/pull/18080#discussion_r4499135983
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -1410,12 +1410,34 @@ paths:
- $ref: '#/components/parameters/prefix'
- $ref: '#/components/parameters/namespace'
- $ref: '#/components/parameters/table'
+ - $ref: '#/components/parameters/data-access'
post:
tags:
- Catalog API
summary: Remotely signs requests to object storage
operationId: signRequest
+ description: >
+ Signs a request to object storage on the client's behalf.
+
+
+ `RemoteSignResult` reports the mechanism that produced the signature in
+ `signing-mechanism`. With `remote-signing`, `headers` carries the signature. With
+ `remote-presigning`, `uri` carries the signature.
+
+
+ On this operation `X-Iceberg-Access-Delegation` lists the access delegation mechanisms
+ the client supports for this request: `remote-signing`, `remote-presigning`, or both.
+ Other values, such as `vended-credentials`, are not supported on this operation and do
+ not count as listed mechanisms. The server SHOULD answer with one of the listed
+ mechanisms, and chooses when both are listed. The server SHOULD respond with 406, rather
+ than answer with another mechanism, when the header lists no mechanism the server
+ implements. For example, `vended-credentials` lists no mechanism, and
+ `vended-credentials,remote-signing` lists only `remote-signing`.
Review Comment:
what if its 'vended-creds,remote-presigning' and the server doesn't support remote-presigning we expect server to give 406 right ?
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -5847,6 +5872,11 @@ components:
description: The storage provider for which the request is to be signed. The provider should correspond to
the scheme used for a storage native URI. For example `s3` for AWS S3 paths. For backwards compatibility,
if this is not specified, the provider is assumed to be `s3`.
+ plan-id:
+ type: string
+ description: >
+ Optional identifier of the server side scan plan, supplied for audit and correlation. A server
+ MUST NOT fail a request because the referenced plan is unknown, expired, or cancelled.
Review Comment:
Q: why can't server fail if the plan is cancelled if the plan is cancelled with a bad request ? i would rather just remove the statement of when to fail ?
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -5847,6 +5872,11 @@ components:
description: The storage provider for which the request is to be signed. The provider should correspond to
the scheme used for a storage native URI. For example `s3` for AWS S3 paths. For backwards compatibility,
if this is not specified, the provider is assumed to be `s3`.
+ plan-id:
+ type: string
+ description: >
+ Optional identifier of the server side scan plan, supplied for audit and correlation. A server
+ MUST NOT fail a request because the referenced plan is unknown, expired, or cancelled.
Review Comment:
why can't server fail if the plan is cancelled if the plan is cancelled with a bad request ? i would rather just remove the statement of when to fail ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org