You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "jrandall (via GitHub)" <gi...@apache.org> on 2023/11/21 13:32:14 UTC
[I] Cannot set range, override_content_disposition, override_cache_control, or override_content_type for S3 presigned stat URLs [incubator-opendal]
jrandall opened a new issue, #3642:
URL: https://github.com/apache/incubator-opendal/issues/3642
I am building a service using opendal that refers certain GET requests to S3 using a presigned URL generated with the `Operator::presign_read_with` method and setting the range, override_content_disposition, override_cache_control, and override_content_type on the resulting `FuturePresignRead` object. This works great.
However, I would now like my service to implement HEAD for these requests. I have done that using the `Operator::presign_stat` method, and this does work. However, I have test cases that validate the behavior for HEAD as specified in RFC7231 section 4.3.2 (https://datatracker.ietf.org/doc/html/rfc7231#section-4.3.2):
```
The server SHOULD send the same
header fields in response to a HEAD request as it would have sent if
the request had been a GET, except that the payload header fields
([Section 3.3](https://datatracker.ietf.org/doc/html/rfc7231#section-3.3)) MAY be omitted
```
Since I am asking S3 to override response headers and support a Range header in the request, I need to be able to pass these to the stat (HEAD) request as well in order to pass this test. It appears that S3 supports this, it is just that opendal does not expose these interfaces for stat.
It would be useful if `Operator::presign_stat_with` and `opendal::operator_futures::FuturePresignStat` could both be implemented (though I believe it would be sensible for the latter to have exactly the same interface as `opendal::operator_futures::FuturePresignRead`, since a major use case for this would be to achieve parity between the results returned from a GET (S3 read) and HEAD (S3 stat) request).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cannot set range, override_content_disposition, override_cache_control, or override_content_type for S3 presigned stat URLs [incubator-opendal]
Posted by "jrandall (via GitHub)" <gi...@apache.org>.
jrandall commented on issue #3642:
URL: https://github.com/apache/incubator-opendal/issues/3642#issuecomment-1823299063
I agree the AWS API documentation is unclear on this point. The [HeadObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html) do say that "A HEAD request has the same options as a GET action on an object." which is consistent with the defined semantics of the HTTP method, but then they go on to enumerate [parameters](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html#API_HeadObject_RequestParameters) that do not match the [parameters listed for GetObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html#API_GetObject_RequestParameters).
However, I have already verified that passing the 'response-*' parameters defined for `GetObject` to `HeadObject` does in fact work with AWS S3. This is true both for presigned URLs as well as for normal requests.
My guess would be that when they added the 'response-*' parameters to the documentation for `GetObject`, they forgot to add them to the documentation for `HeadObject` as well, but that the actual implementation is shared between the two methods (as would be suggested by the blanket statement that they accept the same options). I suppose it would probably be a good idea to raise this with AWS so that they can fix the docs.
Note also that the open source (minio)[https://min.io/] server, which aims to be S3 compatible, also implements all of the `response-*` parameters for (both HEAD and GET)[https://github.com/minio/minio/blob/0b074d0fae3123205eb74529f14383158eba9e88/cmd/object-handlers.go#L65-L73].
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cannot set range, override_content_disposition, override_cache_control, or override_content_type for S3 presigned stat URLs [incubator-opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #3642:
URL: https://github.com/apache/incubator-opendal/issues/3642#issuecomment-1846662690
Let's go this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cannot set range, override_content_disposition, override_cache_control, or override_content_type for S3 presigned stat URLs [incubator-opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #3642:
URL: https://github.com/apache/incubator-opendal/issues/3642#issuecomment-1821064405
Thanks for starting this issue!
AWS S3 API docs doesn't mention that if `HeadObject` supports override `content_disposition` like in `GetObject`. I will give it a try first!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Cannot set range, override_content_disposition, override_cache_control, or override_content_type for S3 presigned stat URLs [incubator-opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo closed issue #3642: Cannot set range, override_content_disposition, override_cache_control, or override_content_type for S3 presigned stat URLs
URL: https://github.com/apache/incubator-opendal/issues/3642
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org