You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Wei-Chiu Chuang (Jira)" <ji...@apache.org> on 2025/05/03 19:25:00 UTC
[jira] [Updated] (HDDS-12935) Fix S3G signature mismatch for STREAMING-UNSIGNED-PAYLOAD-TRAILER
[ https://issues.apache.org/jira/browse/HDDS-12935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wei-Chiu Chuang updated HDDS-12935:
-----------------------------------
Target Version/s: 2.0.1
> Fix S3G signature mismatch for STREAMING-UNSIGNED-PAYLOAD-TRAILER
> -----------------------------------------------------------------
>
> Key: HDDS-12935
> URL: https://issues.apache.org/jira/browse/HDDS-12935
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
> Labels: pull-request-available
>
> HDDS-12488 fixes the issue where the AWS checksum trailer data is silently appended to the final payload. This works for http endpoint which will by default provide with signed payload (i.e. x-amz-content-sha256 is set to the payload signature).
> When the request is sent against https endpoint, the x-amz-content-sha256 is set to STREAMING-UNSIGNED-PAYLOAD-TRAILER. HDDS-12488 handled it by using string "UNSIGNED-PAYLOAD: when building the canonical request (StringToSignProducer#buildCanonicalRequest) as specified in https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html. However, this causes the mismatch between the AWS SDK canonical request and the calculated canonical request which results in signature mismatch and all PutObject requests for the newer AWS SDK version to fail.
> Did manual testing
> AWS CLI canonical request
> {code}
> PUT
> /ivan-bucket/test4.txt
> content-encoding:aws-chunked
> host:<redacted>
> x-amz-content-sha256:STREAMING-UNSIGNED-PAYLOAD-TRAILER
> x-amz-date:20250430T071131Z
> x-amz-decoded-content-length:5
> x-amz-sdk-checksum-algorithm:CRC64NVME
> x-amz-trailer:x-amz-checksum-crc64nvme
> content-encoding;host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-sdk-checksum-algorithm;x-amz-trailer
> STREAMING-UNSIGNED-PAYLOAD-TRAILER
> {code}
> Ozone canonical request
> {code}
> PUT
> /ivan-bucket/test4.txt
> content-encoding:aws-chunked
> host:<redacted>
> x-amz-content-sha256:STREAMING-UNSIGNED-PAYLOAD-TRAILER
> x-amz-date:20250430T071131Z
> x-amz-decoded-content-length:5
> x-amz-sdk-checksum-algorithm:CRC64NVME
> x-amz-trailer:x-amz-checksum-crc64nvme
> content-encoding;host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-sdk-checksum-algorithm;x-amz-trailer
> UNSIGNED-PAYLOAD
> {code}
> AWS CLI StrToSign
> {code}
> AWS4-HMAC-SHA256
> 20250430T065600Z
> 20250430/us-east-1/s3/aws4_request
> a3b489c10cf8e4878c7d7b870d3eda15c9c17f52541bd4742361a538c2b458fe
> {code}
> Ozone StrToSign
> {code}
> AWS4-HMAC-SHA256
> 20250430T065600Z
> 20250430/us-east-1/s3/aws4_request
> cb995d592067f026dc30bb4416340afa7e9d38abee57e8e8a44a5604380a22ab
> {code}
> The correct behavior seems to use the "STREAMING-UNSIGNED-PAYLOAD-TRAILER" instead of "UNSIGNED-PAYLOAD" when building the canonical request.
> Ref: https://github.com/aws/aws-sdk-java-v2/blob/f6adeaa5b24a4c203106d122647537f83a5ecfbc/core/auth/src/main/java/software/amazon/awssdk/auth/signer/internal/AbstractAwsS3V4Signer.java#L206
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org