You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@jclouds.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/07/10 02:36:00 UTC
[jira] [Commented] (JCLOUDS-1516) First putblob should be signed with specific region rather than with default region during createcontainer API

    [ https://issues.apache.org/jira/browse/JCLOUDS-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378380#comment-17378380 ] 

ASF subversion and git services commented on JCLOUDS-1516:
----------------------------------------------------------

Commit 779bc2db19dd36f40cb5668c2c41620f0e74b8c4 in jclouds's branch refs/heads/master from didixith
[ https://gitbox.apache.org/repos/asf?p=jclouds.git;h=779bc2d ]

JCLOUDS-1516: specify host name when creating bucket



> First putblob should be signed with specific region rather than with default region during createcontainer API 
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: JCLOUDS-1516
>                 URL: https://issues.apache.org/jira/browse/JCLOUDS-1516
>             Project: jclouds
>          Issue Type: New Feature
>          Components: jclouds-blobstore
>    Affects Versions: 2.1.2
>         Environment: Linux
>            Reporter: Dileep Dixith
>            Priority: Major
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When container in non default region exists, only first time put operation will be applied to find out whether bucket exists and have proper access or not. Aws sigv4 signature will be created based on the default region only as create bucket method was not honoring the region specified.
> So, Put request was first signed with default region and if the user does not have access to default(us-east-1) region,. Then it will be re-directed to sa-east-1 region, but the request is signed with us-east-1, the request is rejected and throws "The authorization header is malformed".
> Flow in case of user has access to default region:
> 17:22:24.460 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking CreateBucket
> 17:22:24.460 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking CreateBucket
> 17:22:24.461 [bscThread-02] DEBUG jclouds.signature - << PUT
> /
> content-length:105
> content-type:text/xml
> host:test3.s3.amazonaws.com
> x-amz-content-sha256:f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> x-amz-date:20190818T115218Z
> content-length;content-type;host;x-amz-content-sha256;x-amz-date
> f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:22:24.461 [bscThread-02] DEBUG jclouds.signature - << AWS4-HMAC-SHA256
> 20190818T115218Z
> 20190818/us-east-1/s3/aws4_request
> 089a5248f5eff6e8b6378154acdf07bff7d208029c98c67af44c99b4a8f2df39
> 17:22:24.463 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Sending request -1533211628: PUT https://test3.s3.amazonaws.com/ HTTP/1.1
> 17:22:24.463 [bscThread-02] DEBUG jclouds.wire - >> "<CreateBucketConfiguration><LocationConstraint>sa-east-1</LocationConstraint></CreateBucketConfiguration>"
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> PUT https://test3.s3.amazonaws.com/ HTTP/1.1
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Host: test3.s3.amazonaws.com
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> x-amz-content-sha256: f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> X-Amz-Date: 20190818T115218Z
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Authorization: AWS4-HMAC-SHA256 Credential=AKIAIGKQ7V52FQQJFYJQ/20190818/us-east-1/s3/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=637d42fbf6684430ab0f08fd82cbae69f3261859e0031ad40054bccb829473da
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:22:24.464 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:22:25.671 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Receiving response -1533211628: HTTP/1.1 409 Conflict
> 17:22:25.671 [bscThread-02] DEBUG jclouds.headers - << HTTP/1.1 409 Conflict
> 17:22:25.672 [bscThread-02] DEBUG jclouds.headers - << Transfer-Encoding: chunked
> 17:22:25.672 [bscThread-02] DEBUG jclouds.headers - << Server: AmazonS3
> 17:22:25.672 [bscThread-02] DEBUG jclouds.headers - << x-amz-request-id: 09E5163C51F25F34
> 17:22:25.673 [bscThread-02] DEBUG jclouds.headers - << x-amz-id-2: WuN84GYMs47Nn6+48XYDpLZNvp0NPszokKyhxlzZk+ub8RhjbLpkfEI8E2tKWVCFKtJiXrhdpkc=
> 17:22:25.673 [bscThread-02] DEBUG jclouds.headers - << Date: Sun, 18 Aug 2019 11:52:11 GMT
> 17:22:25.673 [bscThread-02] DEBUG jclouds.headers - << x-amz-bucket-region: sa-east-1
> 17:22:25.673 [bscThread-02] DEBUG jclouds.headers - << Content-Type: application/xml
> 17:22:25.673 [bscThread-02] DEBUG jclouds.wire - << "<?xml version="1.0" encoding="UTF-8"?>[\n]"
> 17:22:25.673 [bscThread-02] DEBUG jclouds.wire - << "<Error><Code>BucketAlreadyOwnedByYou</Code><Message>Your previous request to create the named bucket succeeded and you already own it.</Message><BucketName>test3</BucketName><RequestId>09E5163C51F25F34</RequestId><HostId>WuN84GYMs47Nn6+48XYDpLZNvp0NPszokKyhxlzZk+ub8RhjbLpkfEI8E2tKWVCFKtJiXrhdpkc=</HostId></Error>"
> 17:22:25.680 [bscThread-03] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking BucketExists
> 17:22:25.681 [bscThread-03] DEBUG jclouds.signature - << HEAD
> /
> Flow in case of user has access to default region
> host:test3.s3-sa-east-1.amazonaws.com
> x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
> x-amz-date:20190818T115218Z
>  
> Flow in case of user does not have access to default region.
> 17:28:41.464 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking CreateBucket
> 17:28:41.464 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking CreateBucket
> 17:28:41.466 [bscThread-02] DEBUG jclouds.signature - << PUT
> /
> content-length:105
> content-type:text/xml
> host:test-poc-spectrum-scale.s3.amazonaws.com
> x-amz-content-sha256:f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> x-amz-date:20190818T115835Z
> content-length;content-type;host;x-amz-content-sha256;x-amz-date
> f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:28:41.466 [bscThread-02] DEBUG jclouds.signature - << AWS4-HMAC-SHA256
> 20190818T115835Z
> 20190818/us-east-1/s3/aws4_request
> d8c68a44175c3b0b49182ca5658bff291b3f382d7d02b3fe25d6970912df1697
> 17:28:41.467 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Sending request -541426552: PUT https://test-poc-spectrum-scale.s3.amazonaws.com/ HTTP/1.1
> 17:28:41.467 [bscThread-02] DEBUG jclouds.wire - >> "<CreateBucketConfiguration><LocationConstraint>sa-east-1</LocationConstraint></CreateBucketConfiguration>"
> 17:28:41.467 [bscThread-02] DEBUG jclouds.headers - >> PUT https://test-poc-spectrum-scale.s3.amazonaws.com/ HTTP/1.1
> 17:28:41.467 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> Host: test-poc-spectrum-scale.s3.amazonaws.com
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> x-amz-content-sha256: f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> X-Amz-Date: 20190818T115835Z
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> Authorization: AWS4-HMAC-SHA256 Credential=AKIA25YU33MFMAZO3B7F/20190818/us-east-1/s3/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=8cf0d9f58bd0ba5fc607f5018e701cdf14587741169ba7deaaa7a9cad0548cb6
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:28:41.468 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:28:43.016 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Receiving response -541426552: HTTP/1.1 400 Bad Request
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << HTTP/1.1 400 Bad Request
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << Transfer-Encoding: chunked
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << Server: AmazonS3
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << Connection: close
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << x-amz-request-id: 43F137234826AA08
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << x-amz-id-2: nzJhRvv6j87VgSKV/mlGNhAqdietQCH4V5ArVUdIX2j1SnTChg2t1/GH20Zs3iz6Bc0MPzzAzwc=
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << Date: Sun, 18 Aug 2019 11:58:29 GMT
> 17:28:43.017 [bscThread-02] DEBUG jclouds.headers - << Content-Type: application/xml
> 17:28:43.019 [bscThread-02] DEBUG jclouds.wire - << "<?xml version="1.0" encoding="UTF-8"?>[\n]"
> 17:28:43.019 [bscThread-02] DEBUG jclouds.wire - << "<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'sa-east-1'</Message><Region>sa-east-1</Region><RequestId>43F137234826AA08</RequestId><HostId>nzJhRvv6j87VgSKV/mlGNhAqdietQCH4V5ArVUdIX2j1SnTChg2t1/GH20Zs3iz6Bc0MPzzAzwc=</HostId></Error>"
> EXC:org.jclouds.aws.AWSResponseException: request PUT https://test-poc-spectrum-scale.s3.amazonaws.com/ HTTP/1.1 failed with code 400, error: AWSError\{requestId='43F137234826AA08', requestToken='nzJhRvv6j87VgSKV/mlGNhAqdietQCH4V5ArVUdIX2j1SnTChg2t1/GH20Zs3iz6Bc0MPzzAzwc=', code='AuthorizationHeaderMalformed', message='The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'sa-east-1'', context='{Region=sa-east-1, HostId=nzJhRvv6j87VgSKV/mlGNhAqdietQCH4V5ArVUdIX2j1SnTChg2t1/GH20Zs3iz6Bc0MPzzAzwc=}'}
> I have a fix so that the first put operation honors the custom region specified and request will be signed with specific region rather than with default region. Tested with default region as well and worked well.
> After my fix, flow with stand-alone program is as below.
> 17:36:09.043 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking CreateBucket
> 17:36:09.043 [bscThread-02] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking CreateBucket
> 17:36:09.044 [bscThread-02] DEBUG jclouds.signature - << PUT
> /
> content-length:105
> content-type:text/xml
> host:test-poc-spectrum-scale.s3-sa-east-1.amazonaws.com
> x-amz-content-sha256:f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> x-amz-date:20190818T120602Z
> content-length;content-type;host;x-amz-content-sha256;x-amz-date
> f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:36:09.044 [bscThread-02] DEBUG jclouds.signature - << AWS4-HMAC-SHA256
> 20190818T120602Z
> 20190818/sa-east-1/s3/aws4_request
> 2996a3b4d85ab8c68e1378ab68db2f7d80b244969d1792391a49b9b7390bc920
> 17:36:09.046 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Sending request -719121422: PUT https://test-poc-spectrum-scale.s3-sa-east-1.amazonaws.com/ HTTP/1.1
> 17:36:09.046 [bscThread-02] DEBUG jclouds.wire - >> "<CreateBucketConfiguration><LocationConstraint>sa-east-1</LocationConstraint></CreateBucketConfiguration>"
> 17:36:09.046 [bscThread-02] DEBUG jclouds.headers - >> PUT https://test-poc-spectrum-scale.s3-sa-east-1.amazonaws.com/ HTTP/1.1
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Host: test-poc-spectrum-scale.s3-sa-east-1.amazonaws.com
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> x-amz-content-sha256: f5d7dd57e1e23b516fc3543b9f24fc19a8409557905f48c6f412b3a67946ce96
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> X-Amz-Date: 20190818T120602Z
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Authorization: AWS4-HMAC-SHA256 Credential=AKIA25YU33MFMAZO3B7F/20190818/sa-east-1/s3/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=982a3a62e85e03c68f4ac1e0da5cfa753cb81cf750a6fbff157b34681ed54774
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Content-Type: text/xml
> 17:36:09.047 [bscThread-02] DEBUG jclouds.headers - >> Content-Length: 105
> 17:36:11.076 [bscThread-02] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Receiving response -719121422: HTTP/1.1 409 Conflict
> 17:36:11.076 [bscThread-02] DEBUG jclouds.headers - << HTTP/1.1 409 Conflict
> 17:36:11.076 [bscThread-02] DEBUG jclouds.headers - << Transfer-Encoding: chunked
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << Server: AmazonS3
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << x-amz-request-id: 5FB8D9C1B41E48EB
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << x-amz-id-2: 54oqA5Lc+yl6Y9Ppz6kMd6hZs/iGFGfNFPqrOPX90Q731UH5KkYEYc7RS/4W9btahu0dxQKd3iI=
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << Date: Sun, 18 Aug 2019 12:05:57 GMT
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << x-amz-bucket-region: sa-east-1
> 17:36:11.077 [bscThread-02] DEBUG jclouds.headers - << Content-Type: application/xml
> 17:36:11.077 [bscThread-02] DEBUG jclouds.wire - << "<?xml version="1.0" encoding="UTF-8"?>[\n]"
> 17:36:11.077 [bscThread-02] DEBUG jclouds.wire - << "<Error><Code>BucketAlreadyOwnedByYou</Code><Message>Your previous request to create the named bucket succeeded and you already own it.</Message><BucketName>test-poc-spectrum-scale</BucketName><RequestId>5FB8D9C1B41E48EB</RequestId><HostId>54oqA5Lc+yl6Y9Ppz6kMd6hZs/iGFGfNFPqrOPX90Q731UH5KkYEYc7RS/4W9btahu0dxQKd3iI=</HostId></Error>"
> 17:36:11.083 [bscThread-03] DEBUG o.j.rest.internal.InvokeHttpMethod -



--
This message was sent by Atlassian Jira
(v8.3.4#803005)