You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/09/03 08:58:53 UTC
[GitHub] [hadoop-ozone] elek opened a new pull request #1384: HDDS-4193. Range used by S3 MultipartUpload copy-from-source should be inclusive.
elek opened a new pull request #1384:
URL: https://github.com/apache/hadoop-ozone/pull/1384
## What changes were proposed in this pull request?
S3 API provides a feature to copy a specific range from an existing key.
Based on the documentation, this range definitions is inclusive:
https://docs.aws.amazon.com/cli/latest/reference/s3api/upload-part-copy.html
>
> -copy-source-range (string)
>
> The range of bytes to copy from the source object. The range value must use the form bytes=first-last, where the first and last are the zero-based byte offsets to copy. For example, bytes=0-9 indicates that you want to copy the first 10 bytes of the source. You can copy a range only if the source object is greater than 5 MB.
>
But as it's visible from our robot test, in our case we use exclusive range:
```
upload-part-copy ... --copy-source-range bytes=0-10485758
upload-part-copy ... --copy-source-range bytes=10485758-10485760
```
Based on this AWS documentation it will return with a (10485758 + 1) + 3 bytes long key, which is impossible if our original source key is just 10485760.
I think the right usage to get the original key is the following:
```shell
upload-part-copy ... --copy-source-range bytes=0-10485757
upload-part-copy ... --copy-source-range bytes=10485758-10485759
```
(Note, this bug is found with the script in HDDS-4194, which showed that AWS S3 is working in different way).
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4193
## How was this patch tested?
Unit test and acceptance tests are updated.
The truth of the specification is tested with https://issues.apache.org/jira/browse/HDDS-4194
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek edited a comment on pull request #1384: HDDS-4193. Range used by S3 MultipartUpload copy-from-source should be inclusive.
Posted by GitBox <gi...@apache.org>.
elek edited a comment on pull request #1384:
URL: https://github.com/apache/hadoop-ozone/pull/1384#issuecomment-688251278
Thanks the review @ChenSammi. I am merging it now (only the coverage upload is failed which was fixed by HDDS-4205)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #1384: HDDS-4193. Range used by S3 MultipartUpload copy-from-source should be inclusive.
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #1384:
URL: https://github.com/apache/hadoop-ozone/pull/1384#issuecomment-688251278
Thanks the review @ChenSammi. I am merging it now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek merged pull request #1384: HDDS-4193. Range used by S3 MultipartUpload copy-from-source should be inclusive.
Posted by GitBox <gi...@apache.org>.
elek merged pull request #1384:
URL: https://github.com/apache/hadoop-ozone/pull/1384
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] ChenSammi commented on pull request #1384: HDDS-4193. Range used by S3 MultipartUpload copy-from-source should be inclusive.
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on pull request #1384:
URL: https://github.com/apache/hadoop-ozone/pull/1384#issuecomment-686883510
+1. LGTM. Thanks @elek.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org