You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Bharat Viswanadham (Jira)" <ji...@apache.org> on 2021/11/12 21:19:00 UTC
[jira] [Commented] (HDDS-5980) MPU Key read on a key which is
overwritten on a TDE bucket the data does not match
[ https://issues.apache.org/jira/browse/HDDS-5980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442932#comment-17442932 ]
Bharat Viswanadham commented on HDDS-5980:
------------------------------------------
Scenario steps to repro:
{code:java}
###Create Key for the first time
bash-4.2$ alias s3api='aws s3api --endpoint http://s3g:9878'
bash-4.2$ s3api create-multipart-upload --bucket b1234 --key mpu
{
"Bucket": "b1234",
"UploadId": "83c54df0-17b3-4b09-9777-a13a5ace8943-107266114571141120",
"Key": "mpu"
}
bash-4.2$ dd if=/dev/urandom of=/tmp/part1 bs=1048576 count=5
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.154451 s, 33.9 MB/s
bash-4.2$ dd if=/dev/urandom of=/tmp/part2 bs=1048576 count=5
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.152602 s, 34.4 MB/s
bash-4.2$ export UPLOAD_ID=83c54df0-17b3-4b09-9777-a13a5ace8943-107266114571141120
bash-4.2$ s3api upload-part --bucket b1234 --key mpu --part-number 1 --body /tmp/part1 --upload-id=$UPLOAD_ID
{
"ETag": "/s3v/b1234/mpu-83c54df0-17b3-4b09-9777-a13a5ace8943-107266114571141120-1"
}
bash-4.2$ s3api upload-part --bucket b1234 --key mpu --part-number 2 --body /tmp/part2 --upload-id=$UPLOAD_ID
{
"ETag": "/s3v/b1234/mpu-83c54df0-17b3-4b09-9777-a13a5ace8943-107266114571141120-2"
}
bash-4.2$ s3api complete-multipart-upload --upload-id=$UPLOAD_ID --bucket b1234 --key mpu --multipart-upload 'Parts=[{ETag="/s3v/b1234/mpu-83c54df0-17b3-4b09-9777-a13a5ace8943-107266114571141120-1",PartNumber=1},{ETag="/s3v/b1234/mpu-83c54df0-17b3-4b09-9777-a13a5ace8943-107266114571141120-2",PartNumber=2}]'
{
"ETag": "a0a9dacf589e2b364e185673026930e425d93a0fb3603be952ea0b20537a0992",
"Bucket": "b1234",
"Location": "b1234",
"Key": "mpu"
}
bash-4.2$ cat /tmp/part1 /tmp/part2 >> /tmp/orig
bash-4.2$ diff /tmp/orig /tmp/testr
### Override Key
bash-4.2$ s3api create-multipart-upload --bucket b1234 --key mpu
{
"Bucket": "b1234",
"UploadId": "fb261147-a424-4519-8b81-a67e2fea386a-107266137300074504",
"Key": "mpu"
}
bash-4.2$ dd if=/dev/urandom of=/tmp/part1-1 bs=1048576 count=5
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.154172 s, 34.0 MB/s
bash-4.2$ dd if=/dev/urandom of=/tmp/part2-1 bs=1048576 count=5
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.155693 s, 33.7 MB/s
bash-4.2$ export UPLOAD_ID=fb261147-a424-4519-8b81-a67e2fea386a-107266137300074504
bash-4.2$ s3api upload-part --bucket b1234 --key mpu --part-number 1 --body /tmp/part1-1 --upload-id=$UPLOAD_ID
{
"ETag": "/s3v/b1234/mpu-fb261147-a424-4519-8b81-a67e2fea386a-107266137300074504-1"
}
bash-4.2$ s3api upload-part --bucket b1234 --key mpu --part-number 2 --body /tmp/part2-1 --upload-id=$UPLOAD_ID
{
"ETag": "/s3v/b1234/mpu-fb261147-a424-4519-8b81-a67e2fea386a-107266137300074504-2"
}
bash-4.2$ s3api complete-multipart-upload --upload-id=$UPLOAD_ID --bucket b1234 --key mpu --multipart-upload 'Parts=[{ETag="/s3v/b1234/mpu-fb261147-a424-4519-8b81-a67e2fea386a-107266137300074504-1",PartNumber=1},{ETag="/s3v/b1234/mpu-fb261147-a424-4519-8b81-a67e2fea386a-107266137300074504-2",PartNumber=2}]'
{
"ETag": "a0a9dacf589e2b364e185673026930e425d93a0fb3603be952ea0b20537a0992",
"Bucket": "b1234",
"Location": "b1234",
"Key": "mpu"
}
bash-4.2$ s3api get-object --bucket b1234 --key mpu /tmp/testr1
{
"AcceptRanges": "bytes",
"ContentType": "application/octet-stream",
"LastModified": "Fri, 12 Nov 2021 21:11:14 GMT",
"ContentLength": 10485760,
"Expires": "Fri, 12 Nov 2021 21:11:22 GMT",
"CacheControl": "no-cache",
"Metadata": {}
}
bash-4.2$ cat /tmp/part1-1 /tmp/part2-1 >> /tmp/orig1
bash-4.2$ diff /tmp/orig1 /tmp/testr1
Binary files /tmp/orig1 and /tmp/testr1 differ
{code}
> MPU Key read on a key which is overwritten on a TDE bucket the data does not match
> ----------------------------------------------------------------------------------
>
> Key: HDDS-5980
> URL: https://issues.apache.org/jira/browse/HDDS-5980
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Major
>
> *Scenario:*
> 1. Upload a key through MPU API's
> 2. Read Key, and compare with originally uploaded. It matches
> 3. Upload a key to override again through MPU API's.
> 4. Read Key, and compare with originally uploaded. Expectation is it should match, but it does not match.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org