You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "ivandika3 (via GitHub)" <gi...@apache.org> on 2024/02/22 06:41:27 UTC
[PR] HDDS-10403. CopyObject should set eTag based on the key content [ozone]
ivandika3 opened a new pull request, #6251:
URL: https://github.com/apache/ozone/pull/6251
## What changes were proposed in this pull request?
Currently copyObject does not seem to support eTag based on key's content. It uses a random UUID in CopyObjectResponse.
We might need to change this to be the eTag of the key instead.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10403
## How was this patch tested?
Unit tests, acceptance test, and manual tests.
Compatibilities
- If the source key does not contain eTag field, the copied destination key will have the eTag field
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10403. CopyObject should set eTag based on the key content [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #6251:
URL: https://github.com/apache/ozone/pull/6251#issuecomment-1959101831
@kerneltime @myskov Could you help to review this when convenient? Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10403. CopyObject should set ETag based on the key content [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #6251:
URL: https://github.com/apache/ozone/pull/6251#issuecomment-1961254673
Thank you for the reviews @myskov @vtutrinov and @adoroszlai for the merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10403. CopyObject should set ETag based on the key content [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6251:
URL: https://github.com/apache/ozone/pull/6251#issuecomment-1961251639
Thanks @ivandika3 for the patch, @myskov, @vtutrinov for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10403. CopyObject should set eTag based on the key content [ozone]
Posted by "vtutrinov (via GitHub)" <gi...@apache.org>.
vtutrinov commented on PR #6251:
URL: https://github.com/apache/ozone/pull/6251#issuecomment-1959209373
@ivandika3 thanks for the patch! @myskov LGTM, +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10403. CopyObject should set eTag based on the key content [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #6251:
URL: https://github.com/apache/ozone/pull/6251#discussion_r1499125746
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpointStreaming.java:
##########
@@ -122,15 +122,19 @@ public static long copyKeyWithStream(
int bufferSize,
ReplicationConfig replicationConfig,
Map<String, String> keyMetadata,
- InputStream body, PerformanceStringBuilder perf, long startNanos)
+ DigestInputStream body, PerformanceStringBuilder perf, long startNanos)
throws IOException {
long writeLen = 0;
+ String eTag;
try (OzoneDataStreamOutput streamOutput = bucket.createStreamKey(keyPath,
length, replicationConfig, keyMetadata)) {
long metadataLatencyNs =
METRICS.updateCopyKeyMetadataStats(startNanos);
- perf.appendMetaLatencyNanos(metadataLatencyNs);
writeLen = writeToStreamOutput(streamOutput, body, bufferSize, length);
+ eTag = DatatypeConverter.printHexBinary(body.getMessageDigest().digest())
Review Comment:
Thanks for the review. Updated. PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10403. CopyObject should set eTag based on the key content [ozone]
Posted by "myskov (via GitHub)" <gi...@apache.org>.
myskov commented on PR #6251:
URL: https://github.com/apache/ozone/pull/6251#issuecomment-1959170417
@vtutrinov could you also take a look, please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10403. CopyObject should set ETag based on the key content [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #6251:
URL: https://github.com/apache/ozone/pull/6251
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10403. CopyObject should set eTag based on the key content [ozone]
Posted by "myskov (via GitHub)" <gi...@apache.org>.
myskov commented on code in PR #6251:
URL: https://github.com/apache/ozone/pull/6251#discussion_r1499089610
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/ObjectEndpointStreaming.java:
##########
@@ -122,15 +122,19 @@ public static long copyKeyWithStream(
int bufferSize,
ReplicationConfig replicationConfig,
Map<String, String> keyMetadata,
- InputStream body, PerformanceStringBuilder perf, long startNanos)
+ DigestInputStream body, PerformanceStringBuilder perf, long startNanos)
throws IOException {
long writeLen = 0;
+ String eTag;
try (OzoneDataStreamOutput streamOutput = bucket.createStreamKey(keyPath,
length, replicationConfig, keyMetadata)) {
long metadataLatencyNs =
METRICS.updateCopyKeyMetadataStats(startNanos);
- perf.appendMetaLatencyNanos(metadataLatencyNs);
writeLen = writeToStreamOutput(streamOutput, body, bufferSize, length);
+ eTag = DatatypeConverter.printHexBinary(body.getMessageDigest().digest())
Review Comment:
As the eTag var is not used anywhere else, let's move eTag declaration here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org