You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "adoroszlai (via GitHub)" <gi...@apache.org> on 2024/04/12 14:01:11 UTC
[PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
adoroszlai opened a new pull request, #6519:
URL: https://github.com/apache/ozone/pull/6519
## What changes were proposed in this pull request?
Fix test failures related to in S3A contract test. The problem is that some parts of the code use "modification time" instead of content hash as ETag.
```
Tests run: 11, Failures: 0, Errors: 3, Skipped: 1, Time elapsed: 2.998 s <<< FAILURE! - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename
testRenamePopulatesFileAncestors2(org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename) Time elapsed: 1.04 s <<< ERROR!
org.apache.hadoop.fs.s3a.RemoteFileChangedException: copy `s3a://obs-bucket/test/testRenamePopulatesFileAncestors2/src/dir1/dir2/dir3/fileA': ETag change detected on client during copy. Expected 2024-03-29T18:34:05.086Z got f41ee8006d26bd7fc73828b0a0b350a8
testRenameWithNonEmptySubDir(org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename) Time elapsed: 0.281 s <<< ERROR!
org.apache.hadoop.fs.s3a.RemoteFileChangedException: copy `s3a://obs-bucket/test/testRenameWithNonEmptySubDir/src1/source.txt': ETag change detected on client during copy. Expected 2024-03-29T18:34:05.352Z got 046d1988b7a01316bb47905ccf3b584a
testRenamePopulatesFileAncestors(org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename) Time elapsed: 0.219 s <<< ERROR!
org.apache.hadoop.fs.s3a.RemoteFileChangedException: copy `s3a://obs-bucket/test/testRenamePopulatesFileAncestors/source/dir1/dir2/dir3/file4': ETag change detected on client during copy. Expected 2024-03-29T18:34:05.609Z got 67e95334b751fbfc6570e8228183c074
Tests run: 4, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 1.314 s <<< FAILURE! - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractEtag
testLocatedStatusAlsoHasEtag(org.apache.hadoop.fs.contract.s3a.ITestS3AContractEtag) Time elapsed: 0.117 s <<< FAILURE!
org.junit.ComparisonFailure: [etag of listLocatedStatus (S3ALocatedFileStatus{path=s3a://obs-bucket/test/testLocatedStatusAlsoHasEtag/src; isDirectory=false; length=11; replication=1; blocksize=33554432; modification_time=1711737272164; access_time=0; owner=...; group=...; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false}[eTag='2024-03-29T18:34:32.164Z', versionId='']) compared to HEAD value of S3AFileStatus{path=s3a://obs-bucket/test/testLocatedStatusAlsoHasEtag/src; isDirectory=false; length=11; replication=1; blocksize=33554432; modification_time=1711737272000; access_time=0; owner=...; group=...; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE eTag=26e078b87fdaa3206ab8bf63a6096c07 versionId=null] expected:<"2[6e078b87fdaa3206ab8bf63a6096c07]"> but was:<"2[024-03-29T18:34:32.164Z]">
```
https://issues.apache.org/jira/browse/HDDS-10615
## How was this patch tested?
`ITestS3AContractEtag` is enabled as part of the change.
`ITestS3AContractRename` is still failing with another error for FSO buckets only (HDDS-10665), but the ones related to ETag change are fixed. The test is passing with OBS bucket:
```
Tests run: 11, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 5.289 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractRename
```
https://github.com/adoroszlai/ozone/actions/runs/8661440172
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
Posted by "vtutrinov (via GitHub)" <gi...@apache.org>.
vtutrinov commented on code in PR #6519:
URL: https://github.com/apache/ozone/pull/6519#discussion_r1566999408
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/BucketEndpoint.java:
##########
@@ -709,7 +710,10 @@ private void addKey(ListObjectResponse response, OzoneKey next) {
keyMetadata.setKey(EncodingTypeObject.createNullable(next.getName(),
response.getEncodingType()));
keyMetadata.setSize(next.getDataSize());
- keyMetadata.setETag("" + next.getModificationTime());
+ String eTag = next.getMetadata().get(ETAG);
+ if (eTag != null) {
+ keyMetadata.setETag(eTag);
Review Comment:
@ivandika3 sure thing, the RFC (https://datatracker.ietf.org/doc/html/rfc7232#section-2.3) says that the ETag header value is a quoted string
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6519:
URL: https://github.com/apache/ozone/pull/6519#issuecomment-2051827641
@ivandika3 @vtutrinov @whbing plese review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #6519:
URL: https://github.com/apache/ozone/pull/6519#discussion_r1563836962
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/BucketEndpoint.java:
##########
@@ -709,7 +710,10 @@ private void addKey(ListObjectResponse response, OzoneKey next) {
keyMetadata.setKey(EncodingTypeObject.createNullable(next.getName(),
response.getEncodingType()));
keyMetadata.setSize(next.getDataSize());
- keyMetadata.setETag("" + next.getModificationTime());
+ String eTag = next.getMetadata().get(ETAG);
+ if (eTag != null) {
+ keyMetadata.setETag(eTag);
Review Comment:
Should we standardize so that the ETag returned needs to be surrounded by wrap in quotes? Similar to `CopyObjectResponse`?
From https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html
However, it seems that the S3A pass even without the quotes. From what I briefly saw from `AbstractContractEtagTest`, the test does not assert the quotations of the ETag so it will pass regardless of the quotations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #6519:
URL: https://github.com/apache/ozone/pull/6519#discussion_r1567277516
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/BucketEndpoint.java:
##########
@@ -709,7 +710,10 @@ private void addKey(ListObjectResponse response, OzoneKey next) {
keyMetadata.setKey(EncodingTypeObject.createNullable(next.getName(),
response.getEncodingType()));
keyMetadata.setSize(next.getDataSize());
- keyMetadata.setETag("" + next.getModificationTime());
+ String eTag = next.getMetadata().get(ETAG);
+ if (eTag != null) {
+ keyMetadata.setETag(eTag);
Review Comment:
Thanks for the pointers. Updated to wrap in quotes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6519:
URL: https://github.com/apache/ozone/pull/6519#issuecomment-2059004662
Thanks @ivandika3, @vtutrinov for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #6519:
URL: https://github.com/apache/ozone/pull/6519#discussion_r1563836962
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/BucketEndpoint.java:
##########
@@ -709,7 +710,10 @@ private void addKey(ListObjectResponse response, OzoneKey next) {
keyMetadata.setKey(EncodingTypeObject.createNullable(next.getName(),
response.getEncodingType()));
keyMetadata.setSize(next.getDataSize());
- keyMetadata.setETag("" + next.getModificationTime());
+ String eTag = next.getMetadata().get(ETAG);
+ if (eTag != null) {
+ keyMetadata.setETag(eTag);
Review Comment:
Should we standardize so that the ETag returned needs to be surrounded by wrap in quotes? Similar to `CopyObjectResponse`?
From https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html
However, it seems that the S3A pass even without the quotes. From what I briefly saw from `AbstractContractEtagTest`, the test does not assert the quotations of the ETag.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #6519:
URL: https://github.com/apache/ozone/pull/6519#discussion_r1566187178
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/BucketEndpoint.java:
##########
@@ -709,7 +710,10 @@ private void addKey(ListObjectResponse response, OzoneKey next) {
keyMetadata.setKey(EncodingTypeObject.createNullable(next.getName(),
response.getEncodingType()));
keyMetadata.setSize(next.getDataSize());
- keyMetadata.setETag("" + next.getModificationTime());
+ String eTag = next.getMetadata().get(ETAG);
+ if (eTag != null) {
+ keyMetadata.setETag(eTag);
Review Comment:
In the AWS page it is interesting that both `Key` and `ETag` are defined as `string`, but only that latter has quotes as part of the value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #6519:
URL: https://github.com/apache/ozone/pull/6519
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #6519:
URL: https://github.com/apache/ozone/pull/6519#discussion_r1563836962
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/BucketEndpoint.java:
##########
@@ -709,7 +710,10 @@ private void addKey(ListObjectResponse response, OzoneKey next) {
keyMetadata.setKey(EncodingTypeObject.createNullable(next.getName(),
response.getEncodingType()));
keyMetadata.setSize(next.getDataSize());
- keyMetadata.setETag("" + next.getModificationTime());
+ String eTag = next.getMetadata().get(ETAG);
+ if (eTag != null) {
+ keyMetadata.setETag(eTag);
Review Comment:
Shall we standardize so that the ETag returned needs to be surrounded by wrap in quotes? Similar to `CopyObjectResponse`?
From https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html
However, it seems that the S3A pass even without the quotes. From what I briefly saw from `AbstractContractEtagTest`, the test does not assert the quotations of the ETag so it will pass regardless of the quotations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10615. ETag change detected in S3A contract test [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #6519:
URL: https://github.com/apache/ozone/pull/6519#discussion_r1566632329
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/BucketEndpoint.java:
##########
@@ -709,7 +710,10 @@ private void addKey(ListObjectResponse response, OzoneKey next) {
keyMetadata.setKey(EncodingTypeObject.createNullable(next.getName(),
response.getEncodingType()));
keyMetadata.setSize(next.getDataSize());
- keyMetadata.setETag("" + next.getModificationTime());
+ String eTag = next.getMetadata().get(ETAG);
+ if (eTag != null) {
+ keyMetadata.setETag(eTag);
Review Comment:
From what I understand the ETag is specified by RFC to always be surrounded by double quotes.
Ref: https://github.com/aws/aws-sdk-go-v2/issues/1696
@vtutrinov Could you help confirm this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org