You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/01/06 17:14:00 UTC
[jira] [Work logged] (HADOOP-14334) S3 SSEC tests to downgrade when running against a mandatory encryption object store
[ https://issues.apache.org/jira/browse/HADOOP-14334?focusedWorklogId=704659&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-704659 ]
ASF GitHub Bot logged work on HADOOP-14334:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 06/Jan/22 17:13
Start Date: 06/Jan/22 17:13
Worklog Time Spent: 10m
Work Description: monthonk opened a new pull request #3870:
URL: https://github.com/apache/hadoop/pull/3870
If you run the S3A tests against a bucket with mandatory encryption, you need to set the encryption in auth keys. This breaks the SSEC tests because the encryption.key property being set breaks them.
This changes catch `AccessDeniedException` in the setup step of SSE tests, if it occurs then the test will be skipped. This exception is thrown by S3A when using encryption method that is not allowed by the bucket.
Tested in `eu-west-1` with `mvn -Dparallel-tests -DtestsThreadCount=32 clean verify`
```
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestS3AContractUnbuffer>AbstractContractUnbufferTest.testMultipleUnbuffers:100->AbstractContractUnbufferTest.validateFullFileContents:132->AbstractContractUnbufferTest.validateFileContents:139->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89 failed to read expected number of bytes from stream. This may be transient expected:<1024> but was:<605>
[ERROR] ITestS3AInconsistency.testGetFileStatus:123->Assert.fail:89 getFileStatus should fail due to delayed visibility.
[INFO]
[ERROR] Tests run: 1475, Failures: 2, Errors: 0, Skipped: 509
```
Rerun failures individually
```
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.contract.s3a.ITestS3AContractUnbuffer
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.894 s - in org.apache.hadoop.fs.contract.s3a.ITestS3AContractUnbuffer
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0
```
```
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.s3a.ITestS3AInconsistency
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.108 s - in org.apache.hadoop.fs.s3a.ITestS3AInconsistency
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 704659)
Remaining Estimate: 0h
Time Spent: 10m
> S3 SSEC tests to downgrade when running against a mandatory encryption object store
> ------------------------------------------------------------------------------------
>
> Key: HADOOP-14334
> URL: https://issues.apache.org/jira/browse/HADOOP-14334
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 2.9.0
> Reporter: Steve Loughran
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If you run the S3A tests against a bucket with mandatory encryption, you need to set the encryption in auth keys. This breaks the SSEC tests because the encryption.key property being set breaks them.
> That can be fixed (unset it), but they also need to handle failure from the wrong encryption mech being used. Proposed: catch and downgrade to skipped.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org