You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/08/01 13:14:00 UTC
[jira] [Commented] (HADOOP-16477) S3 delegation token tests fail if
fs.s3a.encryption.key set
[ https://issues.apache.org/jira/browse/HADOOP-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898075#comment-16898075 ]
Steve Loughran commented on HADOOP-16477:
-----------------------------------------
unsetting things works for the basic test setup here.
Even there though, the test {{ITestRoleDelegationInFileystem.testDelegatedFileSystem}} fails when writing an object, 403.
Assumption: the role policy created for the DT doesn't include the permission to encrypt data with SSE-KMS. Which would count as a bug in the code, rather than just test setup
{code}
2019-08-01 14:08:37,777 [teardown] INFO contract.AbstractFSContractTestBase (AbstractFSContractTestBase.java:describe(255)) - closing file system
java.nio.file.AccessDeniedException: test/testDTFileSystemClient: put on test/testDTFileSystemClient: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: E86544FF1D029857; S3 Extended Request ID: TDRYwBniy+TkRNAx3Mqn57CY/m5ggxuJA6FgizI6koEwWJkqKo7vHst6tQZAaHX7gXp/jlDREaw=), S3 Extended Request ID: TDRYwBniy+TkRNAx3Mqn57CY/m5ggxuJA6FgizI6koEwWJkqKo7vHst6tQZAaHX7gXp/jlDREaw=:AccessDenied
at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:243)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:111)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:314)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:406)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:310)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:285)
at org.apache.hadoop.fs.s3a.WriteOperationHelper.retry(WriteOperationHelper.java:150)
at org.apache.hadoop.fs.s3a.WriteOperationHelper.putObject(WriteOperationHelper.java:460)
at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.lambda$putObject$0(S3ABlockOutputStream.java:438)
at org.apache.hadoop.util.SemaphoredDelegatingExecutor$CallableWithPermitRelease.call(SemaphoredDelegatingExecutor.java:219)
at org.apache.hadoop.util.SemaphoredDelegatingExecutor$CallableWithPermitRelease.call(SemaphoredDelegatingExecutor.java:219)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: E86544FF1D029857; S3 Extended Request ID: TDRYwBniy+TkRNAx3Mqn57CY/m5ggxuJA6FgizI6koEwWJkqKo7vHst6tQZAaHX7gXp/jlDREaw=), S3 Extended Request ID: TDRYwBniy+TkRNAx3Mqn57CY/m5ggxuJA6FgizI6koEwWJkqKo7vHst6tQZAaHX7gXp/jlDREaw=
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4920)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4866)
at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:389)
at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:5800)
at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1789)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1749)
at org.apache.hadoop.fs.s3a.S3AFileSystem.putObjectDirect(S3AFileSystem.java:1935)
at org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$putObject$5(WriteOperationHelper.java:462)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
... 15 more
{code}
> S3 delegation token tests fail if fs.s3a.encryption.key set
> -----------------------------------------------------------
>
> Key: HADOOP-16477
> URL: https://issues.apache.org/jira/browse/HADOOP-16477
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Priority: Major
>
> if you set an s3a encryption key, the Session and Role DelegationToken tests fail...the test setup needs to unset that key for config and bucket
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org