You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Ahmar Suhail (Jira)" <ji...@apache.org> on 2022/07/27 11:53:00 UTC
[jira] [Commented] (HADOOP-18372) ILoadTestS3ABulkDeleteThrottling failing
[ https://issues.apache.org/jira/browse/HADOOP-18372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571875#comment-17571875 ]
Ahmar Suhail commented on HADOOP-18372:
---------------------------------------
btw once you set the config properly, it fails with
{code:java}
java.util.concurrent.ExecutionException: org.apache.hadoop.fs.s3a.audit.AuditFailureException: 043b7ed5-1a44-41a0-8e52-ae97185c7720-00000003 unaudited operation executing a request outside an audit span {object_bulk_delete_request 'test/testDeleteObjectThrottling/file-0000' size=250, mutating=true} at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at org.apache.hadoop.fs.s3a.scale.ILoadTestS3ABulkDeleteThrottling.deleteFiles(ILoadTestS3ABulkDeleteThrottling.java:271)
at org.apache.hadoop.fs.s3a.scale.ILoadTestS3ABulkDeleteThrottling.test_020_DeleteThrottling(ILoadTestS3ABulkDeleteThrottling.java:196)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.hadoop.fs.s3a.audit.AuditFailureException: 043b7ed5-1a44-41a0-8e52-ae97185c7720-00000003 unaudited operation executing a request outside an audit span {object_bulk_delete_request 'test/testDeleteObjectThrottling/file-0000' size=250, mutating=true}
at org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor$WarningSpan.beforeExecution(LoggingAuditor.java:422)
at org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A$WrappingAuditSpan.beforeExecution(ActiveAuditManagerS3A.java:784)
at org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A.beforeExecution(ActiveAuditManagerS3A.java:473)
at org.apache.hadoop.fs.s3a.audit.impl.ActiveAuditManagerS3A$SdkRequestHandler.beforeExecution(ActiveAuditManagerS3A.java:619)
at com.amazonaws.AmazonWebServiceClient.beforeClientExecution(AmazonWebServiceClient.java:787)
at com.amazonaws.services.s3.AmazonS3Client.deleteObjects(AmazonS3Client.java:2304)
at org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$deleteObjects$17(S3AFileSystem.java:2742)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:468)
at org.apache.hadoop.fs.s3a.S3AFileSystem.deleteObjects(S3AFileSystem.java:2732)
at org.apache.hadoop.fs.s3a.S3AFileSystem.removeKeysS3(S3AFileSystem.java:2973)
at org.apache.hadoop.fs.s3a.S3AFileSystem.removeKeys(S3AFileSystem.java:3041)
at org.apache.hadoop.fs.s3a.scale.ILoadTestS3ABulkDeleteThrottling.lambda$deleteFiles$0(ILoadTestS3ABulkDeleteThrottling.java:256) {code}
> ILoadTestS3ABulkDeleteThrottling failing
> ----------------------------------------
>
> Key: HADOOP-18372
> URL: https://issues.apache.org/jira/browse/HADOOP-18372
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Ahmar Suhail
> Priority: Minor
>
> the test ILoadTestS3ABulkDeleteThrottling; looks like the fs config is being set up too late in the test suite. it should be moved from setup to createConf
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org