You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2022/11/29 17:23:00 UTC
[jira] [Updated] (HADOOP-18545) hadoop-azure integration tests are failing on the trunk branch against non-HNS stores
[ https://issues.apache.org/jira/browse/HADOOP-18545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-18545:
------------------------------------
Summary: hadoop-azure integration tests are failing on the trunk branch against non-HNS stores (was: hadoop-azure integration tests are failing on the trunk branch)
> hadoop-azure integration tests are failing on the trunk branch against non-HNS stores
> -------------------------------------------------------------------------------------
>
> Key: HADOOP-18545
> URL: https://issues.apache.org/jira/browse/HADOOP-18545
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, test
> Affects Versions: 3.4.0
> Reporter: Carl
> Priority: Major
>
> I am getting errors when running the hadoop-azure integration test from the trunk branch (commit 0ef572abed624011ad9a9e69d725774d6d00b75b).
> This is running on an Azure instance, with HNS disabled and using a Shared Key (NonHNS-SharedKey).
> Here are the logs when running those tests:
>
> {code:java}
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsHttpOperation
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsClient
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsInputStream
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAzureADAuthenticator
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestShellDecryptionKeyProvider
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsOutputStream
> [INFO] Running org.apache.hadoop.fs.azurebfs.TestAccountConfiguration
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsClientThrottlingAnalyzer
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.976 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsHttpOperation
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.174 s - in org.apache.hadoop.fs.azurebfs.services.TestShellDecryptionKeyProvider
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsPerfTracker
> [ERROR] Tests run: 9, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.921 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.TestAccountConfiguration
> [ERROR] testConfigPropNotFound(org.apache.hadoop.fs.azurebfs.TestAccountConfiguration) Time elapsed: 0.102 s <<< FAILURE!
> java.lang.AssertionError: Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
> at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499)
> at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:384)
> at org.apache.hadoop.fs.azurebfs.TestAccountConfiguration.testMissingConfigKey(TestAccountConfiguration.java:399)
> at org.apache.hadoop.fs.azurebfs.TestAccountConfiguration.testConfigPropNotFound(TestAccountConfiguration.java:386)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.396 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsClient
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestTextFileBasedIdentityHandler
> [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.043 s - in org.apache.hadoop.fs.azurebfs.services.TestTextFileBasedIdentityHandler
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestQueryParams
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.2 s - in org.apache.hadoop.fs.azurebfs.services.TestQueryParams
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsRenameRetryRecovery
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.TestExponentialRetryPolicy
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.088 s - in org.apache.hadoop.fs.azurebfs.services.TestAzureADAuthenticator
> [INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsInputStreamStatistics
> [INFO] Running org.apache.hadoop.fs.azurebfs.utils.TestCachedSASToken
> [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.651 s - in org.apache.hadoop.fs.azurebfs.utils.TestCachedSASToken
> [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.133 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsPerfTracker
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.278 s - in org.apache.hadoop.fs.azurebfs.services.TestExponentialRetryPolicy
> [INFO] Running org.apache.hadoop.fs.azurebfs.utils.TestUriUtils
> [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.713 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsOutputStream
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.861 s - in org.apache.hadoop.fs.azurebfs.TestAbfsInputStreamStatistics
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.668 s - in org.apache.hadoop.fs.azurebfs.utils.TestUriUtils
> [INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsOutputStreamStatistics
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.598 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsRenameRetryRecovery
> [INFO] Running org.apache.hadoop.fs.azurebfs.diagnostics.TestConfigurationValidators
> [INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsNetworkStatistics
> [INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.322 s - in org.apache.hadoop.fs.azurebfs.diagnostics.TestConfigurationValidators
> [INFO] Running org.apache.hadoop.fs.azurebfs.TestTracingContext
> [INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsConfigurationFieldsValidation
> [INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsErrorTranslation
> [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.61 s - in org.apache.hadoop.fs.azurebfs.TestAbfsConfigurationFieldsValidation
> [INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsStatistics
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.534 s - in org.apache.hadoop.fs.azurebfs.TestAbfsErrorTranslation
> [INFO] Running org.apache.hadoop.fs.azurebfs.extensions.TestCustomOauthTokenProvider
> [INFO] Running org.apache.hadoop.fs.azurebfs.extensions.TestDTManagerLifecycle
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.441 s - in org.apache.hadoop.fs.azurebfs.TestAbfsOutputStreamStatistics
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.849 s - in org.apache.hadoop.fs.azurebfs.extensions.TestCustomOauthTokenProvider
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.786 s - in org.apache.hadoop.fs.azurebfs.TestAbfsNetworkStatistics
> [WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 6.956 s - in org.apache.hadoop.fs.azurebfs.TestTracingContext
> [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.59 s - in org.apache.hadoop.fs.azurebfs.extensions.TestDTManagerLifecycle
> [INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsCrc64
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.061 s - in org.apache.hadoop.fs.azurebfs.TestAbfsCrc64
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.686 s - in org.apache.hadoop.fs.azurebfs.TestAbfsStatistics
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.877 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsClientThrottlingAnalyzer
> [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.459 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsInputStream
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] TestAccountConfiguration.testConfigPropNotFound:386->testMissingConfigKey:399 Expected a org.apache.hadoop.fs.azurebfs.contracts.exceptions.TokenAccessProviderException to be thrown, but got the result: : "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider"
> [INFO]
> [ERROR] Tests run: 107, Failures: 1, Errors: 0, Skipped: 2
> [INFO]
> [ERROR] There are test failures.
> Please refer to /hadoop/hadoop-tools/hadoop-azure/target/surefire-reports for the individual test results.
> Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
> [INFO]
> [INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ hadoop-azure ---
> [INFO]
> [INFO] --- maven-jar-plugin:2.5:test-jar (default) @ hadoop-azure ---
> [INFO] Building jar: /hadoop/hadoop-tools/hadoop-azure/target/hadoop-azure-3.4.0-SNAPSHOT-tests.jar
> [INFO]
> [INFO] --- maven-site-plugin:3.11.0:attach-descriptor (attach-descriptor) @ hadoop-azure ---
> [INFO] Skipping because packaging 'jar' is not pom.
> [INFO]
> [INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test (integration-test-abfs-parallel-classesAndMethods) @ hadoop-azure ---
> Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.jar
> Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.jar (93 kB at 2.4 MB/s)
> Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.pom
> Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit47/3.0.0-M1/surefire-junit47-3.0.0-M1.pom (7.3 kB at 292 kB/s)
> Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.pom
> Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.pom (2.4 kB at 70 kB/s)
> Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.pom
> Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.pom (2.6 kB at 146 kB/s)
> Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.jar
> Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.jar
> Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-junit48/3.0.0-M1/common-junit48-3.0.0-M1.jar (22 kB at 1.3 MB/s)
> Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-grouper/3.0.0-M1/surefire-grouper-3.0.0-M1.jar (40 kB at 1.8 MB/s)
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsRestOperationException
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.294 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsRestOperationException
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRenameUnicode
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.288 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRenameUnicode
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsOutputStream
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.765 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsOutputStream
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCLI
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.222 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCLI
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.588 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsNetworkStatistics
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsPositionedRead
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.635 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsPositionedRead
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemBackCompat
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.634 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemBackCompat
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsUnbuffer
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.496 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsUnbuffer
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCopy
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.503 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCopy
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAppend
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.634 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAppend
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAuthorization
> [WARNING] Tests run: 29, Failures: 0, Errors: 0, Skipped: 29, Time elapsed: 2.638 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAuthorization
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestOauthOverAbfsScheme
> [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.277 s - in org.apache.hadoop.fs.azurebfs.ITestOauthOverAbfsScheme
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.317 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsDurationTrackers
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.97 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsDurationTrackers
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelete
> [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.973 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelete
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsClient
> [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 19.652 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsClient
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFilesystemAcl
> [WARNING] Tests run: 66, Failures: 0, Errors: 0, Skipped: 56, Time elapsed: 11.749 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFilesystemAcl
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemProperties
> [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.013 s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemProperties
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemOauth
> [WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.099 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemOauth
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMkDir
> [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.549 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMkDir
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsOutputStreamStatistics
> [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.074 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsOutputStreamStatistics
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsIdentityTransformer
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.745 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsIdentityTransformer
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFlush
> [INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.404 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFlush
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2E
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.549 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2E
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemStoreListStatusWithRange
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.767 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemStoreListStatusWithRange
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess
> [ERROR] Tests run: 12, Failures: 1, Errors: 0, Skipped: 10, Time elapsed: 1.521 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess
> [ERROR] testCheckAccessForAccountWithoutNS(org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess) Time elapsed: 1.15 s <<< FAILURE!
> java.lang.AssertionError: Expecting org.apache.hadoop.security.AccessControlException with text "This request is not authorized to perform this operation using this permission.", 403 but got : "void"
> at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499)
> at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:529)
> at org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS(ITestAzureBlobFileSystemCheckAccess.java:181)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
> at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.lang.Thread.run(Thread.java:829)
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics
> [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.827 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsInputStreamStatistics
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsStatistics
> [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.649 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsStatistics
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCreate
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.393 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCreate
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMainOperation
> [INFO] Tests run: 50, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.373 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMainOperation
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFinalize
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.362 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFinalize
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRandomRead
> [WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 3.196 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRandomRead
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRename
> [INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.622 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRename
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsHugeFiles
> [INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.34 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsHugeFiles
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestWasbAbfsCompatibility
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.802 s - in org.apache.hadoop.fs.azurebfs.ITestWasbAbfsCompatibility
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestClientUrlScheme
> [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.604 s - in org.apache.hadoop.fs.azurebfs.ITestClientUrlScheme
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemRegistration
> [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.775 s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemRegistration
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestSharedKeyAuth
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.186 s - in org.apache.hadoop.fs.azurebfs.ITestSharedKeyAuth
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemInitialization
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.474 s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemInitialization
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestCustomerProvidedKey
> [WARNING] Tests run: 28, Failures: 0, Errors: 0, Skipped: 28, Time elapsed: 0.882 s - in org.apache.hadoop.fs.azurebfs.ITestCustomerProvidedKey
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAttributes
> [WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 0.402 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAttributes
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelegationSAS
> [WARNING] Tests run: 14, Failures: 0, Errors: 0, Skipped: 14, Time elapsed: 0.41 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelegationSAS
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsMsiTokenProvider
> [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.298 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsMsiTokenProvider
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFileStatus
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.921 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFileStatus
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestGetNameSpaceEnabled
> [WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 6.37 s - in org.apache.hadoop.fs.azurebfs.ITestGetNameSpaceEnabled
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsListStatusRemoteIterator
> [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.053 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsListStatusRemoteIterator
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemPermission
> [WARNING] Tests run: 128, Failures: 0, Errors: 0, Skipped: 128, Time elapsed: 9.17 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemPermission
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamSmallFileReads
> [INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.622 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamSmallFileReads
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStream
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.918 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStream
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamReadFooter
> [INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 58.717 s - in org.apache.hadoop.fs.azurebfs.services.ITestAbfsInputStreamReadFooter
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemLease
> [INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 75.544 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemLease
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] ITestAzureBlobFileSystemCheckAccess.testCheckAccessForAccountWithoutNS:181 Expecting org.apache.hadoop.security.AccessControlException with text "This request is not authorized to perform this operation using this permission.", 403 but got : "void"
> [INFO]
> [ERROR] Tests run: 567, Failures: 1, Errors: 0, Skipped: 277
> [INFO]
> [INFO]
> [INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test (integration-test-abfs-parallel-classes) @ hadoop-azure ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsLoadManifestsStage
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestSmallWriteOptimization
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsReadWriteAndSeek
> [INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestReadBufferManager
> [INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCleanupStage
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2EScale
> [INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsRenameStageFailure
> [INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsJobThroughManifestCommitter
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.918 s - in org.apache.hadoop.fs.azurebfs.services.ITestReadBufferManager
> [INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCommitTaskStage
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.879 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsRenameStageFailure
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.657 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCleanupStage
> [INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTaskManifestFileIO
> [INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCreateOutputDirectoriesStage
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.991 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsLoadManifestsStage
> [INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.569 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2EScale
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.159 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCommitTaskStage
> [INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort
> [INFO] Running org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestStoreOperations
> [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.197 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTaskManifestFileIO
> [ERROR] Tests run: 19, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 32.672 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsJobThroughManifestCommitter
> [ERROR] test_0420_validateJob(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsJobThroughManifestCommitter) Time elapsed: 0.74 s <<< ERROR!
> org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0002/test/TestJobThroughManifestCommitter/out%20put/dir-03-00/dir-02-00/dir-01-00/part-00-0033': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0002/test/TestJobThroughManifestCommitter/out put/_temporary/202211291549220355_0002/01/tasks/attempt_202211291549220355_0002_m_000000_1/dir-03-00/dir-02-00/dir-01-00/part-00-0033 with etag 0x8DAD22153D4F458 and length 2 but found a file with etag 0x8DAD22159994F7E and length 2
> at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
> at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.371 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsReadWriteAndSeek
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsStreamStatistics
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRename
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAzureBlobFileSystemBasics
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.86 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsCreateOutputDirectoriesStage
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRootDirectory
> [WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 8.559 s - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestStoreOperations
> [WARNING] Tests run: 48, Failures: 0, Errors: 0, Skipped: 24, Time elapsed: 41.978 s - in org.apache.hadoop.fs.azurebfs.ITestSmallWriteOptimization
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractOpen
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.96 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsStreamStatistics
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractMkdir
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSetTimes
> [WARNING] Tests run: 9, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 9.624 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRootDirectory
> [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.207 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRename
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractEtag
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsContractUnbuffer
> [ERROR] Tests run: 17, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 25.661 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol
> [ERROR] testCommitterWithDuplicatedCommit(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol) Time elapsed: 10.143 s <<< ERROR!
> org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitterWithDuplicatedCommit/part-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitterWithDuplicatedCommit/_temporary/job_202211291549500881_0004/01/tasks/attempt_202211291549500881_0004_m_000000_0/part-m-00000 with etag 0x8DAD2215D8BA025 and length 40 but found a file with etag 0x8DAD2215F25DA02 and length 40
> at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
> at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> [ERROR] testMapFileOutputCommitter(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol) Time elapsed: 1.857 s <<< ERROR!
> org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testMapFileOutputCommitter/part-m-00000/data': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testMapFileOutputCommitter/_temporary/job_202211291550000055_0004/01/tasks/attempt_202211291550000055_0004_m_000000_0/part-m-00000/data with etag 0x8DAD2215FF58EA2 and length 296 but found a file with etag 0x8DAD221603B8E0D and length 296
> at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
> at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> [ERROR] testCommitLifecycle(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol) Time elapsed: 1.462 s <<< ERROR!
> org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitLifecycle/part-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testCommitLifecycle/_temporary/job_202211291550020019_0004/01/tasks/attempt_202211291550020019_0004_m_000000_0/part-m-00000 with etag 0x8DAD22160F41493 and length 40 but found a file with etag 0x8DAD22161466E35 and length 40
> at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
> at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> [ERROR] testConcurrentCommitTaskWithSubDir(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol) Time elapsed: 1.262 s <<< ERROR!
> org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testConcurrentCommitTaskWithSubDir/SUB_DIR/part-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testConcurrentCommitTaskWithSubDir/_temporary/job_202211291550030237_0004/00/tasks/attempt_202211291550030237_0004_m_000000_1/SUB_DIR/part-m-00000 with etag 0x8DAD22161DB6C66 and length 40 but found a file with etag 0x8DAD22162100941 and length 40
> at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
> at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> [ERROR] testParallelJobsToAdjacentPaths(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol) Time elapsed: 1.401 s <<< ERROR!
> org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testParallelJobsToAdjacentPaths/part-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testParallelJobsToAdjacentPaths/_temporary/job_202211291550080667_0004/01/tasks/attempt_202211291550080667_0004_m_000000_0/part-m-00000 with etag 0x8DAD221646E60AB and length 40 but found a file with etag 0x8DAD22164B4AE25 and length 40
> at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
> at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> [ERROR] testOutputFormatIntegration(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol) Time elapsed: 0.925 s <<< ERROR!
> org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testOutputFormatIntegration/part-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testOutputFormatIntegration/_temporary/job_202211291550110055_0004/01/tasks/attempt_202211291550110055_0004_m_000000_0/part-m-00000 with etag 0x8DAD221662398A8 and length 4 but found a file with etag 0x8DAD221664D1399 and length 4
> at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
> at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> [ERROR] testTwoTaskAttemptsCommit(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsManifestCommitProtocol) Time elapsed: 1.291 s <<< ERROR!
> org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.OutputValidationException: `abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testTwoTaskAttemptsCommit/attempt2-m-00000': Expected the file renamed from abfs://****@****.dfs.core.windows.net/fork-0004/test/ITestAbfsManifestCommitProtocol-testTwoTaskAttemptsCommit/_temporary/job_202211291550130427_0004/01/tasks/attempt_202211291550130427_0004_m_000000_1/attempt2-m-00000 with etag 0x8DAD221676CB12D and length 40 but found a file with etag 0x8DAD22167ACBE08 and length 40
> at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.ValidateRenamedFilesStage.validateOneFile(ValidateRenamedFilesStage.java:166)
> at org.apache.hadoop.util.functional.TaskPool$Builder.lambda$runParallel$0(TaskPool.java:410)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> [WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 7.58 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSetTimes
> [INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.177 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractMkdir
> [INFO] Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.624 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractOpen
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDistCp
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractAppend
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractConcat
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDelete
> [INFO] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.35 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAzureBlobFileSystemBasics
> [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.322 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsContractUnbuffer
> [WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 11.125 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractEtag
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractCreate
> [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 6.888 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractConcat
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractGetFileStatus
> [INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSeek
> [WARNING] Tests run: 8, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 9.429 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractAppend
> [INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemListStatus
> [INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.189 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDelete
> [INFO] Running org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens
> [INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.352 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractCreate
> [INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.206 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSeek
> [INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.014 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractGetFileStatus
> [INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.823 s - in org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens
> [INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.969 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemListStatus
> [INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.734 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDistCp
> [ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 100.527 s <<< FAILURE! - in org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort
> [ERROR] test_110_teragen(org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort) Time elapsed: 25.881 s <<< FAILURE!
> java.lang.AssertionError: teragen(1000, abfs://****@****.dfs.core.windows.net/ITestAbfsTerasort/sortin) failed expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:647)
> at org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort.executeStage(ITestAbfsTerasort.java:211)
> at org.apache.hadoop.fs.azurebfs.commit.ITestAbfsTerasort.test_110_teragen(ITestAbfsTerasort.java:244)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
> at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.lang.Thread.run(Thread.java:829)
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] ITestAbfsTerasort.test_110_teragen:244->executeStage:211->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89 teragen(1000, abfs://****@****.dfs.core.windows.net/ITestAbfsTerasort/sortin) failed expected:<0> but was:<1>
> [ERROR] Errors:
> [ERROR] ITestAbfsJobThroughManifestCommitter.test_0420_validateJob ? OutputValidation ...
> [ERROR] ITestAbfsManifestCommitProtocol.testCommitLifecycle ? OutputValidation `abfs:/...
> [ERROR] ITestAbfsManifestCommitProtocol.testCommitterWithDuplicatedCommit ? OutputValidation
> [ERROR] ITestAbfsManifestCommitProtocol.testConcurrentCommitTaskWithSubDir ? OutputValidation
> [ERROR] ITestAbfsManifestCommitProtocol.testMapFileOutputCommitter ? OutputValidation ...
> [ERROR] ITestAbfsManifestCommitProtocol.testOutputFormatIntegration ? OutputValidation
> [ERROR] ITestAbfsManifestCommitProtocol.testParallelJobsToAdjacentPaths ? OutputValidation
> [ERROR] ITestAbfsManifestCommitProtocol.testTwoTaskAttemptsCommit ? OutputValidation `...
> [INFO]
> [ERROR] Tests run: 335, Failures: 1, Errors: 8, Skipped: 46 {code}
>
> And here are the commands I run on a new instance to get there:
>
> {code:java}
> cd /hadoop/hadoop-maven-plugins
> mvn install
> cd /hadoop/hadoop-tools/hadoop-azure
> cp ./src/test/resources/azure-auth-keys.xml.template ./src/test/resources/azure-auth-keys.xml
> az login --identity
> ACCOUNT_NAME=****
> CONTAINER_NAME=****
> RESOURCE_GROUP=****
> ACCOUNT_ACCESS_KEY=$(az storage account keys list \
> --resource-group "$RESOURCE_GROUP" \
> --account-name "$ACCOUNT_NAME" \
> --output tsv \
> --query '[].{value:value} | [0]')
> cat <<EOT > src/test/resources/azure-auth-keys.xml
> <?xml version="1.0" encoding="UTF-8"?>
> <configuration>
> <!-- Account names without domain -->
> <property>
> <name>fs.azure.hnsTestAccountName</name>
> <value>$ACCOUNT_NAME</value>
> </property>
> <property>
> <name>fs.azure.nonHnsTestAccountName</name>
> <value>$ACCOUNT_NAME</value>
> </property>
> </configuration>
> EOT
> cp ./src/test/resources/accountSettings/accountName_settings.xml.template ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml
> sed -i -e "s/ACCOUNTNAME/$ACCOUNT_NAME/g" ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml
> sed -i -e "s/CONTAINER_NAME/$CONTAINER_NAME/g" ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml
> sed -i -e "s/IS_NAMESPACE_ENABLED/false/g" ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml
> sed -i -e "s#ACCOUNT_KEY#$ACCOUNT_ACCESS_KEY#g" ./src/test/resources/accountSettings/${ACCOUNT_NAME}_settings.xml
> dev-support/testrun-scripts/runtests.sh{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org