You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Da Zhou (JIRA)" <ji...@apache.org> on 2018/08/29 02:03:00 UTC
[jira] [Commented] (HADOOP-15664) ABFS: Reduce test run time via
parallelization and grouping
[ https://issues.apache.org/jira/browse/HADOOP-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16595839#comment-16595839 ]
Da Zhou commented on HADOOP-15664:
----------------------------------
Attaching patch: HADOOP-15664-HADOOP-15407-001.patch.
With this patch, the WASB test run time can be *reduced from 40 mins to 20 mins*,
and ABFS test run time can be *reduced from 10 mins to 5 mins*. It used to took me around *1 hour* to run all the tests with the old mvn profile. In future if the code changes are only related to ABFS/WASB, we can only choose one profile for mvn test.
Below are the changes:
- Added two more mvn profile to build WASB and ABFS separately, the old build cmd is still supported:
To run both WASB and ABFS tests:
{code:java}
mvn -T 1C -Pparallel-tests -DtestsThreadCount=8 clean verify
{code}
To run only run WASB tests
{code:java}
mvn -T 1C -Pparallel-tests-wasb -DtestsThreadCount=8 clean verify
{code}
To run only run ABFS tests.
{code:java}
mvn -T 1C -Pparallel-tests-abfs -DtestsThreadCount=8 clean verify
{code}
- Refactored tests code to improve parallelization of the tests.
- Enabled method level parallel executions for some ITest to further reduce the test run time.
Test result:
*WASB*
*mvn -T 1C -Pparallel-tests-wasb -DtestsThreadCount=8 clean verify*
{noformat}
[INFO] Scanning for projects...
[INFO]
[INFO] Using the MultiThreadedBuilder implementation with a thread count of 2
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Hadoop Azure support 3.2.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hadoop-azure ---
[INFO] Deleting /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target
[INFO] Deleting /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure (includes = [dependency-reduced-pom.xml], excludes = [])
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-azure ---
[INFO] Executing tasks
main:
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hadoop-azure ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-azure ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-azure ---
[INFO] Compiling 138 source files to /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/classes
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: Some input files use or override a deprecated API.
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: Recompile with -Xlint:deprecation for details.
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/security/TokenUtils.java: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/security/TokenUtils.java uses unchecked or unsafe operations.
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/security/TokenUtils.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:list (deplist) @ hadoop-azure ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hadoop-azure ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 8 resources
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hadoop-azure ---
[INFO] Compiling 133 source files to /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-classes
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestShellDecryptionKeyProvider.java: Some input files use or override a deprecated API.
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestShellDecryptionKeyProvider.java: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (create-parallel-tests-dirs) @ hadoop-azure ---
[INFO] Executing tasks
main:
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/1
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/2
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/3
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/4
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/5
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/6
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/7
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/8
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/1
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/2
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/3
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/4
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/5
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/6
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/7
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/8
[INFO] Executed tasks
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hadoop-azure ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azure.TestClientThrottlingAnalyzer
[INFO] Running org.apache.hadoop.fs.azure.TestBlobMetadata
[INFO] Running org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations
[INFO] Running org.apache.hadoop.fs.azure.TestShellDecryptionKeyProvider
[INFO] Running org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked
[INFO] Running org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked
[INFO] Running org.apache.hadoop.fs.azure.TestNativeAzureFileSystemUploadLogic
[INFO] Running org.apache.hadoop.fs.azure.TestWasbFsck
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 1.004 s - in org.apache.hadoop.fs.azure.TestNativeAzureFileSystemUploadLogic
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 1.481 s - in org.apache.hadoop.fs.azure.TestShellDecryptionKeyProvider
[INFO] Running org.apache.hadoop.fs.azure.TestNativeAzureFileSystemConcurrency
[INFO] Running org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 7.112 s - in org.apache.hadoop.fs.azure.TestWasbFsck
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.235 s - in org.apache.hadoop.fs.azure.TestBlobMetadata
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.5 s - in org.apache.hadoop.fs.azure.TestOutOfBandAzureBlobOperations
[INFO] Running org.apache.hadoop.fs.azure.metrics.TestNativeAzureFileSystemMetricsSystem
[INFO] Running org.apache.hadoop.fs.azure.metrics.TestBandwidthGaugeUpdater
[INFO] Running org.apache.hadoop.fs.azure.TestNativeAzureFileSystemAuthorization
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.62 s - in org.apache.hadoop.fs.azure.TestNativeAzureFileSystemConcurrency
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.604 s - in org.apache.hadoop.fs.azure.metrics.TestBandwidthGaugeUpdater
[INFO] Tests run: 50, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.852 s - in org.apache.hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.971 s - in org.apache.hadoop.fs.azure.metrics.TestNativeAzureFileSystemMetricsSystem
[INFO] Running org.apache.hadoop.fs.azure.TestBlobOperationDescriptor
[INFO] Running org.apache.hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck
[WARNING] Tests run: 43, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 14.512 s - in org.apache.hadoop.fs.azure.TestNativeAzureFileSystemContractMocked
[INFO] Running org.apache.hadoop.fs.azure.TestNativeAzureFileSystemBlockCompaction
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.003 s - in org.apache.hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.596 s - in org.apache.hadoop.fs.azure.TestBlobOperationDescriptor
[INFO] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.649 s - in org.apache.hadoop.fs.azure.TestNativeAzureFileSystemMocked
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.521 s - in org.apache.hadoop.fs.azure.TestNativeAzureFileSystemBlockCompaction
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.23 s - in org.apache.hadoop.fs.azure.TestClientThrottlingAnalyzer
[INFO] Tests run: 59, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 221.037 s - in org.apache.hadoop.fs.azure.TestNativeAzureFileSystemAuthorization
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 240, Failures: 0, Errors: 0, Skipped: 11
[INFO]
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (serialized-test-wasb) @ hadoop-azure ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azure.metrics.TestRollingWindowAverage
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.273 s - in org.apache.hadoop.fs.azure.metrics.TestRollingWindowAverage
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ hadoop-azure ---
[INFO] Building jar: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/hadoop-azure-3.2.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-jar-plugin:2.5:test-jar (default) @ hadoop-azure ---
[INFO] Building jar: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/hadoop-azure-3.2.0-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-site-plugin:3.6:attach-descriptor (attach-descriptor) @ hadoop-azure ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-failsafe-plugin:2.21.0:integration-test (default-integration-test-wasb) @ hadoop-azure ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azure.ITestFileSystemOperationExceptionMessage
[INFO] Running org.apache.hadoop.fs.azure.ITestWasbUriAndConfiguration
[INFO] Running org.apache.hadoop.fs.azure.integration.ITestAzureHugeFiles
[INFO] Running org.apache.hadoop.fs.azure.ITestOutOfBandAzureBlobOperationsLive
[INFO] Running org.apache.hadoop.fs.azure.ITestPageBlobInputStream
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemLive
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFSAuthWithBlobSpecificKeys
[INFO] Running org.apache.hadoop.fs.azure.ITestBlobDataValidation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.535 s - in org.apache.hadoop.fs.azure.ITestFileSystemOperationExceptionMessage
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemAppend
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.286 s - in org.apache.hadoop.fs.azure.ITestBlobDataValidation
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFSPageBlobLive
[WARNING] Tests run: 19, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 26.332 s - in org.apache.hadoop.fs.azure.ITestWasbUriAndConfiguration
[INFO] Running org.apache.hadoop.fs.azure.ITestListPerformance
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.339 s - in org.apache.hadoop.fs.azure.ITestOutOfBandAzureBlobOperationsLive
[INFO] Running org.apache.hadoop.fs.azure.ITestOutputStreamSemantics
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.482 s - in org.apache.hadoop.fs.azure.integration.ITestAzureHugeFiles
[INFO] Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractOpen
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.596 s - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractOpen
[INFO] Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractSeek
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 68.438 s - in org.apache.hadoop.fs.azure.ITestPageBlobInputStream
[INFO] Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractRename
[INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.377 s - in org.apache.hadoop.fs.azure.ITestOutputStreamSemantics
[INFO] Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractDistCp
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 93.184 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemAppend
[WARNING] Tests run: 6, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 30.206 s - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractDistCp
[INFO] Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractDelete
[INFO] Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractGetFileStatus
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 72.773 s - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractRename
[INFO] Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractAppend
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.073 s - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractDelete
[INFO] Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractCreate
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 97.868 s - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractSeek
[INFO] Running org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractMkdir
[WARNING] Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 38.193 s - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractAppend
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFSAuthorizationCaching
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.924 s - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractCreate
[INFO] Running org.apache.hadoop.fs.azure.ITestBlockBlobInputStream
[INFO] Tests run: 59, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 222.255 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFSAuthWithBlobSpecificKeys
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemClientLogging
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.637 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemClientLogging
[INFO] Running org.apache.hadoop.fs.azure.ITestFileSystemOperationsWithThreads
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 80.441 s - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractMkdir
[INFO] Running org.apache.hadoop.fs.azure.ITestWasbRemoteCallHelper
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 137.117 s - in org.apache.hadoop.fs.azure.contract.ITestAzureNativeContractGetFileStatus
[INFO] Running org.apache.hadoop.fs.azure.ITestAzureConcurrentOutOfBandIo
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 5.769 s - in org.apache.hadoop.fs.azure.ITestWasbRemoteCallHelper
[INFO] Running org.apache.hadoop.fs.azure.ITestReadAndSeekPageBlobAfterWrite
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.47 s - in org.apache.hadoop.fs.azure.ITestAzureConcurrentOutOfBandIo
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractLive
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 126.008 s - in org.apache.hadoop.fs.azure.ITestReadAndSeekPageBlobAfterWrite
[INFO] Running org.apache.hadoop.fs.azure.ITestAzureFileSystemErrorConditions
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.247 s - in org.apache.hadoop.fs.azure.ITestAzureFileSystemErrorConditions
[INFO] Running org.apache.hadoop.fs.azure.metrics.ITestAzureFileSystemInstrumentation
[INFO] Tests run: 60, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 223.965 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFSAuthorizationCaching
[INFO] Running org.apache.hadoop.fs.azure.ITestFileSystemOperationsExceptionHandlingMultiThreaded
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.206 s - in org.apache.hadoop.fs.azure.metrics.ITestAzureFileSystemInstrumentation
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractEmulator
[WARNING] Tests run: 43, Failures: 0, Errors: 0, Skipped: 43, Time elapsed: 0.897 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractEmulator
[INFO] Running org.apache.hadoop.fs.azure.ITestFileSystemOperationExceptionHandling
[WARNING] Tests run: 43, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 204.169 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractLive
[INFO] Running org.apache.hadoop.fs.azure.ITestBlobTypeSpeedDifference
[INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 81.178 s - in org.apache.hadoop.fs.azure.ITestFileSystemOperationsExceptionHandlingMultiThreaded
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.179 s - in org.apache.hadoop.fs.azure.ITestBlobTypeSpeedDifference
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemAtomicRenameDirList
[INFO] Running org.apache.hadoop.fs.azure.ITestContainerChecks
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.657 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemAtomicRenameDirList
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractPageBlobLive
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.924 s - in org.apache.hadoop.fs.azure.ITestContainerChecks
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemConcurrencyLive
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.056 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemConcurrencyLive
[INFO] Running org.apache.hadoop.fs.azure.ITestAzureConcurrentOutOfBandIoWithSecureMode
[INFO] Tests run: 51, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 539.459 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemLive
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.742 s - in org.apache.hadoop.fs.azure.ITestAzureConcurrentOutOfBandIoWithSecureMode
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 96.73 s - in org.apache.hadoop.fs.azure.ITestFileSystemOperationExceptionHandling
[INFO] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 541.367 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFSPageBlobLive
[INFO] Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 421.291 s - in org.apache.hadoop.fs.azure.ITestBlockBlobInputStream
[WARNING] Tests run: 43, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 214.146 s - in org.apache.hadoop.fs.azure.ITestNativeAzureFileSystemContractPageBlobLive
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 660.565 s - in org.apache.hadoop.fs.azure.ITestFileSystemOperationsWithThreads
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 890.351 s - in org.apache.hadoop.fs.azure.ITestListPerformance
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 620, Failures: 0, Errors: 0, Skipped: 71
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:2.21.0:integration-test (sequential-integration-tests-wasb) @ hadoop-azure ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azure.ITestNativeFileSystemStatistics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.569 s - in org.apache.hadoop.fs.azure.ITestNativeFileSystemStatistics
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- animal-sniffer-maven-plugin:1.16:check (signature-check) @ hadoop-azure ---
[INFO] Checking unresolved references to org.codehaus.mojo.signature:java18:1.0
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (depcheck) @ hadoop-azure ---
[INFO]
[INFO] --- maven-failsafe-plugin:2.21.0:verify (default-integration-test-wasb) @ hadoop-azure ---
[INFO]
[INFO] --- maven-failsafe-plugin:2.21.0:verify (sequential-integration-tests-wasb) @ hadoop-azure ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 19:42 min (Wall Clock)
[INFO] Finished at: 2018-08-28T22:45:31+00:00
[INFO] Final Memory: 62M/655M
{noformat}
*ABFS tests (using namespace-enabled account, Oauth authenticated):
mvn -T 1C -Pparallel-tests-wasb -DtestsThreadCount=8 clean verify*
{noformat}
[INFO] Scanning for projects...
[INFO]
[INFO] Using the MultiThreadedBuilder implementation with a thread count of 2
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Hadoop Azure support 3.2.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hadoop-azure ---
[INFO] Deleting /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target
[INFO] Deleting /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure (includes = [dependency-reduced-pom.xml], excludes = [])
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-azure ---
[INFO] Executing tasks
main:
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hadoop-azure ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-azure ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-azure ---
[INFO] Compiling 138 source files to /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/classes
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: Some input files use or override a deprecated API.
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/StorageInterfaceImpl.java: Recompile with -Xlint:deprecation for details.
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/security/TokenUtils.java: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/security/TokenUtils.java uses unchecked or unsafe operations.
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/security/TokenUtils.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-dependency-plugin:3.0.2:list (deplist) @ hadoop-azure ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hadoop-azure ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 7 resources
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hadoop-azure ---
[INFO] Compiling 133 source files to /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-classes
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestShellDecryptionKeyProvider.java: Some input files use or override a deprecated API.
[WARNING] /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestShellDecryptionKeyProvider.java: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (create-parallel-tests-dirs) @ hadoop-azure ---
[INFO] Executing tasks
main:
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/1
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/2
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/3
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/4
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/5
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/6
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/7
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test-dir/8
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/1
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/2
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/3
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/4
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/5
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/6
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/7
[mkdir] Created dir: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/test/8
[INFO] Executed tasks
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hadoop-azure ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azurebfs.utils.TestUriUtils
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestAbfsClient
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestShellDecryptionKeyProvider
[INFO] Running org.apache.hadoop.fs.azurebfs.diagnostics.TestConfigurationValidators
[INFO] Running org.apache.hadoop.fs.azurebfs.services.TestQueryParams
[INFO] Running org.apache.hadoop.fs.azurebfs.TestAbfsConfigurationFieldsValidation
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.531 s - in org.apache.hadoop.fs.azurebfs.services.TestQueryParams
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.651 s - in org.apache.hadoop.fs.azurebfs.utils.TestUriUtils
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.806 s - in org.apache.hadoop.fs.azurebfs.diagnostics.TestConfigurationValidators
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.084 s - in org.apache.hadoop.fs.azurebfs.services.TestShellDecryptionKeyProvider
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.477 s - in org.apache.hadoop.fs.azurebfs.TestAbfsConfigurationFieldsValidation
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.987 s - in org.apache.hadoop.fs.azurebfs.services.TestAbfsClient
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 24, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ hadoop-azure ---
[INFO] Building jar: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/hadoop-azure-3.2.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-jar-plugin:2.5:test-jar (default) @ hadoop-azure ---
[INFO] Building jar: /home/zhoda/dev/Projects/apache-hadoop/hadoop/hadoop-tools/hadoop-azure/target/hadoop-azure-3.2.0-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-site-plugin:3.6:attach-descriptor (attach-descriptor) @ hadoop-azure ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- maven-failsafe-plugin:2.21.0:integration-test (integration-test-abfs-parallel-classesAndMethods) @ hadoop-azure ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRandomRead
[WARNING] Tests run: 9, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 0.131 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRandomRead
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.433 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemInitAndCreate
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFinalize
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.118 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFinalize
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRenameUnicode
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.914 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRenameUnicode
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCopy
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.706 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCopy
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemRegistration
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.345 s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemRegistration
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemBackCompat
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.004 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemBackCompat
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFilesystemAcl
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFileStatus
[WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 17.427 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFileStatus
[WARNING] Tests run: 50, Failures: 0, Errors: 0, Skipped: 4, Time elapsed: 17.076 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFilesystemAcl
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestWasbAbfsCompatibility
[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 5, Time elapsed: 0.099 s - in org.apache.hadoop.fs.azurebfs.ITestWasbAbfsCompatibility
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemOauth
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.246 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemOauth
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemInitialization
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.506 s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemInitialization
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsClient
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.765 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsClient
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMkDir
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.991 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemMkDir
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCreate
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.039 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemCreate
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAppend
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.502 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemAppend
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestFileSystemProperties
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.521 s - in org.apache.hadoop.fs.azurebfs.ITestFileSystemProperties
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2E
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32.195 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2E
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemPermission
[INFO] Tests run: 128, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.01 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemPermission
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRename
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.139 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemRename
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelete
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.747 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemDelete
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFlush
[WARNING] Tests run: 12, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 81.725 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFlush
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 260, Failures: 0, Errors: 0, Skipped: 23
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:2.21.0:integration-test (integration-test-abfs-parallel-classes) @ hadoop-azure ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDelete
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractGetFileStatus
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAbfsReadWriteAndSeek
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAzureBlobFileSystemBasics
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractOpen
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRename
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractCreate
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSecureDistCp
[WARNING] Tests run: 6, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 12.35 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSecureDistCp
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSeek
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.479 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractOpen
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractMkdir
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.484 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDelete
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRootDirectory
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.823 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRename
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractConcat
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.66 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractCreate
[WARNING] Tests run: 9, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 9.286 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRootDirectory
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDistCp
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractAppend
[WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 4, Time elapsed: 7.879 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractConcat
[INFO] Running org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSetTimes
[WARNING] Tests run: 6, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 6.999 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractDistCp
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.988 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractMkdir
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemListStatus
[INFO] Running org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2EScale
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 4.728 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSetTimes
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.844 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractGetFileStatus
[WARNING] Tests run: 9, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 13.497 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractAppend
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32.007 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractSeek
[INFO] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 76.843 s - in org.apache.hadoop.fs.azurebfs.contract.ITestAzureBlobFileSystemBasics
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 146.49 s - in org.apache.hadoop.fs.azurebfs.ITestAbfsReadWriteAndSeek
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 114.558 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemListStatus
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 138.724 s - in org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemE2EScale
[INFO]
[INFO] Results:
[INFO]
[WARNING] Tests run: 167, Failures: 0, Errors: 0, Skipped: 27
[INFO]
[INFO]
[INFO] --- animal-sniffer-maven-plugin:1.16:check (signature-check) @ hadoop-azure ---
[INFO] Checking unresolved references to org.codehaus.mojo.signature:java18:1.0
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (depcheck) @ hadoop-azure ---
[INFO]
[INFO] --- maven-failsafe-plugin:2.21.0:verify (integration-test-abfs-parallel-classesAndMethods) @ hadoop-azure ---
[INFO]
[INFO] --- maven-failsafe-plugin:2.21.0:verify (integration-test-abfs-parallel-classes) @ hadoop-azure ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:49 min (Wall Clock)
[INFO] Finished at: 2018-08-29T01:56:54+00:00
[INFO] Final Memory: 79M/616M
{noformat}
> ABFS: Reduce test run time via parallelization and grouping
> -----------------------------------------------------------
>
> Key: HADOOP-15664
> URL: https://issues.apache.org/jira/browse/HADOOP-15664
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Reporter: Thomas Marquardt
> Assignee: Da Zhou
> Priority: Major
> Attachments: HADOOP-15664-HADOOP-15407-001.patch
>
>
> 1) Let's reduce the total test runtime by improving parallelization of the tests.
> 2) Let's make it possible to select WASB tests, ABFS tests, or both so developers can run only the tests appropriate for the change they've made.
> 3) Update the testing-azure.md accordingly
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org