You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Jim Brennan (Jira)" <ji...@apache.org> on 2019/09/19 15:31:00 UTC
[jira] [Commented] (YARN-9844) TestCapacitySchedulerPerf test
errors in branch-2
[ https://issues.apache.org/jira/browse/YARN-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16933510#comment-16933510 ]
Jim Brennan commented on YARN-9844:
-----------------------------------
Here is a output from running the tests:
{noformat}
mvn test -DRunCapacitySchedulerPerfTests=true -Dtest=TestCapacitySchedulerPerf
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.10.0-SNAPSHOT
[WARNING] 'dependencyManagement.dependencies.dependency.(groupId:artifactId:type:classifier)' must be unique: com.microsoft.azure:azure-storage:jar -> version 7.0.0 vs 5.4.0 @ org.apache.hadoop:hadoop-project:2.10.0-SNAPSHOT, /Users/jbrennan02/git/apache-hadoop/hadoop-project/pom.xml, line 1175, column 19
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Hadoop YARN ResourceManager 2.10.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-yarn-server-resourcemanager ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- hadoop-maven-plugins:2.10.0-SNAPSHOT:protoc (compile-protoc) @ hadoop-yarn-server-resourcemanager ---
[INFO] Wrote protoc checksums to file /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/hadoop-maven-plugins-protoc-checksums.json
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hadoop-yarn-server-resourcemanager ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hadoop-yarn-server-resourcemanager ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/resources
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hadoop-yarn-server-resourcemanager ---
[INFO] Compiling 3 source files to /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/classes
[INFO]
[INFO] --- hadoop-maven-plugins:2.10.0-SNAPSHOT:test-protoc (compile-test-protoc) @ hadoop-yarn-server-resourcemanager ---
[INFO] Wrote protoc checksums to file /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/hadoop-maven-plugins-protoc-checksums.json
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ hadoop-yarn-server-resourcemanager ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 11 resources
[INFO] Copying 1 resource
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hadoop-yarn-server-resourcemanager ---
[INFO] Compiling 1 source file to /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes
[INFO]
[INFO] --- maven-jar-plugin:2.5:test-jar (default) @ hadoop-yarn-server-resourcemanager ---
[INFO] Building jar: /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/hadoop-yarn-server-resourcemanager-2.10.0-SNAPSHOT-tests.jar
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ hadoop-yarn-server-resourcemanager ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf
[ERROR] Tests run: 4, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 42.365 s <<< FAILURE! - in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf
[ERROR] testUserLimitThroughputForFiveResources(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf) Time elapsed: 0.038 s <<< ERROR!
java.lang.ArrayIndexOutOfBoundsException: 2
at org.apache.hadoop.yarn.api.records.Resource.getResourceInformation(Resource.java:241)
at org.apache.hadoop.yarn.api.records.Resource.setResourceValue(Resource.java:351)
at org.apache.hadoop.yarn.util.resource.ResourceUtils.getResourceTypesMinimumAllocation(ResourceUtils.java:534)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getMinimumAllocation(AbstractYarnScheduler.java:1341)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:322)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:395)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:749)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1146)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:302)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:161)
at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:140)
at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:136)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputWithNumberOfResourceTypes(TestCapacitySchedulerPerf.java:114)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputForFiveResources(TestCapacitySchedulerPerf.java:263)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
[ERROR] testUserLimitThroughputForFourResources(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf) Time elapsed: 0.031 s <<< ERROR!
java.lang.ArrayIndexOutOfBoundsException: 2
at org.apache.hadoop.yarn.api.records.Resource.getResourceInformation(Resource.java:241)
at org.apache.hadoop.yarn.api.records.Resource.setResourceValue(Resource.java:351)
at org.apache.hadoop.yarn.util.resource.ResourceUtils.getResourceTypesMinimumAllocation(ResourceUtils.java:534)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getMinimumAllocation(AbstractYarnScheduler.java:1341)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:322)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:395)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:749)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1146)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:302)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:161)
at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:140)
at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:136)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputWithNumberOfResourceTypes(TestCapacitySchedulerPerf.java:114)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputForFourResources(TestCapacitySchedulerPerf.java:258)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
[ERROR] testUserLimitThroughputForThreeResources(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf) Time elapsed: 0.031 s <<< ERROR!
java.lang.ArrayIndexOutOfBoundsException: 2
at org.apache.hadoop.yarn.api.records.Resource.getResourceInformation(Resource.java:241)
at org.apache.hadoop.yarn.api.records.Resource.setResourceValue(Resource.java:351)
at org.apache.hadoop.yarn.util.resource.ResourceUtils.getResourceTypesMinimumAllocation(ResourceUtils.java:534)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.getMinimumAllocation(AbstractYarnScheduler.java:1341)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:322)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:395)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:749)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1146)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:302)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:161)
at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:140)
at org.apache.hadoop.yarn.server.resourcemanager.MockRM.<init>(MockRM.java:136)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputWithNumberOfResourceTypes(TestCapacitySchedulerPerf.java:114)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerPerf.testUserLimitThroughputForThreeResources(TestCapacitySchedulerPerf.java:253)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestCapacitySchedulerPerf.testUserLimitThroughputForFiveResources:263->testUserLimitThroughputWithNumberOfResourceTypes:114 » ArrayIndexOutOfBounds
[ERROR] TestCapacitySchedulerPerf.testUserLimitThroughputForFourResources:258->testUserLimitThroughputWithNumberOfResourceTypes:114 » ArrayIndexOutOfBounds
[ERROR] TestCapacitySchedulerPerf.testUserLimitThroughputForThreeResources:253->testUserLimitThroughputWithNumberOfResourceTypes:114 » ArrayIndexOutOfBounds
[INFO]
[ERROR] Tests run: 4, Failures: 0, Errors: 3, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 52.113 s
[INFO] Finished at: 2019-09-19T09:57:09-05:00
[INFO] Final Memory: 49M/626M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test (default-test) on project hadoop-yarn-server-resourcemanager: There are test failures.
[ERROR]
[ERROR] Please refer to /Users/jbrennan02/git/apache-hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExcep
{noformat}
cc: [~jhung]
> TestCapacitySchedulerPerf test errors in branch-2
> -------------------------------------------------
>
> Key: YARN-9844
> URL: https://issues.apache.org/jira/browse/YARN-9844
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, yarn
> Affects Versions: 2.10.0
> Reporter: Jim Brennan
> Priority: Major
>
> These TestCapacitySchedulerPerf throughput tests are failing in branch-2:
> {{[ERROR] TestCapacitySchedulerPerf.testUserLimitThroughputForFiveResources:263->testUserLimitThroughputWithNumberOfResourceTypes:114 » ArrayIndexOutOfBounds}}
> {{[ERROR] TestCapacitySchedulerPerf.testUserLimitThroughputForFourResources:258->testUserLimitThroughputWithNumberOfResourceTypes:114 » ArrayIndexOutOfBounds}}
> {{[ERROR] TestCapacitySchedulerPerf.testUserLimitThroughputForThreeResources:253->testUserLimitThroughputWithNumberOfResourceTypes:114 » ArrayIndexOutOfBounds}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org