You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Josh Rosen (JIRA)" <ji...@apache.org> on 2016/01/05 22:05:39 UTC
[jira] [Resolved] (SPARK-6689) MiniYarnCLuster still test failed
with hadoop-2.2
[ https://issues.apache.org/jira/browse/SPARK-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Rosen resolved SPARK-6689.
-------------------------------
Resolution: Cannot Reproduce
Resolving as "cannot reproduce" for now.
> MiniYarnCLuster still test failed with hadoop-2.2
> -------------------------------------------------
>
> Key: SPARK-6689
> URL: https://issues.apache.org/jira/browse/SPARK-6689
> Project: Spark
> Issue Type: Test
> Components: Tests, YARN
> Affects Versions: 1.3.0
> Reporter: Zhang, Liye
> Priority: Minor
>
> when running unit test *YarnClusterSuite* with *hadoop-2.2*, exception will throw because *Timed out waiting for RM to come up*. Some previously related discussion can be traced in [spark-3710|https://issues.apache.org/jira/browse/SPARK-3710] ([PR2682|https://github.com/apache/spark/pull/2682]) and [spark-2778|https://issues.apache.org/jira/browse/SPARK-2778] ([PR2605|https://github.com/apache/spark/pull/2605]).
> With command *build/sbt -Pyarn -Phadoop-2.2 "test-only org.apache.spark.deploy.yarn.YarnClusterSuite"*, will get following exceptions:
> {noformat}
> [info] Exception encountered when attempting to run a suite with class name: org.apache.spark.deploy.yarn.YarnClusterSuite *** ABORTED *** (15 seconds, 799 milliseconds)
> [info] java.lang.IllegalStateException: Timed out waiting for RM to come up.
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.beforeAll(YarnClusterSuite.scala:114)
> [info] at org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.beforeAll(YarnClusterSuite.scala:44)
> [info] at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:253)
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.run(YarnClusterSuite.scala:44)
> [info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:462)
> [info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:671)
> [info] at sbt.ForkMain$Run$2.call(ForkMain.java:294)
> [info] at sbt.ForkMain$Run$2.call(ForkMain.java:284)
> [info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [info] at java.lang.Thread.run(Thread.java:745)
> {noformat}
> And without *-Phadoop-2.2* or replace it with "*-Dhadoop.version*" (e.g. build/sbt -Pyarn "test-only org.apache.spark.deploy.yarn.YarnClusterSuite") more info will come out:
> {noformat}
> Exception in thread "Thread-7" java.lang.NoClassDefFoundError: org/mortbay/jetty/servlet/Context
> at org.apache.hadoop.yarn.webapp.WebApps.$for(WebApps.java:309)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:602)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:655)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper$2.run(MiniYARNCluster.java:219)
> Caused by: java.lang.ClassNotFoundException: org.mortbay.jetty.servlet.Context
> at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> [info] Resolving org.apache.hadoop#hadoop-yarn-server-common;2.2.0 ...
> Exception in thread "Thread-18" java.lang.NoClassDefFoundError: org/mortbay/jetty/servlet/Context
> at org.apache.hadoop.yarn.webapp.WebApps.$for(WebApps.java:309)
> at org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer.serviceStart(WebServer.java:62)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
> at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:199)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper$1.run(MiniYARNCluster.java:337)
> Caused by: java.lang.ClassNotFoundException: org.mortbay.jetty.servlet.Context
> at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> [info] Resolving org.apache.hadoop#hadoop-yarn;2.2.0 ...
> [info] Exception encountered when attempting to run a suite with class name: org.apache.spark.deploy.yarn.YarnClusterSuite *** ABORTED *** (15 seconds, 799 milliseconds)
> [info] java.lang.IllegalStateException: Timed out waiting for RM to come up.
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.beforeAll(YarnClusterSuite.scala:114)
> [info] at org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.beforeAll(YarnClusterSuite.scala:44)
> [info] at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:253)
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.run(YarnClusterSuite.scala:44)
> [info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:462)
> [info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:671)
> [info] at sbt.ForkMain$Run$2.call(ForkMain.java:294)
> [info] at sbt.ForkMain$Run$2.call(ForkMain.java:284)
> [info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [info] at java.lang.Thread.run(Thread.java:745)
> {noformat}
> If we add the following code into *yarn/pom.xml*,
> {code:xml}
> <dependency>
> <groupId>javax.ws.rs</groupId>
> <artifactId>jsr311-api</artifactId>
> <version>1.1.1</version>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.mortbay.jetty</groupId>
> <artifactId>jetty</artifactId>
> <version>6.1.26</version>
> <scope>test</scope>
> </dependency>
> {code}
> The error message would be :
> {noformat}
> Exception in thread "Thread-188" org.apache.hadoop.yarn.exceptions.YarnRuntimeException: NMWebapps failed to start.
> at org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer.serviceStart(WebServer.java:76)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
> at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:199)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper$1.run(MiniYARNCluster.java:337)
> Caused by: org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
> at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:262)
> at org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer.serviceStart(WebServer.java:62)
> ... 5 more
> Caused by: java.io.IOException: Problem in starting http server. Server handlers failed
> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:700)
> at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:257)
> [info] Resolving org.apache.hadoop#hadoop-yarn-common;2.2.0 ...
> [info] Exception encountered when attempting to run a suite with class name: org.apache.spark.deploy.yarn.YarnClusterSuite *** ABORTED *** (3 seconds, 455 milliseconds)
> [info] org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException: NodeManager 0 failed to start
> [info] at org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:352)
> [info] at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> [info] at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
> [info] at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.beforeAll(YarnClusterSuite.scala:94)
> [info] at org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.beforeAll(YarnClusterSuite.scala:44)
> [info] at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:253)
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.run(YarnClusterSuite.scala:44)
> [info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:462)
> [info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:671)
> [info] at sbt.ForkMain$Run$2.call(ForkMain.java:294)
> [info] at sbt.ForkMain$Run$2.call(ForkMain.java:284)
> [info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [info] at java.lang.Thread.run(Thread.java:745)
> [info] Cause: java.io.IOException: NodeManager 0 failed to start
> [info] at org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:348)
> [info] at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> [info] at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
> [info] at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.beforeAll(YarnClusterSuite.scala:94)
> [info] at org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.beforeAll(YarnClusterSuite.scala:44)
> [info] at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:253)
> [info] at org.apache.spark.deploy.yarn.YarnClusterSuite.run(YarnClusterSuite.scala:44)
> [info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:462)
> [info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:671)
> [info] at sbt.ForkMain$Run$2.call(ForkMain.java:294)
> [info] at sbt.ForkMain$Run$2.call(ForkMain.java:284)
> [info] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [info] at java.lang.Thread.run(Thread.java:745)
> {noformat}
> *NOTE: This issue does not exist for hadoop-2.2+*
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org