You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "yuemeng (A)" <yu...@huawei.com> on 2016/04/21 15:49:02 UTC

how to change akka.remote.startup-timeout value in spark

When I run a spark application ,sometimes I will get follow error:
16/04/21 09:26:45 ERROR SparkContext: Error initializing SparkContext.
java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds]
         at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
         at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
         at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
         at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
         at scala.concurrent.Await$.result(package.scala:107)
         at akka.remote.Remoting.start(Remoting.scala:180)
         at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)
         at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:618)
         at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:615)
         at akka.actor.ActorSystemImpl._start(ActorSystem.scala:615)
         at akka.actor.ActorSystemImpl.start(ActorSystem.scala:632)
         at akka.actor.ActorSystem$.apply(ActorSystem.scala:141)
         at akka.actor.ActorSystem$.apply(ActorSystem.scala:118)
         at org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:122)
         at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:54)
         at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53)
         at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1995)
         at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
         at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1986)
         at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:56)
         at org.apache.spark.rpc.akka.AkkaRpcEnvFactory.create(AkkaRpcEnv.scala:245)

And,I find maybe we can increase akka.remote.startup-timeout vaule to solve this problem after I track the code,but I can't find any way to change this value
Can we have some methods to change akka config in spark,or some good way to solve this problem
Any advise will be thankful
Thanks a lot