spark broadcast error

[Ke Wang <jk...@gmail.com>]

I want to ran my spark program on a YARN cluster. But when I tested broadcast
function in my program, 
I got an error. 

Exception in thread "main" org.apache.spark.SparkException: Error sending
message as driverActor is null [message =
UpdateBlockInfo(BlockManagerId(<driver>, in160-011.byted.org, 19704,
0),broadcast_0_piece0,StorageLevel(false, true, false, false, 1),61,0,0)]
	at org.apache.spark.util.AkkaUtils$.askWithReply(AkkaUtils.scala:167)
	at
org.apache.spark.storage.BlockManagerMaster.askDriverWithReply(BlockManagerMaster.scala:218)
	at
org.apache.spark.storage.BlockManagerMaster.updateBlockInfo(BlockManagerMaster.scala:58)
	at
org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$tryToReportBlockStatus(BlockManager.scala:310)
	at
org.apache.spark.storage.BlockManager.reportBlockStatus(BlockManager.scala:286)
	at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:769)
	at org.apache.spark.storage.BlockManager.putBytes(BlockManager.scala:639)
	at
org.apache.spark.broadcast.TorrentBroadcast$$anonfun$writeBlocks$1.apply(TorrentBroadcast.scala:85)
	at
org.apache.spark.broadcast.TorrentBroadcast$$anonfun$writeBlocks$1.apply(TorrentBroadcast.scala:84)
	at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
	at
org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:84)
	at
org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:68)
	at
org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:36)
	at
org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:29)
	at
org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62)
	at org.apache.spark.SparkContext.broadcast(SparkContext.scala:809)

my test code just is :   val broadcastVar = sc.broadcast(Array(1, 2, 3))

I am new to spark, can anyone help me?






--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-broadcast-error-tp19643.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org