You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Arun Ahuja <aa...@gmail.com> on 2014/09/23 15:38:36 UTC

TorrentBroadcast causes java.io.IOException: unexpected exception type

Since upgrading to Spark 1.1 we have been seeing the following error in the
logs:

14/09/23 02:14:42 ERROR executor.Executor: Exception in task 1087.0 in
stage 0.0 (TID 607)
java.io.IOException: unexpected exception type
        at
java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1538)
        at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1025)
        at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
        at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
        at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
        at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
        at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
        at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
        at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
        at
org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:62)
        at
org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:87)
        at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:159)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.spark.SparkException: Failed to get
broadcast_3_piece0 of broadcast_3
        at
org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBlocks$1.apply$mcVI$sp(TorrentBroadcast.scala:124)
        at
org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBlocks$1.apply(TorrentBroadcast.scala:104)

Does anyone have some background on what change could have caused this?  Is
TorrentBroadcast now the default broadcast method?

Thanks

Arun