You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by santhoma <sa...@yahoo.com> on 2014/03/24 10:41:19 UTC

Re: java.io.NotSerializableException Of dependent Java lib.

Can someone answer this question please?
 Specifically about  the Serializable implementation of dependent jars .. ?



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/java-io-NotSerializableException-Of-dependent-Java-lib-tp1973p3087.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: java.io.NotSerializableException Of dependent Java lib.

Posted by yaoxin <ya...@gmail.com>.

We now have a method to work this around.

For the classes that can't easily implement serialized, we wrap this class a
scala object.
For example:
     class A {}   // This class is not serializable, 
     object AHolder {
          private val a: A = new A()
          def get: A = a
    }

This works cause spark will serialize the instance that are used in the
closure.
But the scala object won't be serialized and shipped. It's loaded locally in
each worker JVM.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/java-io-NotSerializableException-Of-dependent-Java-lib-tp1973p3095.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.