Re: Task not serializable?

[Daniel Liu <hp...@gmail.com>]

Hi

I am new to Spark and I encountered this error when I try to map RDD[A] =>
RDD[Array[Double]] then collect the results.

A is a custom class extends Serializable. (Actually it's just a wrapper
class which wraps a few variables that are all serializable).

I also tried KryoSerializer according to this guide
http://spark.apache.org/docs/0.8.1/tuning.html and it gave the same error
message.

Daniel Liu

Re: Task not serializable?

[pedro <sk...@gmail.com>]

I'me still fairly new to this, but I found problems using classes in maps if
they used instance variables in part of the map function. It seems like for
maps and such to work correctly, it needs to be purely functional
programming.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Re-Task-not-serializable-tp3507p5506.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.