You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Jaonary Rabarisoa <ja...@gmail.com> on 2014/03/13 13:36:30 UTC

How to solve : java.io.NotSerializableException: org.apache.hadoop.io.Text ?

Dear all,

I have a SequenceFile[Text,BytesWritable] that I load with :

val data = context.sequenceFile("data", classOf[Text],
classOf[BytesWritable])

I want to view may data with

data.collect().foreach {
      d =>
        println(d)
    }

but I got this java.io.NotSerializableException error.

Any ideas ?

Re: How to solve : java.io.NotSerializableException: org.apache.hadoop.io.Text ?

Posted by Shixiong Zhu <zs...@gmail.com>.

Hi,

Text and BytesWritable do not implement Serializable. You can convert them
to String and Array[Byte]. E.g.,

data.map { case (text, bytes) => (text.toString, bytes.copyBytes)
}.collect().foreach
{
      d =>
        println(d)
}

Best Regards,
Shixiong Zhu


2014-03-13 20:36 GMT+08:00 Jaonary Rabarisoa <ja...@gmail.com>:

> Dear all,
>
> I have a SequenceFile[Text,BytesWritable] that I load with :
>
> val data = context.sequenceFile("data", classOf[Text],
> classOf[BytesWritable])
>
> I want to view may data with
>
> data.collect().foreach {
>       d =>
>         println(d)
>     }
>
> but I got this java.io.NotSerializableException error.
>
> Any ideas ?
>
>
>
>
>