You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Sandy Ryza <sa...@cloudera.com> on 2014/09/25 23:12:41 UTC

Kryo UnsupportedOperationException

We're running into an error (below) when trying to read spilled shuffle
data back in.

Has anybody encountered this before / is anybody familiar with what causes
these Kryo UnsupportedOperationExceptions?

any guidance appreciated,
Sandy

---
com.esotericsoftware.kryo.KryoException
(com.esotericsoftware.kryo.KryoException:
java.lang.UnsupportedOperationException Serialization trace: <omitted
variable name> (<omitted class name>) <omitted variable name> (<omitted
class name>))

com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.read(FieldSerializer.java:626)

com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:221)

com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:648)

com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.read(FieldSerializer.java:605)

com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:221)

com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:729)

com.twitter.chill.TraversableSerializer.read(Traversable.scala:44)
...

Re: Kryo UnsupportedOperationException

Posted by Ian O'Connell <ia...@ianoconnell.com>.

I would guess the field serializer is having issues being able to
reconstruct the class again, its pretty much best effort.

Is this an intermediate type?

On Thu, Sep 25, 2014 at 2:12 PM, Sandy Ryza <sa...@cloudera.com> wrote:

> We're running into an error (below) when trying to read spilled shuffle
> data back in.
>
> Has anybody encountered this before / is anybody familiar with what causes
> these Kryo UnsupportedOperationExceptions?
>
> any guidance appreciated,
> Sandy
>
> ---
> com.esotericsoftware.kryo.KryoException
> (com.esotericsoftware.kryo.KryoException:
> java.lang.UnsupportedOperationException Serialization trace: <omitted
> variable name> (<omitted class name>) <omitted variable name> (<omitted
> class name>))
>
>
> com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.read(FieldSerializer.java:626)
>
>
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:221)
>
> com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:648)
>
>
> com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.read(FieldSerializer.java:605)
>
>
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:221)
>
> com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:729)
>
> com.twitter.chill.TraversableSerializer.read(Traversable.scala:44)
> ...
>
>