You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Flavio Pompermaier <po...@okkam.it> on 2015/08/26 09:04:17 UTC
Encountered unregistered class ID
Hi to all,
I'm running a job (with Flink 0.10-SNAPSHOT) that reads some parquet-thrift
objects and then it performs some joins and I receive the following
exception:
Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
terminated due to an exception: Encountered unregistered class ID: 93
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:784)
Caused by: com.esotericsoftware.kryo.KryoException: Encountered
unregistered class ID: 93
at
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752)
at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:211)
at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:225)
at
org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:235)
at
org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase.copy(TupleSerializerBase.java:71)
at
org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:522)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1328)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:781)
What could be the cause of this?
Best,
Flavio
Re: Encountered unregistered class ID
Posted by Robert Metzger <rm...@apache.org>.
Hi Flavio,
can you share a minimal version of your program to reproduce the issue?
On Wed, Aug 26, 2015 at 10:36 AM, Flavio Pompermaier <po...@okkam.it>
wrote:
> I'm running my job from my Eclipse and I don't register any Kryo class in
> the env.
>
> On Wed, Aug 26, 2015 at 10:34 AM, Stephan Ewen <se...@apache.org> wrote:
>
>> Hi Flavio!
>>
>> That exception means that the Kryo serializers are not in sync. The
>> writers have registered types that the readers do not know.
>>
>> Two possible reasons that I can think of from the top of my head:
>>
>> 1) Do you manually register types? Are you registering new types in the
>> middle of your program?
>>
>> 2) Flink may not be passing the Kryo registration properly down to the
>> sort reader.
>>
>>
>> Greetings,
>> Stephan
>>
>>
>> On Wed, Aug 26, 2015 at 9:04 AM, Flavio Pompermaier <pompermaier@okkam.it
>> > wrote:
>>
>>>
>>> Hi to all,
>>> I'm running a job (with Flink 0.10-SNAPSHOT) that reads some
>>> parquet-thrift objects and then it performs some joins and I receive the
>>> following exception:
>>>
>>> Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
>>> terminated due to an exception: Encountered unregistered class ID: 93
>>> at
>>> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:784)
>>> Caused by: com.esotericsoftware.kryo.KryoException: Encountered
>>> unregistered class ID: 93
>>> at
>>> com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
>>> at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641)
>>> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752)
>>> at
>>> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:211)
>>> at
>>> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:225)
>>> at
>>> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:235)
>>> at
>>> org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase.copy(TupleSerializerBase.java:71)
>>> at
>>> org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:522)
>>> at
>>> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1328)
>>> at
>>> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:781)
>>>
>>> What could be the cause of this?
>>>
>>> Best,
>>> Flavio
>>>
>>>
>>>
>>
>
>
Re: Encountered unregistered class ID
Posted by Flavio Pompermaier <po...@okkam.it>.
I'm running my job from my Eclipse and I don't register any Kryo class in
the env.
On Wed, Aug 26, 2015 at 10:34 AM, Stephan Ewen <se...@apache.org> wrote:
> Hi Flavio!
>
> That exception means that the Kryo serializers are not in sync. The
> writers have registered types that the readers do not know.
>
> Two possible reasons that I can think of from the top of my head:
>
> 1) Do you manually register types? Are you registering new types in the
> middle of your program?
>
> 2) Flink may not be passing the Kryo registration properly down to the
> sort reader.
>
>
> Greetings,
> Stephan
>
>
> On Wed, Aug 26, 2015 at 9:04 AM, Flavio Pompermaier <po...@okkam.it>
> wrote:
>
>>
>> Hi to all,
>> I'm running a job (with Flink 0.10-SNAPSHOT) that reads some
>> parquet-thrift objects and then it performs some joins and I receive the
>> following exception:
>>
>> Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
>> terminated due to an exception: Encountered unregistered class ID: 93
>> at
>> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:784)
>> Caused by: com.esotericsoftware.kryo.KryoException: Encountered
>> unregistered class ID: 93
>> at
>> com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
>> at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641)
>> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752)
>> at
>> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:211)
>> at
>> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:225)
>> at
>> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:235)
>> at
>> org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase.copy(TupleSerializerBase.java:71)
>> at
>> org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:522)
>> at
>> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1328)
>> at
>> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:781)
>>
>> What could be the cause of this?
>>
>> Best,
>> Flavio
>>
>>
>>
>
Re: Encountered unregistered class ID
Posted by Stephan Ewen <se...@apache.org>.
Hi Flavio!
That exception means that the Kryo serializers are not in sync. The writers
have registered types that the readers do not know.
Two possible reasons that I can think of from the top of my head:
1) Do you manually register types? Are you registering new types in the
middle of your program?
2) Flink may not be passing the Kryo registration properly down to the sort
reader.
Greetings,
Stephan
On Wed, Aug 26, 2015 at 9:04 AM, Flavio Pompermaier <po...@okkam.it>
wrote:
>
> Hi to all,
> I'm running a job (with Flink 0.10-SNAPSHOT) that reads some
> parquet-thrift objects and then it performs some joins and I receive the
> following exception:
>
> Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
> terminated due to an exception: Encountered unregistered class ID: 93
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:784)
> Caused by: com.esotericsoftware.kryo.KryoException: Encountered
> unregistered class ID: 93
> at
> com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
> at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641)
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752)
> at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:211)
> at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:225)
> at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:235)
> at
> org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase.copy(TupleSerializerBase.java:71)
> at
> org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:522)
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1328)
> at
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:781)
>
> What could be the cause of this?
>
> Best,
> Flavio
>
>
>