You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/14 11:14:21 UTC
[GitHub] [hudi] KnightChess commented on issue #6621: [SUPPORT]com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: 36
KnightChess commented on issue #6621:
URL: https://github.com/apache/hudi/issues/6621#issuecomment-1278866711
we also encountered this problem, in our env, flink-bundle avro version is 1.10.0 and spark3-bundle avro version is 1.10.2, and after flink write mor table with deletelogblock
spark:
spark query will throw exception, classId - 2 = index is id 36
```shell
Caused by: com.esotericsoftware.kryo.KryoException: java.lang.IndexOutOfBoundsException: Index: 36, Size: 7
Serialization trace:
string (org.apache.hudi.org.apache.avro.util.Utf8)
orderingVal (org.apache.hudi.common.model.DeleteRecord)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:391)
at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:302)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
at org.apache.hudi.common.util.SerializationUtils$KryoSerializerInstance.deserialize(SerializationUtils.java:104)
at org.apache.hudi.common.util.SerializationUtils.deserialize(SerializationUtils.java:78)
at org.apache.hudi.common.table.log.block.HoodieDeleteBlock.deserialize(HoodieDeleteBlock.java:106)
at org.apache.hudi.common.table.log.block.HoodieDeleteBlock.getRecordsToDelete(HoodieDeleteBlock.java:91)
at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.processQueuedBlocksForInstant(AbstractHoodieLogRecordReader.java:476)
at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scanInternal(AbstractHoodieLogRecordReader.java:346)
... 30 more
Caused by: java.lang.IndexOutOfBoundsException: Index: 36, Size: 7
at java.util.ArrayList.rangeCheck(ArrayList.java:659)
at java.util.ArrayList.get(ArrayList.java:435)
at com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:60)
at com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:857)
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:780)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
... 44 more
```
presto:
presto will throw the same exception which you meet, presto avro version is 1.8.2
resovle:
In spark, I change the avro version to 1.10.0, anything will be ok in query, I didn't check for presto, you can have a try in spark @dik111
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org