You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/09/29 09:23:49 UTC
[GitHub] [hudi] KnightChess opened a new issue, #6829: [SUPPORT][REOPEN] Caused by: java.lang.ClassNotFoundException: org.apache.hudi.org.apache.avro.util.Utf8
KnightChess opened a new issue, #6829:
URL: https://github.com/apache/hudi/issues/6829
as desc in #6588
Stacktrace:
```shell
00:34 WARN: [kryo] Unable to load class org.apache.hudi.org.apache.avro.util.Utf8 with kryo's ClassLoader. Retrying with current..
22/09/05 11:17:58 ERROR AbstractHoodieLogRecordReader: Got exception when reading log file
com.esotericsoftware.kryo.KryoException: Unable to find class: org.apache.hudi.org.apache.avro.util.Utf8
Serialization trace:
orderingVal (org.apache.hudi.common.model.DeleteRecord)
at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:160)
at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:693)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
```
Steps to reproduce the behavior:
1.use Flink to insert data
2.use spark to query data
3.has delete block in log file
for HoodieDeleteBlock, it will use kryo serialize
in flink and spark, the type char、varchar、string will use org.apache.avro.util.Utf8
flink:
spark:
The flink bundle has relocation the avro but spark and presto bundle didn't, so will cause ClassNotFound when the log file has delete block. So, how to compatible it will better?
```shell
<relocation>
<pattern>org.apache.avro.</pattern>
<shadedPattern>${flink.bundle.shade.prefix}org.apache.avro.</shadedPattern>
</relocation>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope closed issue #6829: [SUPPORT][REOPEN] Caused by: java.lang.ClassNotFoundException: org.apache.hudi.org.apache.avro.util.Utf8
Posted by GitBox <gi...@apache.org>.
codope closed issue #6829: [SUPPORT][REOPEN] Caused by: java.lang.ClassNotFoundException: org.apache.hudi.org.apache.avro.util.Utf8
URL: https://github.com/apache/hudi/issues/6829
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] KnightChess commented on issue #6829: [SUPPORT][REOPEN] Caused by: java.lang.ClassNotFoundException: org.apache.hudi.org.apache.avro.util.Utf8
Posted by GitBox <gi...@apache.org>.
KnightChess commented on issue #6829:
URL: https://github.com/apache/hudi/issues/6829#issuecomment-1263165665
@codope yes, it's work, but I think need fix to compatible between the multy engines
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on issue #6829: [SUPPORT][REOPEN] Caused by: java.lang.ClassNotFoundException: org.apache.hudi.org.apache.avro.util.Utf8
Posted by GitBox <gi...@apache.org>.
codope commented on issue #6829:
URL: https://github.com/apache/hudi/issues/6829#issuecomment-1276291877
Closing it as we have a PR that fixes the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on issue #6829: [SUPPORT][REOPEN] Caused by: java.lang.ClassNotFoundException: org.apache.hudi.org.apache.avro.util.Utf8
Posted by GitBox <gi...@apache.org>.
codope commented on issue #6829:
URL: https://github.com/apache/hudi/issues/6829#issuecomment-1263160137
@KnightChess Does the solution proposed in https://github.com/apache/hudi/issues/6588#issuecomment-1236580669 work for you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on issue #6829: [SUPPORT][REOPEN] Caused by: java.lang.ClassNotFoundException: org.apache.hudi.org.apache.avro.util.Utf8
Posted by GitBox <gi...@apache.org>.
codope commented on issue #6829:
URL: https://github.com/apache/hudi/issues/6829#issuecomment-1263196874
Yes, I think we should shade/relocate avro in all bundles where it is included. The issue here is that it is shaded in one bundle but not in another. By shading in all bundles, there shouldn't be a conflict between shaded and unshaded class. Created HUDI-4954 to track it. If you would like to contribute, feel free to put up a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org