You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Alexey Kudinkin (Jira)" <ji...@apache.org> on 2023/01/06 21:40:00 UTC
[jira] [Updated] (HUDI-5484) Avoid using GenericRecord in ColumnStatMetadata
[ https://issues.apache.org/jira/browse/HUDI-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexey Kudinkin updated HUDI-5484:
----------------------------------
Reviewers: Alexey Kudinkin
> Avoid using GenericRecord in ColumnStatMetadata
> -----------------------------------------------
>
> Key: HUDI-5484
> URL: https://issues.apache.org/jira/browse/HUDI-5484
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: dzcxzl
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.13.0
>
>
>
>
> {code:java}
> org.apache.hudi.com.esotericsoftware.kryo.KryoException: java.lang.UnsupportedOperationException
> Serialization trace:
> reserved (org.apache.avro.Schema$Field)
> fieldMap (org.apache.avro.Schema$RecordSchema)
> schema (org.apache.avro.generic.GenericData$Record)
> maxValue (org.apache.hudi.avro.model.HoodieMetadataColumnStats)
> columnStatMetadata (org.apache.hudi.metadata.HoodieMetadataPayload)
> at org.apache.hudi.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
> at org.apache.hudi.common.model.HoodieAvroRecord.readRecordPayload(HoodieAvroRecord.java:232)
> at org.apache.hudi.common.model.HoodieAvroRecord.readRecordPayload(HoodieAvroRecord.java:45)
> at org.apache.hudi.common.model.HoodieRecord.read(HoodieRecord.java:339)
> at org.apache.hudi.com.esotericsoftware.kryo.serializers.DefaultSerializers$KryoSerializableSerializer.read(DefaultSerializers.java:520)
> at org.apache.hudi.com.esotericsoftware.kryo.serializers.DefaultSerializers$KryoSerializableSerializer.read(DefaultSerializers.java:512)
> at org.apache.hudi.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
> at org.apache.hudi.common.util.SerializationUtils$KryoSerializerInstance.deserialize(SerializationUtils.java:101)
> at org.apache.hudi.common.util.SerializationUtils.deserialize(SerializationUtils.java:75)
> at org.apache.hudi.common.util.collection.BitCaskDiskMap.get(BitCaskDiskMap.java:210)
> at org.apache.hudi.common.util.collection.BitCaskDiskMap.get(BitCaskDiskMap.java:203)
> at org.apache.hudi.common.util.collection.BitCaskDiskMap.get(BitCaskDiskMap.java:199)
> at org.apache.hudi.common.util.collection.BitCaskDiskMap.get(BitCaskDiskMap.java:68)
> at org.apache.hudi.common.util.collection.ExternalSpillableMap.get(ExternalSpillableMap.java:195)
> at org.apache.hudi.common.util.collection.ExternalSpillableMap.get(ExternalSpillableMap.java:54)
> at org.apache.hudi.io.HoodieCreateHandle.write(HoodieCreateHandle.java:188)
> at org.apache.hudi.table.HoodieSparkCopyOnWriteTable.handleInsert(HoodieSparkCopyOnWriteTable.java:257)
> at org.apache.hudi.table.action.compact.CompactionExecutionHelper.writeFileAndGetWriteStats(CompactionExecutionHelper.java:68)
> at org.apache.hudi.table.action.compact.HoodieCompactor.compact(HoodieCompactor.java:231)
> at org.apache.hudi.table.action.compact.HoodieCompactor.lambda$compact$9cd4b1be$1(HoodieCompactor.java:129)
> at org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070)Caused by: java.lang.UnsupportedOperationException
> at java.util.Collections$UnmodifiableCollection.add(Collections.java:1055)
> at org.apache.hudi.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
> at org.apache.hudi.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
> at org.apache.hudi.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
> at org.apache.hudi.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125){code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)