[jira] [Closed] (HUDI-5484) Avoid using GenericRecord in ColumnStatMetadata

["Alexey Kudinkin (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexey Kudinkin closed HUDI-5484.
---------------------------------
    Resolution: Fixed

> Avoid using GenericRecord in ColumnStatMetadata
> -----------------------------------------------
>
>                 Key: HUDI-5484
>                 URL: https://issues.apache.org/jira/browse/HUDI-5484
>             Project: Apache Hudi
>          Issue Type: Bug
>    Affects Versions: 0.12.1
>            Reporter: dzcxzl
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.13.0
>
>
>  
>  
> {code:java}
> org.apache.hudi.com.esotericsoftware.kryo.KryoException: java.lang.UnsupportedOperationException
> Serialization trace:
> reserved (org.apache.avro.Schema$Field)
> fieldMap (org.apache.avro.Schema$RecordSchema)
> schema (org.apache.avro.generic.GenericData$Record)
> maxValue (org.apache.hudi.avro.model.HoodieMetadataColumnStats)
> columnStatMetadata (org.apache.hudi.metadata.HoodieMetadataPayload)
>     at org.apache.hudi.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)    
>     at org.apache.hudi.common.model.HoodieAvroRecord.readRecordPayload(HoodieAvroRecord.java:232)
>     at org.apache.hudi.common.model.HoodieAvroRecord.readRecordPayload(HoodieAvroRecord.java:45)
>     at org.apache.hudi.common.model.HoodieRecord.read(HoodieRecord.java:339)
>     at org.apache.hudi.com.esotericsoftware.kryo.serializers.DefaultSerializers$KryoSerializableSerializer.read(DefaultSerializers.java:520)
>     at org.apache.hudi.com.esotericsoftware.kryo.serializers.DefaultSerializers$KryoSerializableSerializer.read(DefaultSerializers.java:512)
>     at org.apache.hudi.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
>     at org.apache.hudi.common.util.SerializationUtils$KryoSerializerInstance.deserialize(SerializationUtils.java:101)
>     at org.apache.hudi.common.util.SerializationUtils.deserialize(SerializationUtils.java:75)
>     at org.apache.hudi.common.util.collection.BitCaskDiskMap.get(BitCaskDiskMap.java:210)
>     at org.apache.hudi.common.util.collection.BitCaskDiskMap.get(BitCaskDiskMap.java:203)
>     at org.apache.hudi.common.util.collection.BitCaskDiskMap.get(BitCaskDiskMap.java:199)
>     at org.apache.hudi.common.util.collection.BitCaskDiskMap.get(BitCaskDiskMap.java:68)
>     at org.apache.hudi.common.util.collection.ExternalSpillableMap.get(ExternalSpillableMap.java:195)
>     at org.apache.hudi.common.util.collection.ExternalSpillableMap.get(ExternalSpillableMap.java:54)
>     at org.apache.hudi.io.HoodieCreateHandle.write(HoodieCreateHandle.java:188)
>     at org.apache.hudi.table.HoodieSparkCopyOnWriteTable.handleInsert(HoodieSparkCopyOnWriteTable.java:257)
>     at org.apache.hudi.table.action.compact.CompactionExecutionHelper.writeFileAndGetWriteStats(CompactionExecutionHelper.java:68)
>     at org.apache.hudi.table.action.compact.HoodieCompactor.compact(HoodieCompactor.java:231)
>     at org.apache.hudi.table.action.compact.HoodieCompactor.lambda$compact$9cd4b1be$1(HoodieCompactor.java:129)
>     at org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070)Caused by: java.lang.UnsupportedOperationException
>     at java.util.Collections$UnmodifiableCollection.add(Collections.java:1055)
>     at org.apache.hudi.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
>     at org.apache.hudi.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
>     at org.apache.hudi.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
>     at org.apache.hudi.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125){code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)