[jira] [Created] (HUDI-1375) Fix bug for HoodieAvroUtils.removeMetadataFields

["Wenning Ding (Jira)" <ji...@apache.org>]

Wenning Ding created HUDI-1375:
----------------------------------

             Summary: Fix bug for HoodieAvroUtils.removeMetadataFields
                 Key: HUDI-1375
                 URL: https://issues.apache.org/jira/browse/HUDI-1375
             Project: Apache Hudi
          Issue Type: Bug
            Reporter: Wenning Ding


There's a code bug in HoodieAvroUtils.removeMetadataFields().

Reproduction steps:

 

{{}}
{code:java}
import org.apache.hudi.avro.HoodieAvroUtils
import org.apache.avro.Schema
val EXAMPLE_SCHEMA = """{"type": "record","name": "testrec","fields": [ {"name": "timestamp","type": "double"},{"name": "_row_key", "type": "string"},{"name": "non_pii_col", "type": "string"},{"name": "pii_col", "type": "string", "column_category": "user_profile"}]}"""
val schema = HoodieAvroUtils.addMetadataFields(newSchema.Parser().parse(EXAMPLE_SCHEMA))
HoodieAvroUtils.removeMetadataFields(schema)
{code}
{{}}

Then it would throw an exception:

 

{{}}
{code:java}
org.apache.avro.AvroRuntimeException: Field already used: timestamp type:DOUBLE pos:5 at org.apache.avro.Schema$RecordSchema.setFields(Schema.java:647) at org.apache.hudi.avro.HoodieAvroUtils.removeMetadataFields(HoodieAvroUtils.java:209) ... 49 elided {code}
{{}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)