[jira] [Assigned] (HUDI-1375) Fix bug for HoodieAvroUtils.removeMetadataFields

["Wenning Ding (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-1375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenning Ding reassigned HUDI-1375:
----------------------------------

    Assignee: Wenning Ding

> Fix bug for HoodieAvroUtils.removeMetadataFields
> ------------------------------------------------
>
>                 Key: HUDI-1375
>                 URL: https://issues.apache.org/jira/browse/HUDI-1375
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: Wenning Ding
>            Assignee: Wenning Ding
>            Priority: Major
>
> There's a code bug in HoodieAvroUtils.removeMetadataFields().
> Reproduction steps:
>  
> {{}}
> {code:java}
> import org.apache.hudi.avro.HoodieAvroUtils
> import org.apache.avro.Schema
> val EXAMPLE_SCHEMA = """{"type": "record","name": "testrec","fields": [ {"name": "timestamp","type": "double"},{"name": "_row_key", "type": "string"},{"name": "non_pii_col", "type": "string"},{"name": "pii_col", "type": "string", "column_category": "user_profile"}]}"""
> val schema = HoodieAvroUtils.addMetadataFields(newSchema.Parser().parse(EXAMPLE_SCHEMA))
> HoodieAvroUtils.removeMetadataFields(schema)
> {code}
> {{}}
> Then it would throw an exception:
>  
> {{}}
> {code:java}
> org.apache.avro.AvroRuntimeException: Field already used: timestamp type:DOUBLE pos:5 at org.apache.avro.Schema$RecordSchema.setFields(Schema.java:647) at org.apache.hudi.avro.HoodieAvroUtils.removeMetadataFields(HoodieAvroUtils.java:209) ... 49 elided {code}
> {{}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)