[GitHub] [iceberg] openinx commented on issue #1578: The AvroSchemaUtil will convert iceberg's optional map into confusing union.

[GitBox <gi...@apache.org>]

openinx commented on issue #1578:
URL: https://github.com/apache/iceberg/issues/1578#issuecomment-706550851


   Let's take an example to show why would this break the encode/decode unit tests.  For example, we have the schema: 
   ```java
     Schema schema = new Schema(
           required(0, "id", Types.LongType.get()),
           optional(1, "data", Types.MapType.ofOptional(2, 3,
               Types.LongType.get(),
               Types.StringType.get())));
   ```
   
   And we have an original record :
   
   ```
   record0 =  { id=1,  data = {Map{a=b, c=d, e=f}} }
   ```
   
   Firstly we encode the record0 by using the converted avro schema , then decode the record0 (using avro schema), we will get the unexpected record: 
   
   ```
   record1 = { id=1, data = {List{Map{a=b},  Map{c=d}, Map{e=f}}}}
   ```
   
   The assertion fail because `record0 != record1` . 
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org