[jira] [Comment Edited] (AVRO-3989) Schema conversion for logical types ignored when type is part of union

["Luciano Sabença (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17936427#comment-17936427 ] 

Luciano Sabença edited comment on AVRO-3989 at 3/18/25 9:30 AM:
----------------------------------------------------------------

Seeing the same error in `avro:1.12.0`. I pretty much agree with [~jasmdk] diagnostics. The problem seems that code generation is not taking in consideration logical types in union types. The fix for  AVRO-3536 seems to be on the serialisation flow, the bug we are describing here is on code generation. 

I also created a very simple example with this bug reproduction: https://github.com/luciano-sabenca/avro-3889-example


was (Author: JIRAUSER307271):
Seeing the same error in `avro:1.12.0`. I pretty much agree with [~jasmdk] diagnostics. The problem seems that code generation is not taking in consideration logical types in union types. The fix for  AVRO-3536 seems to be on the serialisation flow, the bug we are describing here is on code generation. 

> Schema conversion for logical types ignored when type is part of union
> ----------------------------------------------------------------------
>
>                 Key: AVRO-3989
>                 URL: https://issues.apache.org/jira/browse/AVRO-3989
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.11.3
>            Reporter: Jacob Stampe Mikkelsen
>            Priority: Major
>         Attachments: image-2024-06-02-21-32-25-461.png
>
>
> I have a schema with an optional timestamp which is to be rendered as millis and the generated Java class fails to get the conversions array and method added, which means that at runtime I get a ClassCastException when an "Instant" object is cast to a "Long" because there was no registered converter:
> The element with an optional timestamp in my schema:
> {{{}}
> {{  "name": "DecisionStartTime",}}
> {{  "type": [}}
> {{    "null",}}
> {{    {}}
> {{      "type": "long",}}
> {{      "logicalType": "timestamp-millis"}}
> {{    }}}
> {{  ],}}
> {{  "default": null}}
> {{}}}
> The problem seems to be here:
> [https://github.com/apache/avro/blame/5a60b5c43fe9c3bb40f4a1303913fdb3598bd4a5/lang/java/compiler/src/main/velocity/org/apache/avro/compiler/specific/templates/java/classic/record.vm#L198]
> Which only renders the conversion part if any of the fields has logicalType set and the actual method:
> [https://github.com/apache/avro/blob/5a60b5c43fe9c3bb40f4a1303913fdb3598bd4a5/lang/java/compiler/src/main/java/org/apache/avro/compiler/specific/SpecificCompiler.java#L976C3-L983C4]
> Does not take into account that a field which is a union does not have a logical type, however one of the union types might.
> *Proposed solution* - replace method
> {{  public boolean hasLogicalTypeField(Schema schema) {}}
> {{    for (Schema.Field field : schema.getFields()) {}}
> {{      if (field.schema().getLogicalType() != null) {}}
> {{        return true;}}
> {{      }}}
> {{    }}}
> {{    return false;}}
> {{  }}}
> With a method like this:
> {{  public boolean hasLogicalTypeField(Schema schema) {}}
> {{    for (Schema.Field field : schema.getFields()) {}}
> {{      if (field.schema().getLogicalType() != null) {}}
> {{        return true;}}
> {{      } else if (field.schema().type == Type.UNION) {}}
> {{        for (Schema type : field.schema().getTypes()) {}}
> {{          if (type.getLogicalType() != null) {}}
> {{            return true;}}
> {{          }}}
> {{        }}}
> {{      }}}
> {{    }}}
> {{    return false;}}
> {{  }}}
> The conversionInstance() method might also need tweaking in order to identify that the given type has a logical type as part of the union.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)