[jira] [Commented] (AVRO-2493) Unable to register Logical Type for custom Conversion class

["Michael Snell (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-2493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921314#comment-16921314 ] 

Michael Snell commented on AVRO-2493:
-------------------------------------

I've found a similar problem. The workaround I've used is:
- My Conversion classes call the static LogicalTypes.register method in their static initialisers, so that when IDLProtocolMojo.doCompile registers the custom conversions, the logical types are also registered
- IDLProtocolMojo.doCompile only registers the custom conversions after parsing the Protocol, which needs the logical types to already be present, otherwise LogicalTypes.fromSchemaImpl will return null, which causes the first file parsed to not use the custom logical types. To avoid this, run the compilation step twice, first for a sacrificial protocol  which is not used, and then again for all the actual protocols
but clearly a proper fix is required.

> Unable to register Logical Type for custom Conversion class
> -----------------------------------------------------------
>
>                 Key: AVRO-2493
>                 URL: https://issues.apache.org/jira/browse/AVRO-2493
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: java, logical types
>    Affects Versions: 1.9.0
>            Reporter: Travis Yocum
>            Priority: Major
>
> I have created a custom conversion class for Java's (1.8) java.time.OffsetDateTime.
> {code:java}
> public class OffsetDateTimeConversion extends Conversion<OffsetDateTime> {
>     private static final DateTimeFormatter DATE_TIME_FORMATTER = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.nnnnnnZZZZZ");
>     @Override
>     public Class<OffsetDateTime> getConvertedType() {
>         return OffsetDateTime.class;
>     }
>     @Override
>     public String getLogicalTypeName() {
>         return "offset-date-time";
>     }
>     @Override
>     public OffsetDateTime fromCharSequence(CharSequence value, Schema schema, LogicalType type) {
>         return OffsetDateTime.parse(value, DATE_TIME_FORMATTER);
>     }
>     @Override
>     public CharSequence toCharSequence(OffsetDateTime value, Schema schema, LogicalType type) {
>         return value.format(DATE_TIME_FORMATTER);
>     }
> }
> {code}
>  And my simple schema to test (including a field that uses the built-in "time-millis" converter in Avro 1.9 for testing purposes):
> TimeTest.avsc
>  
> {code:java}
> {
>    "type": "record",
>    "name": "TimeTest",
>    "namespace": "time.test",
>    "fields": [
>       {
>            "name": "createTime",
>            "type": {
>                "type": "int",
>                "logicalType": "time-millis"
>            }
>       },
>       {
>            "name": "createDateTime",
>            "type": {
>                "type": "string",
>                "logicalType": "offset-date-time"
>            }
>       }
>    ]
> }
> {code}
>  
> I've also created a LogicalType for my conversion field that I need to register:
>  
> {code:java}
> public class OffsetDateTimeLogicalType extends LogicalType {
>     public static final String LOGICAL_DATE_TIME_NAME = "offset-date-time";
>     public OffsetDateTimeLogicalType() {
>         super(LOGICAL_DATE_TIME_NAME);
>     }
>     @Override
>     public void validate(Schema schema) {
>         super.validate(schema);
>         if (schema.getType() != Schema.Type.STRING) {
>             throw new IllegalArgumentException("Logical type 'offset-date-time' must be of type string");
>         }
>     }
> }{code}
>  
> I've been debugging the avro-maven-plugin and have been able to pinpoint where my issue is occurring while parsing the schema:
>  
> {code:java}
> // parse logical type if present
> result.logicalType = LogicalTypes.fromSchemaIgnoreInvalid(result);
> {code}
>  
> The schema's "logicalType" property is being set to null because it's not a registered logical type.
> The code I need to execute is:
>  
> {code:java}
> LogicalTypes.register(getLogicalTypeName(), schema -> new OffsetDateTimeLogicalType());
> {code}
> Without modifying the plugin code, I see no way to execute this register so that the Schema.parse method (which is called before the conversion class is executed) will recognize this LogicalType.
>  
> The plugin has a config property <enableDecimalLogicalType> to allow for registering the Decimal logical types but nothing to enable other logical types.
> Am I missing something or is this a real issue?



--
This message was sent by Atlassian Jira
(v8.3.2#803003)