[jira] [Created] (AVRO-3012) Unknown logical types are not ignored during deserialization but lead to an exception.

["Lucas Heimberg (Jira)" <ji...@apache.org>]

Lucas Heimberg created AVRO-3012:
------------------------------------

             Summary: Unknown logical types are not ignored during deserialization but lead to an exception.
                 Key: AVRO-3012
                 URL: https://issues.apache.org/jira/browse/AVRO-3012
             Project: Apache Avro
          Issue Type: Bug
          Components: csharp
            Reporter: Lucas Heimberg


By specification (see [https://avro.apache.org/docs/current/spec.html#Logical+Types] ), unknown logical types and logical types with invalid specification should be ignored during deserialization and instead, the object of the underlying Avro type should be returned.

The C# implementation of Avro however raises an exception when such a logical type is found during parsing of a schema string. Therefore, it is not possible to parse a schema with a logical type using a version of the C# implementation of Avro that does not supports this logical type yet.

In particular, it is therefore as an example not possible to consume a binary encoded Avro datum that makes use of the logical type UUID (as available in Avro 1.10.1) with a deserializer using C# Avro 1.9.2. (by specification, the deserializer should fall back to the underlying string representation of the UUID).

This severly limits the downwards compability promised by the Avro specification.

 

During schema parsing, a LogicalSchema instance is created using the GetFromLogicalSchema method of the LogicalTypeFactory. This method has an optional parameter to ignore invalid logical  types, which is not used by the caller.

It is not completely obvious to me, whether an unsupported logical type should still be parsed as a LogicalSchema, although without a LogicalType, or whether it should be directly parsed as the underlying Avro type (in the implementation also called BaseSchema) - i.e., whether the downgrading to the underlying Avro type should happen already during parsing or later during deserialization using the schema. 

For the second case, a fix to the problem would also require an update to the ReadLogical method of the GenericReader class in order to support the case that the LogicalType of the writer schema is null.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)