[jira] [Commented] (AVRO-3027) .net consumer Backward compatibility issue.

["Kalle Niemitalo (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-3027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585339#comment-17585339 ] 

Kalle Niemitalo commented on AVRO-3027:
---------------------------------------

[Schema Resolution|https://avro.apache.org/docs/1.11.1/specification/#schema-resolution] says "both schemas are records with the same (unqualified) name", but here the names differ, so I don't see why schema v1 and v2 would match.

Can you solve this with [Aliases|https://avro.apache.org/docs/1.11.1/specification/#aliases], instead? I mean by adding {{"aliases": ["InnerType1"]}} to schema v2.

>  .net consumer Backward compatibility issue.
> --------------------------------------------
>
>                 Key: AVRO-3027
>                 URL: https://issues.apache.org/jira/browse/AVRO-3027
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: csharp, java
>    Affects Versions: 1.10.1
>         Environment: .Net core 3.1
>            Reporter: Yevhen Cherkes
>            Priority: Critical
>              Labels: pull-request-available
>         Attachments: CompatibilityTests.zip
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> We are facing the following exception on the .net core consumer's side:
>  (*Java*-based consumers work with no issues for this particular case.)
> {code:java}
> No matching schema for {"type":"record","name":"InnerType1","namespace":"com.mycorp.mynamespace","fields":[
> {"name":"value","type":"string"}
> ]} in {"type":["null",{"type":"record","name":"InnerType2","namespace":"com.mycorp.mynamespace","fields":[
> {"name":"value","type":"string"}
> ]}]}
> at Avro.Generic.DefaultReader.findBranch(UnionSchema us, Schema s)
>  at Avro.Generic.DefaultReader.ReadUnion(Object reuse, UnionSchema writerSchema, Schema readerSchema, Decoder d)
>  at Avro.Generic.DefaultReader.Read(Object reuse, Schema writerSchema, Schema readerSchema, Decoder d)
>  at Avro.Specific.SpecificDefaultReader.ReadRecord(Object reuse, RecordSchema writerSchema, Schema readerSchema, Decoder dec)
> {code}
> We have 2 versions of the schema on a topic:
> Schema v1:
> {code:json}
> {
>     "fields": [
>       {
>         "name": "InnerComplexValue",
>         "type": [
>           "null",
>           {
> 			"fields": [
> 			  {
> 				"name": "value",
> 				"type": "string"
> 			  }
> 			],
> 			"name": "InnerType1",
> 			"namespace": "com.mycorp.mynamespace",
> 			"type": "record"
> 		  }
>         ]
>       }
>     ],
>     "name": "RootType",
>     "namespace": "com.mycorp.mynamespace",
>     "type": "record"
>   }
> {code}
>  Schema v2:
> {code:json}
> {
>     "fields": [
>       {
>         "name": "InnerComplexValue",
>         "type": [
>           "null",
>           {
> 			"fields": [
> 			  {
> 				"name": "value",
> 				"type": "string"
> 			  }
> 			],
> 			"name": "InnerType2",
> 			"namespace": "com.mycorp.mynamespace",
> 			"type": "record"
> 		  }
>         ]
>       }
>     ],
>     "name": "RootType",
>     "namespace": "com.mycorp.mynamespace",
>     "type": "record"
>   }
> {code}
>  InnerType1 -> InnerType2 is the only change.
> The schema for a topic is configured with *Backward* compatibility.
> An updated version of a Schema is saved with no errors.
> Then we generated a specific record RootType with avrogen tool.
> Reproduction code:
>   
> {code:c#}
> var base64 = "AhR0ZXN0IHZhbHVl";
> using var stream = new MemoryStream(Convert.FromBase64String(base64));
> var schema_v1 = Schema.Parse("{\"type\":\"record\",\"name\":\"RootType\",\"namespace\":\"com.mycorp.mynamespace\",\"fields\":" +
>                                                  "[{\"name\":\"InnerComplexValue\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"InnerType1\"" +
>                                                  ",\"fields\":[{\"name\":\"value\",\"type\":\"string\"}]}]}]}");
> var schema_v2 = Schema.Parse("{\"type\":\"record\",\"name\":\"RootType\",\"namespace\":\"com.mycorp.mynamespace\",\"fields\":" +
>                                          "[{\"name\":\"InnerComplexValue\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"InnerType2\"" +
>                                          ",\"fields\":[{\"name\":\"value\",\"type\":\"string\"}]}]}]}");
> var specificRecord = Deserialize<RootType>(stream, schema_v1, schema_v2);
> {code}
> Full sources see in attachments.
> Method Deserialize was taken from the [SpecificTests.cs#L408-L417|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/test/Specific/SpecificTests.cs#L408-L417]
>  
> Probably, the reason is in a different Schema resolution code:
>  java: [https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/Resolver.java#L647-L659]
>  .net: [https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/RecordSchema.cs#L310-L312]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)