[jira] [Comment Edited] (AVRO-3027) .net consumer Backward compatibility issue.

["Yevhen Cherkes (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-3027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17289942#comment-17289942 ] 

Yevhen Cherkes edited comment on AVRO-3027 at 2/25/21, 10:58 AM:
-----------------------------------------------------------------

See the solution in  [PR#1103|https://github.com/apache/avro/pull/1103]?


was (Author: ycherkes):
Hi, I have created a [PR1103|https://github.com/apache/avro/pull/1103], could someone please review it?

>  .net consumer Backward compatibility issue.
> --------------------------------------------
>
>                 Key: AVRO-3027
>                 URL: https://issues.apache.org/jira/browse/AVRO-3027
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: csharp
>    Affects Versions: 1.10.1
>         Environment: .Net core 3.1
>            Reporter: Yevhen Cherkes
>            Priority: Major
>         Attachments: CompatibilityTests.zip
>
>
> We are facing the following exception on the .net core consumer's side:
>  (*Java*-based consumers work with no issues for this particular case.)
> {code:java}
> No matching schema for {"type":"record","name":"InnerType1","namespace":"com.mycorp.mynamespace","fields":[
> {"name":"value","type":"string"}
> ]} in {"type":["null",{"type":"record","name":"InnerType2","namespace":"com.mycorp.mynamespace","fields":[
> {"name":"value","type":"string"}
> ]}]}
> at Avro.Generic.DefaultReader.findBranch(UnionSchema us, Schema s)
>  at Avro.Generic.DefaultReader.ReadUnion(Object reuse, UnionSchema writerSchema, Schema readerSchema, Decoder d)
>  at Avro.Generic.DefaultReader.Read(Object reuse, Schema writerSchema, Schema readerSchema, Decoder d)
>  at Avro.Specific.SpecificDefaultReader.ReadRecord(Object reuse, RecordSchema writerSchema, Schema readerSchema, Decoder dec)
> {code}
> We have 2 versions of the schema on a topic:
> Schema v1:
> {code:json}
> {
>     "fields": [
>       {
>         "name": "InnerComplexValue",
>         "type": [
>           "null",
>           {
> 			"fields": [
> 			  {
> 				"name": "value",
> 				"type": "string"
> 			  }
> 			],
> 			"name": "InnerType1",
> 			"namespace": "com.mycorp.mynamespace",
> 			"type": "record"
> 		  }
>         ]
>       }
>     ],
>     "name": "RootType",
>     "namespace": "com.mycorp.mynamespace",
>     "type": "record"
>   }
> {code}
>  Schema v2:
> {code:json}
> {
>     "fields": [
>       {
>         "name": "InnerComplexValue",
>         "type": [
>           "null",
>           {
> 			"fields": [
> 			  {
> 				"name": "value",
> 				"type": "string"
> 			  }
> 			],
> 			"name": "InnerType2",
> 			"namespace": "com.mycorp.mynamespace",
> 			"type": "record"
> 		  }
>         ]
>       }
>     ],
>     "name": "RootType",
>     "namespace": "com.mycorp.mynamespace",
>     "type": "record"
>   }
> {code}
>  InnerType1 -> InnerType2 is the only change.
> The schema for a topic is configured with *Backward* compatibility.
> An updated version of a Schema is saved with no errors.
> Then we generated a specific record RootType with avrogen tool.
> Reproduction code:
>   
> {code:c#}
> var base64 = "AhR0ZXN0IHZhbHVl";
> using var stream = new MemoryStream(Convert.FromBase64String(base64));
> var schema_v1 = Schema.Parse("{\"type\":\"record\",\"name\":\"RootType\",\"namespace\":\"com.mycorp.mynamespace\",\"fields\":" +
>                                                  "[{\"name\":\"InnerComplexValue\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"InnerType1\"" +
>                                                  ",\"fields\":[{\"name\":\"value\",\"type\":\"string\"}]}]}]}");
> var schema_v2 = Schema.Parse("{\"type\":\"record\",\"name\":\"RootType\",\"namespace\":\"com.mycorp.mynamespace\",\"fields\":" +
>                                          "[{\"name\":\"InnerComplexValue\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"InnerType2\"" +
>                                          ",\"fields\":[{\"name\":\"value\",\"type\":\"string\"}]}]}]}");
> var specificRecord = Deserialize<RootType>(stream, schema_v1, schema_v2);
> {code}
> Full sources see in attachments.
> Method Deserialize was taken from the [SpecificTests.cs#L408-L417|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/test/Specific/SpecificTests.cs#L408-L417]
>  
> Probably, the reason is in a different Schema resolution code:
>  java: [https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/Resolver.java#L647-L659]
>  .net: [https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/RecordSchema.cs#L310-L312]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)