[jira] [Updated] (AVRO-3969) [C#] Serializing a Map/Dictionary using SpecificDefaultWriter and JsonEncoder fails with AvroTypeException

["Dave Kliczbor (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AVRO-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dave Kliczbor updated AVRO-3969:
--------------------------------
    Language: C#
      Labels: easyfix  (was: )

> [C#] Serializing a Map/Dictionary using SpecificDefaultWriter and JsonEncoder fails with AvroTypeException
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: AVRO-3969
>                 URL: https://issues.apache.org/jira/browse/AVRO-3969
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: csharp
>    Affects Versions: 1.11.3
>         Environment: We believe the environment to not matter here, but as there is a box for that here... :)
>  * Windows 11
>  * .NET 8.0
>  * C# 12
>  * Apache.Avro 1.11.3 (released via nuget)
>            Reporter: Dave Kliczbor
>            Priority: Major
>              Labels: easyfix
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> Hello,
> We think we identified a bug in the SpecificDefaultWriter implementation of the Apache.Avro C# library (as of Release 1.11.3, also main branch as of 2024-04-03).
> h4. Reproduction of the problem:
> The following C# code throws an {{{}AvroTypeException("Attempt to process an array-start when a map-start was expected"){}}}:
> {code:java}
>   var schema = MapSchema.CreateMap(PrimitiveSchema.Create(Schema.Type.String));
>   var ms = new MemoryStream();
>   Encoder enc = new JsonEncoder(schema, ms, false);
>   var writer = new SpecificDefaultWriter(schema); 
>   writer.Write(new Dictionary<string, string>(), enc);
>   enc.Flush();{code}
>  
> We expect this code to not throw up.
> The same code with {{BinaryEncoder(schema, ms)}} instead of {{JsonEncoder(schema, ms, false)}} works as expected: we get a Avro message byte array without an Exception being thrown.
> h4. Preliminary Solution Analysis:
> We believe the root cause to be in this 13 years old code:
> [https://github.com/apache/avro/blob/8cb32f05d50382e5a5e97d6d45b080bef0c0e6a5/lang/csharp/src/apache/main/Specific/SpecificWriter.cs#L152]
> {code:java}
>         protected override void WriteMap(MapSchema schema, object value, Encoder encoder)
>         {
>             var map = value as System.Collections.IDictionary;
>             if (map == null)
>                 throw new AvroTypeException("Map does not implement non-generic IDictionary");
>             encoder.WriteArrayStart();  // <----- This is L.152
>             encoder.SetItemCount(map.Count);
>             foreach (System.Collections.DictionaryEntry de in map)
>             {
>                 encoder.StartItem();
>                 encoder.WriteString(de.Key as string);
>                 Write(schema.ValueSchema, de.Value, encoder);
>             }
>             encoder.WriteMapEnd();
>         } {code}
> In method {{{}SpecificDefaultWriter.WriteMap(MapSchema, object, Encoder){}}}, the call {{encoder.WriteArrayStart()}} is used. Which is in stark contrast to the usage of {{encoder.WriteMapEnd()}} later and also matches the Exception message above (using the BinaryEncoder works without problems, as {{WriteArrayStart}} and {{WriteMapStart }}are identical empty methods there).
> Therefore, we believe that {{encoder.WriteMapStart();}} is the intended call in L.152.
>  
> h4. Additional information:
> Here’s our test case (code using xUnit/FluentAssertions and Apache.Avro 1.11.3) that fails with the bug present:
>  
> {code:java}
> [Fact]
> public void AvroLibraryJsonEncoderDoesNotChokeOnMaps()
> {
>     Action action = () =>
>     {
>         var schema = MapSchema.CreateMap(PrimitiveSchema.Create(Schema.Type.String));
>         var ms = new MemoryStream();
>         Encoder enc = new JsonEncoder(schema, ms, false);
>         var writer = new SpecificDefaultWriter(schema);
>         writer.Write(new Dictionary<string, string>(), enc);
>         enc.Flush();
>     };
>     action.Should().NotThrow<AvroTypeException>();
> }{code}
>  
> Our workaround is to use this derived SpecificDefaultWriter class instead of the original from the Apache.Avro C# library. This works for our purposes, but up to now, we are mere users of the library and do not know yet whether this might break something else.
> {code:java}
> public class SpecificDefaultWriterWithWriteMapToJsonBugfix(Schema schema) : SpecificDefaultWriter(schema) {
>     protected override void WriteMap(MapSchema schema, object value, Encoder encoder)
>     {
>         if (!(value is IDictionary dictionary))
>             throw new AvroTypeException("Map does not implement non-generic IDictionary");
>         encoder.WriteMapStart();
>         encoder.SetItemCount((long)dictionary.Count);
>         foreach (DictionaryEntry dictionaryEntry in dictionary)
>         {
>             encoder.StartItem();
>             encoder.WriteString(dictionaryEntry.Key as string);
>             this.Write(schema.ValueSchema, dictionaryEntry.Value, encoder);
>         }
>         encoder.WriteMapEnd();
>     }
> } 
> {code}
> Cheers!
>  Dave Kliczbor (Materna SE)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)