[jira] [Created] (AVRO-3969) [C#] Serializing a Map/Dictionary using SpecificDefaultWriter and JsonEncoder fails with AvroTypeException

["Dave Kliczbor (Jira)" <ji...@apache.org>]

Dave Kliczbor created AVRO-3969:
-----------------------------------

             Summary: [C#] Serializing a Map/Dictionary using SpecificDefaultWriter and JsonEncoder fails with AvroTypeException
                 Key: AVRO-3969
                 URL: https://issues.apache.org/jira/browse/AVRO-3969
             Project: Apache Avro
          Issue Type: Bug
          Components: csharp
    Affects Versions: 1.11.3
         Environment: We believe the environment to not matter here, but as there is a box for that here... :)
 * Windows 11
 * .NET 8.0
 * C# 12
 * Apache.Avro 1.11.3 (released via nuget)
            Reporter: Dave Kliczbor


Hello,

We think we identified a bug in the SpecificDefaultWriter implementation of the Apache.Avro C# library (as of Release 1.11.3, also main branch as of 2024-04-03).
h4. Reproduction of the problem:

The following C# code throws an {{{}AvroTypeException("Attempt to process an array-start when a map-start was expected"){}}}:
{code:java}
  var schema = MapSchema.CreateMap(PrimitiveSchema.Create(Schema.Type.String));
  var ms = new MemoryStream();
  Encoder enc = new JsonEncoder(schema, ms, false);
  var writer = new SpecificDefaultWriter(schema); 
  writer.Write(new Dictionary<string, string>(), enc);
  enc.Flush();{code}
 

We expect this code to not throw up.

The same code with {{BinaryEncoder(schema, ms)}} instead of {{JsonEncoder(schema, ms, false)}} works as expected: we get a Avro message byte array without an Exception being thrown.
h4. Preliminary Solution Analysis:

We believe the root cause to be in this 13 years old code:

[https://github.com/apache/avro/blob/8cb32f05d50382e5a5e97d6d45b080bef0c0e6a5/lang/csharp/src/apache/main/Specific/SpecificWriter.cs#L152]
{code:java}
        protected override void WriteMap(MapSchema schema, object value, Encoder encoder)
        {
            var map = value as System.Collections.IDictionary;
            if (map == null)
                throw new AvroTypeException("Map does not implement non-generic IDictionary");
            encoder.WriteArrayStart();  // <----- This is L.152
            encoder.SetItemCount(map.Count);
            foreach (System.Collections.DictionaryEntry de in map)
            {
                encoder.StartItem();
                encoder.WriteString(de.Key as string);
                Write(schema.ValueSchema, de.Value, encoder);
            }
            encoder.WriteMapEnd();
        } {code}
In method {{{}SpecificDefaultWriter.WriteMap(MapSchema, object, Encoder){}}}, the call {{encoder.WriteArrayStart()}} is used. Which is in stark contrast to the usage of {{encoder.WriteMapEnd()}} later and also matches the Exception message above (using the BinaryEncoder works without problems, as {{WriteArrayStart}} and {{WriteMapStart }}are identical empty methods there).

Therefore, we believe that {{encoder.WriteMapStart();}} is the intended call in L.152.

 
h4. Additional information:

Here’s our test case (code using xUnit/FluentAssertions and Apache.Avro 1.11.3) that fails with the bug present:

 
{code:java}
[Fact]
public void AvroLibraryJsonEncoderDoesNotChokeOnMaps()
{
    Action action = () =>
    {
        var schema = MapSchema.CreateMap(PrimitiveSchema.Create(Schema.Type.String));
        var ms = new MemoryStream();
        Encoder enc = new JsonEncoder(schema, ms, false);
        var writer = new SpecificDefaultWriter(schema);
        writer.Write(new Dictionary<string, string>(), enc);
        enc.Flush();
    };
    action.Should().NotThrow<AvroTypeException>();
}{code}
 

Our workaround is to use this derived SpecificDefaultWriter class instead of the original from the Apache.Avro C# library. This works for our purposes, but up to now, we are mere users of the library and do not know yet whether this might break something else.
{code:java}
public class SpecificDefaultWriterWithWriteMapToJsonBugfix(Schema schema) : SpecificDefaultWriter(schema) {
    protected override void WriteMap(MapSchema schema, object value, Encoder encoder)
    {
        if (!(value is IDictionary dictionary))
            throw new AvroTypeException("Map does not implement non-generic IDictionary");
        encoder.WriteMapStart();
        encoder.SetItemCount((long)dictionary.Count);
        foreach (DictionaryEntry dictionaryEntry in dictionary)
        {
            encoder.StartItem();
            encoder.WriteString(dictionaryEntry.Key as string);
            this.Write(schema.ValueSchema, dictionaryEntry.Value, encoder);
        }
        encoder.WriteMapEnd();
    }
} 
{code}
Cheers!

 Dave Kliczbor (Materna SE)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)