[jira] [Commented] (AVRO-2485) ClassCastException on Nullable Map Value Schemas with Backward Compatible Changes (Field Addition)

["Nandor Kollar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892879#comment-16892879 ] 

Nandor Kollar commented on AVRO-2485:
-------------------------------------

[~wisegas] I think this is a know issue (see AVRO-2400 and AVRO-2415), which is fixed on master, and will be released in 1.9.1.

> ClassCastException on Nullable Map Value Schemas with Backward Compatible Changes (Field Addition)
> --------------------------------------------------------------------------------------------------
>
>                 Key: AVRO-2485
>                 URL: https://issues.apache.org/jira/browse/AVRO-2485
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.9.0
>         Environment: Operating System: Mac OS 10.13.6
> JDK version: 1.8_172
> JUnit version: 4.12
>            Reporter: Sage Pierce
>            Priority: Critical
>
> In one of our Java applications, upon upgrading from {{org.apache.avro:avro:1.8.2}} to {{org.apache.avro:avro:1.9.0}}, we are seeing deserialization failures with the following exception:
> {code:java}
> java.lang.ClassCastException: org.apache.avro.Resolver$ReaderUnion cannot be cast to org.apache.avro.Resolver$Container
> 	at org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:99)
> 	at org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:110)
> 	at org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:141)
> 	at org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:65)
> 	at org.apache.avro.io.ResolvingDecoder.resolve(ResolvingDecoder.java:85)
> 	at org.apache.avro.io.ResolvingDecoder.<init>(ResolvingDecoder.java:50)
> 	at org.apache.avro.io.DecoderFactory.resolvingDecoder(DecoderFactory.java:297)
> 	at org.apache.avro.generic.GenericDatumReader.getResolver(GenericDatumReader.java:128)
> 	at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
> {code}
> Further inspection of the stack trace and problematic data shows the issue lies with nullable (unioned) Map Schemas, whose values are complex types, and those types having backward compatible changes between the Writer and Reader Schemas. The following test exposes the issue:
> {code:java|title=AvroUpgradeTest.java}
> import java.io.ByteArrayOutputStream;
> import java.util.Collections;
> import org.apache.avro.Schema;
> import org.apache.avro.io.DecoderFactory;
> import org.apache.avro.io.EncoderFactory;
> import org.apache.avro.reflect.ReflectData;
> import org.apache.avro.reflect.ReflectDatumWriter;
> import org.junit.Test;
> import static org.junit.Assert.assertEquals;
> public class AvroUpgradeTest {
>     @Test
>     public void nullableMapWithBackwardCompatibleValueChangesCanBeDeserialized() throws Exception {
>     TestDataWithNullableMap data = new TestDataWithNullableMap();
>     data.setMap(Collections.singletonMap("KEY", TestData.create()));
>     Schema writerSchema = ReflectData.get().getSchema(TestDataWithNullableMap.class);
>     Schema readerMapSchema = Schema.createMap(Schema.createRecord(TestData.class.getName(), null, null, false,
>         Collections.singletonList(new Schema.Field("data1", Schema.create(Schema.Type.STRING), null, Object.class.cast(null)))));
>     Schema readerSchema = Schema.createRecord(TestDataWithNullableMap.class.getName(), null, null, false,
>         Collections.singletonList(new Schema.Field("map", ReflectData.makeNullable(readerMapSchema), null, Object.class.cast(null))));
>     ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
>     new ReflectDatumWriter<>(writerSchema, ReflectData.get()).write(data, EncoderFactory.get().directBinaryEncoder(outputStream, null));
>     byte[] serialized = outputStream.toByteArray();
>     TestDataWithNullableMap deserialized = new ReflectDecoderAvroDeserializer.ReflectDecoderDatumReader<TestDataWithNullableMap>(writerSchema, readerSchema, ReflectData.get())
>         .read(null, DecoderFactory.get().binaryDecoder(serialized, 0, serialized.length, null));
>     assertEquals(data.getMap().get("KEY").getData1(), deserialized.getMap().get("KEY").getData1());
>     }
> }
> {code}
> Note: I'm having to do a some manual Schema generation to force the appearance of the Writer's Schema making a (backward compatible) field addition.
> Relevant POJOs are as follows:
> {code:java|title=TestDataWithNullableMap.java}
> import java.util.Map;
> import org.apache.avro.reflect.Nullable;
> public class TestDataWithNullableMap {
>     @Nullable
>     private Map<String, TestData> map;
>     public Map<String, TestData> getMap() {
>         return map;
>     }
>     public void setMap(Map<String, TestData> map) {
>         this.map = map;
>     }
> }
> {code}
> —
> {code:java|title=TestData.java}
> public class TestData {
>     private String data1;
>     private String data2;
>     public static TestData create() {
>         TestData data = new TestData();
>         data.setData1("DATA 1");
>         data.setData2("DATA 2");
>         return data;
>     }
>     public String getData1() {
>         return data1;
>     }
>     public void setData1(String data1) {
>         this.data1 = data1;
>     }
>     public String getData2() {
>         return data2;
>     }
>     public void setData2(String data2) {
>         this.data2 = data2;
>     }
> }
> {code}
>  
> The preceding test succeeds with Avro 1.8.2, but fails on Avro 1.9.0 with the mentioned Exception.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)