You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Martin Mucha (JIRA)" <ji...@apache.org> on 2019/07/30 11:15:00 UTC
[jira] [Created] (AVRO-2490) Unable to add field to existing avro
schema
Martin Mucha created AVRO-2490:
----------------------------------
Summary: Unable to add field to existing avro schema
Key: AVRO-2490
URL: https://issues.apache.org/jira/browse/AVRO-2490
Project: Apache Avro
Issue Type: Bug
Reporter: Martin Mucha
Fix For: 1.8.2, 1.9.0
As described in
[https://docs.oracle.com/database/nosql-12.1.3.4/GettingStartedGuide/schemaevolution.html#changeschema-rules]
I should be able to extend AVRO schema by adding optional and(or?) default-having field. If I do that though, deserialization fails for me. It happens for version 1.8.2 and 1.9.0 as well.
Some cases:
A) If I add new field defined as:
{code:java}
{ "name": "newColumn", "type": ["null","string"], "default": null}{code}
deserialization will fail with:
{code:java}
Caused by: java.lang.ArrayIndexOutOfBoundsException: 5 at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:424) at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290) at org.apache.avro.io.parsing.Parser.advance(Parser.java:88) at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267) at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232) at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222) at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232) at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222) at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145) at tech.allegro.schema.json2avro.converter.JsonAvroConverter.convertToJson(JsonAvroConverter.java:83){code}
if I add new field defined as:
{code:java}
{ "name": "newColumn", "type": "string", "default": ""}{code}
{{or}}
{code:java}
{ "name": "newColumn", "type": "string", "default": "incorrect"}{code}
(assuming I will fix it later in app)
deserialization will fail with:
{code:java}
Caused by: org.apache.avro.AvroRuntimeException: Malformed data. Length is negative: -1 at org.apache.avro.io.BinaryDecoder.doReadBytes(BinaryDecoder.java:336) at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:263) at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201) at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:422) at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:414) at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:181) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232) at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222) at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232) at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222) at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145) at tech.allegro.schema.json2avro.converter.JsonAvroConverter.convertToJson(JsonAvroConverter.java:83){code}
{{what code does fail like that(that ± content of method convertToJson in JsonAvroConverter as seen in stacktrace.):}}
{code:java}
BinaryDecoder binaryDecoder = DecoderFactory.get().binaryDecoder(avro, (BinaryDecoder)null);
GenericRecord record = (GenericRecord)(new GenericDatumReader(schema)).read((Object)null, binaryDecoder);{code}
the second line produces given errors. Without new column added, deserialization works.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)