[jira] [Commented] (AVRO-2184) Unable to decode JSON data file if a property is renamed in reader schema

["Nandor Kollar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497974#comment-16497974 ] 

Nandor Kollar commented on AVRO-2184:
-------------------------------------

This looks like a bug for me, I attached a simple test case (based on {{TestReadingWritingDataInEvolvedSchemas}}, which only tests binary encoding) to demonstrate the problem. [~prateekkohli2112] is the attached test case demonstrate your issue correctly? Do you have a fix for it? Contributions are welcome, if you do, I'm happy to review and commit it!

> Unable to decode JSON data file if a property is renamed in reader schema
> -------------------------------------------------------------------------
>
>                 Key: AVRO-2184
>                 URL: https://issues.apache.org/jira/browse/AVRO-2184
>             Project: Avro
>          Issue Type: Bug
>            Reporter: Prateek Kohli
>            Priority: Major
>
> I am unable to decode JSON data file if a property is renamed in reader schema:
> As per the documentation it is a compatible change.
> Also, Datatype promotion is not being supported, if I try to change the datatype of favourite_number field in the writer's schema, decoding fails.
> All of the above scenarios are supported if I use Binary decoding instead of JSON.
> *Writer Schema :*
> {"namespace": "example.avro",
>  "type": "record",
>  "name": "User",
>  "fields": [
>  \{"name": "name", "type": "string"},
>  \{"name": "favorite_number", "type": ["int", "null"]},
>  \{"name": "favorite_color", "type": ["string", "null"]}
>  ]}
>  
> *Reader Schema :* 
> {"namespace": "example.avro",
>  "type": "record",
>  "name": "User",
>  "fields": [
>  \{"name": "fname", "type": "string", "aliases" : [ "name" ]},
>  \{"name": "favorite_number", "type": ["int", "null"]},
>  \{"name": "favorite_color", "type": ["string", "null"]}
>  ]}
>  
> *I have written the below code to decode JSON data:*
> FileInputStream fin = new FileInputStream(file);
>  byte fileContent[] = new byte[(int)file.length()];
>  fin.read(fileContent);
>  InputStream input = new ByteArrayInputStream(fileContent);
>  DataInputStream din = new DataInputStream(input);
>  
>  while (true) {
>  try {
>          Decoder decoder = DecoderFactory.get().jsonDecoder(schema, din);
>          ResolvingDecoder resolvingDecoder = DecoderFactory.get().resolvingDecoder(writer,                             reader, decoder);
>          Object datum = datumReader.read(null, resolvingDecoder);
>          System.out.println(datum);
>      } catch (EOFException eofException) {
>           break;
>        }
>  }
> *Below is the Exception I get :*
> Exception in thread "main" org.apache.avro.AvroTypeException: Found example.avro.User, expecting example.avro.User, missing required field fname
>  at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:292)
>  at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>  at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:196)
>  at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201)
>  at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:422)
>  at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:414)
>  at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:181)
>  at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
>  at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232)
>  at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222)
>  at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
>  at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
>  at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
>  at com.ericsson.avroTest.avroCheck.WithoutCodeTest.main(WithoutCodeTest.java:134)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)