You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/12/11 18:11:00 UTC
[jira] [Commented] (AVRO-2184) Unable to decode JSON data file if a
property is renamed in reader schema
[ https://issues.apache.org/jira/browse/AVRO-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16717678#comment-16717678 ]
ASF subversion and git services commented on AVRO-2184:
-------------------------------------------------------
Commit 595643cba16a2b4d7be68d46ee8f79c4e380cbf7 in avro's branch refs/heads/master from nandorKollar
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=595643c ]
AVRO-2184: Unable to decode JSON data file if a property is renamed in reader schema (#316)
* AVRO-2184: Unable to decode JSON data file if a property is renamed in reader schema
JsonDecoder doesn't honor aliases
* No need to wrap aliases to unmodifiableSet, since getter Schema#aliases already does it
* Remove unused import to pass Checkstyle check
> Unable to decode JSON data file if a property is renamed in reader schema
> -------------------------------------------------------------------------
>
> Key: AVRO-2184
> URL: https://issues.apache.org/jira/browse/AVRO-2184
> Project: Apache Avro
> Issue Type: Bug
> Reporter: Prateek Kohli
> Assignee: Nandor Kollar
> Priority: Major
> Attachments: TestAliasesInSchemaEvolution.java
>
>
> I am unable to decode JSON data file if a property is renamed in reader schema:
> As per the documentation it is a compatible change.
> Also, Datatype promotion is not being supported, if I try to change the datatype of favourite_number field in the writer's schema, decoding fails.
> All of the above scenarios are supported if I use Binary decoding instead of JSON.
> *Writer Schema :*
> {"namespace": "example.avro",
> "type": "record",
> "name": "User",
> "fields": [
> \{"name": "name", "type": "string"},
> \{"name": "favorite_number", "type": ["int", "null"]},
> \{"name": "favorite_color", "type": ["string", "null"]}
> ]}
>
> *Reader Schema :*
> {"namespace": "example.avro",
> "type": "record",
> "name": "User",
> "fields": [
> \{"name": "fname", "type": "string", "aliases" : [ "name" ]},
> \{"name": "favorite_number", "type": ["int", "null"]},
> \{"name": "favorite_color", "type": ["string", "null"]}
> ]}
>
> *I have written the below code to decode JSON data:*
> FileInputStream fin = new FileInputStream(file);
> byte fileContent[] = new byte[(int)file.length()];
> fin.read(fileContent);
> InputStream input = new ByteArrayInputStream(fileContent);
> DataInputStream din = new DataInputStream(input);
>
> while (true) {
> try {
> Decoder decoder = DecoderFactory.get().jsonDecoder(schema, din);
> ResolvingDecoder resolvingDecoder = DecoderFactory.get().resolvingDecoder(writer, reader, decoder);
> Object datum = datumReader.read(null, resolvingDecoder);
> System.out.println(datum);
> } catch (EOFException eofException) {
> break;
> }
> }
> *Below is the Exception I get :*
> Exception in thread "main" org.apache.avro.AvroTypeException: Found example.avro.User, expecting example.avro.User, missing required field fname
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:292)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:196)
> at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201)
> at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:422)
> at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:414)
> at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:181)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
> at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232)
> at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222)
> at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
> at com.ericsson.avroTest.avroCheck.WithoutCodeTest.main(WithoutCodeTest.java:134)
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)