You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Ryan Skraba (Jira)" <ji...@apache.org> on 2020/10/14 11:10:00 UTC
[jira] [Resolved] (AVRO-2817) Avro file generated using avro-1.8.2
is not readable from avro-1.9.2 for certian type of schemas
[ https://issues.apache.org/jira/browse/AVRO-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan Skraba resolved AVRO-2817.
-------------------------------
Resolution: Fixed
> Avro file generated using avro-1.8.2 is not readable from avro-1.9.2 for certian type of schemas
> ------------------------------------------------------------------------------------------------------
>
> Key: AVRO-2817
> URL: https://issues.apache.org/jira/browse/AVRO-2817
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.9.2
> Reporter: Manoj Kumar Jha
> Assignee: Ryan Skraba
> Priority: Blocker
> Fix For: 1.10.1
>
> Attachments: records.avro
>
>
> Hello,
> We observed avro file generated using avro-1.8.2 is not readable while using avro-1.9.2 if schema contains a field "default:null". Please see below.
>
> --------------------------------------------------------------------------
> [mxj142:Jars]$ ls -ltr
> total 168520
> -rwx------@ 1 mxj142 staff 34798932 Jan 16 14:45 avro-tools-1.8.2.jar
> -rwxr-xr-x@ 1 mxj142 staff 51303364 Mar 6 17:43 avro-tools-1.9.2.jar
> -rw-r--r-- 1 mxj142 staff 715 Apr 24 11:19 records.avro
> [mxj142:Jars]$ java -jar avro-tools-1.8.2.jar tojson records.avro
> log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
> {"enrichmentHeader":\{"correlationId":"1","sourceId":"09b5d770-7373-462d-9517-8e5ce957793f"},"cx":\{"dateFirst":"20200419","dateLast":"20200419"}}
> {"enrichmentHeader":\{"correlationId":"2","sourceId":"0285dc49-cc5c-4717-8ea2-45e50de2f65f"},"cx":\{"dateFirst":"20200420","dateLast":"20200420"}}
> [mxj142:Jars]$
> [mxj142:Jars]$
> [mxj142:Jars]$
> [mxj142:Jars]$
> [mxj142:Jars]$
> [mxj142:Jars]$ java -jar avro-tools-1.9.2.jar tojson records.avro
> 20/04/24 11:20:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> Exception in thread "main" org.apache.avro.AvroTypeException: Invalid default for field enrichmentHeader: null not a \{"type":"record","name":"EnrichmentHeader","namespace":"com.mxj142.commons.domain","fields":[{"name":"correlationId","type":{"type":"string","avro.java.string":"String"}},\{"name":"sourceId","type":{"type":"string","avro.java.string":"String"}}]}
> at org.apache.avro.Schema.validateDefault(Schema.java:1540)
> at org.apache.avro.Schema.access$500(Schema.java:87)
> at org.apache.avro.Schema$Field.<init>(Schema.java:521)
> at org.apache.avro.Schema.parse(Schema.java:1647)
> at org.apache.avro.Schema$Parser.parse(Schema.java:1394)
> at org.apache.avro.Schema$Parser.parse(Schema.java:1382)
> at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:130)
> at org.apache.avro.file.DataFileStream.<init>(DataFileStream.java:90)
> at org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:93)
> at org.apache.avro.tool.Main.run(Main.java:66)
> at org.apache.avro.tool.Main.main(Main.java:55)
> [mxj142:Jars]$
> --------------------------------------------------------------------------
>
> The file "records.avro" is generated using avro-1.8.2 . It contains two records and using "avro-tools-1.8.2.jar" one can read it. If we use "avro-tools-1.9.2.jar", then "records.avro" is not readable (as shown above). The file "records.avro" is also attached for your convenience.
>
> Let me know if you need any more information.
>
> Thanks,
> Manoj
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)