You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Sam Schlegel (JIRA)" <ji...@apache.org> on 2017/07/27 01:03:00 UTC
[jira] [Commented] (AVRO-1855) Avro-mapred not evaluating map
schema correctly when values are expected to be strings
[ https://issues.apache.org/jira/browse/AVRO-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102544#comment-16102544 ]
Sam Schlegel commented on AVRO-1855:
------------------------------------
I believe this related to AVRO-966 and is caused by the {{isArray}} check coming before {{isMap}} in {{org.apache.avro.generic.GenericData#getSchemaName}}. I'm not sure why, but {{isArray(datum)}} is returning true when {{datum}} is a `java.util.HashMap`, even though {{java.util.HashMap}} is not an instance of {{java.util.Collection}} and {{datum instanceof Collection}} returns false when evaluated in the debugger.
> Avro-mapred not evaluating map schema correctly when values are expected to be strings
> --------------------------------------------------------------------------------------
>
> Key: AVRO-1855
> URL: https://issues.apache.org/jira/browse/AVRO-1855
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.8.0
> Reporter: Mikko Kupsu
> Priority: Critical
> Attachments: 20160530_AVRO-1855.patch
>
>
> When reading bunch of Avro file and concatenating them using avro-mapred, there is an issue with following schema definition line:
> {code}
> {"name": "headers", "type": ["null", {"type": "map", "values": "string"}]},
> {code}
> Below exceptions are thrown:
> {code}
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union ["null",{"type":"map","values":"string"}]: {range=bytes=91553252-91557347, accept=*/*, response_status_code=206, host=108.175.39.172}
> at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:709)
> at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:150)
> at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
> at org.apache.avro.reflect.ReflectDatumWriter.writeField(ReflectDatumWriter.java:182)
> at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:150)
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
> {code}
> I've fixed this in my own [GitHub fork|https://github.com/mikkokupsu/avro/tree/hotfix/20160530/avro-schema-map-string-problem] and I've attached the patch too.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)