[jira] [Commented] (HIVE-10687) AvroDeserializer fails to deserialize evolved union fields

["Swarnim Kulkarni (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14540519#comment-14540519 ] 

Swarnim Kulkarni commented on HIVE-10687:
-----------------------------------------

[~brocknoland] If you get a chance, mind taking a quick look since you had done some work on HIVE-9462 which seems related.

> AvroDeserializer fails to deserialize evolved union fields
> ----------------------------------------------------------
>
>                 Key: HIVE-10687
>                 URL: https://issues.apache.org/jira/browse/HIVE-10687
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>            Reporter: Swarnim Kulkarni
>            Assignee: Swarnim Kulkarni
>         Attachments: HIVE-10687.1.patch
>
>
> Consider the union field:
> {noformat}
> union {int, string}
> {noformat}
> and now this field evolves to
> {noformat}
> union {null, int, string}.
> {noformat}
> Running it through the avro schema compatibility check[1], they are actually compatible which means that the latter could be used to deserialize the data written with former. However the avro deserializer fails to do that. Mainly because of the way it reads the tags from the reader schema and then reds the corresponding data from the writer schema. [2]
> [1] http://pastebin.cerner.corp/31078
> [2] https://github.com/cloudera/hive/blob/cdh5.4.0-release/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L354



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)