[jira] [Updated] (HIVE-10187) Avro backed tables don't handle cyclical or recursive records

["Mark Wagner (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-10187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Wagner updated HIVE-10187:
-------------------------------
    Attachment: HIVE-10187.demo.patch

Here's a patch which extends the testcase added in HIVE-7653 to illustrate the issue.

The problem here is that the generated TypeInfo for these data sets replaces the recursive fields with void, so any serializer will faithfully serialize nulls instead of any relevant data. However, making the TypeInfo properly recursive is likely to not work with other components of Hive. At the very least, the 'describe' statement will cause an infinite loop. I believe these issues could be fixed, but it would require making cyclical/recursive datasets a first class citizen in Hive's data model. 

> Avro backed tables don't handle cyclical or recursive records
> -------------------------------------------------------------
>
>                 Key: HIVE-10187
>                 URL: https://issues.apache.org/jira/browse/HIVE-10187
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>    Affects Versions: 1.2.0
>            Reporter: Mark Wagner
>         Attachments: HIVE-10187.demo.patch
>
>
> [HIVE-7653] changed the Avro SerDe to make it generate TypeInfos even for recursive/cyclical schemas. However, any attempt to serialize data which exploits that ability results in silently dropped fields.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)