[jira] [Commented] (PARQUET-53) Error in reading a list type data in hive which was generated through pig

["Ryan Blue (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PARQUET-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221713#comment-14221713 ] 

Ryan Blue commented on PARQUET-53:
----------------------------------

[~abhishek.agarwal]: We've been working on compatibility rules and a spec for how to store list structures in PARQUET-113. HIVE-8909 implements the read-side support for Hive and I think it will fix this problem. Is this something you can test with a version of Hive that includes that?

> Error in reading a list type data in hive which was generated through pig 
> --------------------------------------------------------------------------
>
>                 Key: PARQUET-53
>                 URL: https://issues.apache.org/jira/browse/PARQUET-53
>             Project: Parquet
>          Issue Type: Bug
>          Components: parquet-mr
>         Environment: Parquet-Pig - 1.3.2
> Hive - 0.13.1
> Elephant-Bird - 4.4
>            Reporter: Abhishek Agarwal
>
> ArrayWritableGroupConverter in Parquet-Hive binding expects that any array field contains either 1 or 2 fields. However, the schema which PigSchemaConverter generates, doesn't honour this condition. It can be reproduced through following code 
> {noformat}
> MessageType messageType = new PigSchemaConverter().convert(ThriftToPig.toSchema(Requestlist.class));
> DataWritableGroupConverter converter = new DataWritableGroupConverter(messageType, messageType);
> {noformat}
> Requestlist is a thrift class 
> {noformat}
> struct Request {
>     1: required string id,
>     2: i64 time,
>     3: required string source
> }
> struct Requestlist {
>     1: optional list<Request> requests
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)