[jira] [Created] (HIVE-6783) Incompatible schema for maps between parquet-hive and parquet-pig

["Tongjie Chen (JIRA)" <ji...@apache.org>]

Tongjie Chen created HIVE-6783:
----------------------------------

             Summary: Incompatible schema for maps between parquet-hive and parquet-pig
                 Key: HIVE-6783
                 URL: https://issues.apache.org/jira/browse/HIVE-6783
             Project: Hive
          Issue Type: Bug
          Components: File Formats
    Affects Versions: 0.13.0
            Reporter: Tongjie Chen


see also in following parquet issue:
https://github.com/Parquet/parquet-mr/issues/290

The schema written for maps isn't compatible between hive and pig. This means any files written in one cannot be properly read in the other.

More specifically,  for the same map column c1, parquet-pig generates schema:

message pig_schema {
  optional group c1 (MAP) {
    repeated group map (MAP_KEY_VALUE) {
      required binary key (UTF8);
      optional binary value;
    }   
  }
}

while parquet-hive generates schema:
message hive_schema {
   optional group c1 (MAP_KEY_VALUE) {
     repeated group map {
       required binary key;
       optional binary value;
   }
 }
}




--
This message was sent by Atlassian JIRA
(v6.2#6252)