You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hcatalog-commits@incubator.apache.org by "Julien Le Dem (JIRA)" <ji...@apache.org> on 2011/04/06 21:54:05 UTC
[jira] [Created] (HCATALOG-2) Support nested schema conversion
between Hive an Pig
Support nested schema conversion between Hive an Pig
----------------------------------------------------
Key: HCATALOG-2
URL: https://issues.apache.org/jira/browse/HCATALOG-2
Project: HCatalog
Issue Type: Improvement
Reporter: Julien Le Dem
Hive supports STRUCT and ARRAY
Pig supports Tuple and Bag
In both cases the types can be arbitrarily nested.
example:
STRUCT< foo : ARRAY<STRUCT < bar1: STRING, bar2: STRING > > mystruct
mytuple: ( foo: { t: ( bar1:chararray, bar2:chararray ) } )
HCatalog should support nested schema conversion.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HCATALOG-2) Support nested schema conversion
between Hive an Pig
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan reassigned HCATALOG-2:
----------------------------------
Assignee: Jakob Homan
> Support nested schema conversion between Hive an Pig
> ----------------------------------------------------
>
> Key: HCATALOG-2
> URL: https://issues.apache.org/jira/browse/HCATALOG-2
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Julien Le Dem
> Assignee: Jakob Homan
>
> Hive supports STRUCT and ARRAY
> Pig supports Tuple and Bag
> In both cases the types can be arbitrarily nested.
> example:
> STRUCT< foo : ARRAY<STRUCT < bar1: STRING, bar2: STRING > > mystruct
> mytuple: ( foo: { t: ( bar1:chararray, bar2:chararray ) } )
> HCatalog should support nested schema conversion.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HCATALOG-2) Support nested schema conversion
between Hive an Pig
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HCATALOG-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036355#comment-13036355 ]
Ashutosh Chauhan commented on HCATALOG-2:
-----------------------------------------
HCatalog uses following mapping:
TUPLE -> STRUCT
ARRAY -> BAG
One caveat here is HIve's Array can nest any other type, but Pig restricts bag to contain only tuples. So, HCatalog gets rid of this nested tuple when writing out data coming from Pig and inserts in a tuple when reading in. So, Pig's bag { tuple : (i : int)}
gets translated into Hive's Array <i:int> and NOT into Array<Struct<i:int>>
> Support nested schema conversion between Hive an Pig
> ----------------------------------------------------
>
> Key: HCATALOG-2
> URL: https://issues.apache.org/jira/browse/HCATALOG-2
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Julien Le Dem
> Assignee: Jakob Homan
>
> Hive supports STRUCT and ARRAY
> Pig supports Tuple and Bag
> In both cases the types can be arbitrarily nested.
> example:
> STRUCT< foo : ARRAY<STRUCT < bar1: STRING, bar2: STRING > > mystruct
> mytuple: ( foo: { t: ( bar1:chararray, bar2:chararray ) } )
> HCatalog should support nested schema conversion.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira