You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Jeff Hammerbacher (JIRA)" <ji...@apache.org> on 2010/08/15 01:47:16 UTC
[jira] Created: (HIVE-1540) Read-only, columnar data file for
nested data structures
Read-only, columnar data file for nested data structures
--------------------------------------------------------
Key: HIVE-1540
URL: https://issues.apache.org/jira/browse/HIVE-1540
Project: Hadoop Hive
Issue Type: New Feature
Reporter: Jeff Hammerbacher
RCFile is a great start on an optimized layout for working with structured data with Hive. Given that Hive's data model supports nested lists and maps, and taking inspiration from the recent work by Google on Dremel, it may be useful for the Hive community to think about how to improve the RCFile format for nested data structures.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1540) Read-only, columnar data file for
nested data structures
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899199#action_12899199 ]
Joydeep Sen Sarma commented on HIVE-1540:
-----------------------------------------
are there a lot of use cases for nested data structures? Google's approach is motivated by widespread use of Protocol Buffers. At Facebook - thrift serialized data sets (that motivated the initial support for nested data types) hasn't taken off.
I think what's much more common is json serialized data (or map types more restrictively). it would be much more worthwhile, to begin with, to have optimized codecs and deserializers for map types.
> Read-only, columnar data file for nested data structures
> --------------------------------------------------------
>
> Key: HIVE-1540
> URL: https://issues.apache.org/jira/browse/HIVE-1540
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Jeff Hammerbacher
>
> RCFile is a great start on an optimized layout for working with structured data with Hive. Given that Hive's data model supports nested lists and maps, and taking inspiration from the recent work by Google on Dremel, it may be useful for the Hive community to think about how to improve the RCFile format for nested data structures.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1540) Read-only, columnar data file for
nested data structures
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899255#action_12899255 ]
Jeff Hammerbacher commented on HIVE-1540:
-----------------------------------------
We've got an increasing number of customers using Avro's serialization format for working with data in Hadoop, and that's where our nested data structures come from. Any design which could incorporate a serialization framework like Avro would be of interest to me.
> Read-only, columnar data file for nested data structures
> --------------------------------------------------------
>
> Key: HIVE-1540
> URL: https://issues.apache.org/jira/browse/HIVE-1540
> Project: Hadoop Hive
> Issue Type: New Feature
> Reporter: Jeff Hammerbacher
>
> RCFile is a great start on an optimized layout for working with structured data with Hive. Given that Hive's data model supports nested lists and maps, and taking inspiration from the recent work by Google on Dremel, it may be useful for the Hive community to think about how to improve the RCFile format for nested data structures.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.