You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Jakob Homan (JIRA)" <ji...@apache.org> on 2011/08/18 02:51:27 UTC
[jira] [Created] (HIVE-2390) LazyBinarySerde (and others) don't
support unions
LazyBinarySerde (and others) don't support unions
-------------------------------------------------
Key: HIVE-2390
URL: https://issues.apache.org/jira/browse/HIVE-2390
Project: Hive
Issue Type: Bug
Reporter: Jakob Homan
When the union type was introduced, full support for it wasn't provided. For instance, when working with a union that gets passed to LazyBinarySerde:
{noformat}Caused by: java.lang.RuntimeException: Unrecognized type: UNION
at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(LazyBinarySerDe.java:468)
at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serializeStruct(LazyBinarySerDe.java:230)
at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(LazyBinarySerDe.java:184)
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2390) LazyBinarySerde (and others) don't
support unions
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086714#comment-13086714 ]
Jakob Homan commented on HIVE-2390:
-----------------------------------
Part of the problem is that the term union has been overloaded. In SQL it means the actual set union of two compatible data types, whereas in Avro and programming languages it means one value that can be at any one time an instance of two or different types. Union was added as a full-on, first-class type by its inclusion in ObjectInspector's Category enum. Is there any reason not to expand this use to be more along the line of programming language's take on unions? If so, it should be marked as not really being a first-class type. If not, support for unions in all the serdes, in the grammar and in the documentation should be provided.
I would lobby for expanding its support as it's an important type in Avro and we're quite hobbled by the inability to manipulate unioned values. (Avro handles nullable values by unioning them with their type T and null, but Haivvreo transparently converts these just to the type and returns null where appropriate. The problem lies in actual unions of non-null types, which are less frequent but still valid.)
> LazyBinarySerde (and others) don't support unions
> -------------------------------------------------
>
> Key: HIVE-2390
> URL: https://issues.apache.org/jira/browse/HIVE-2390
> Project: Hive
> Issue Type: Bug
> Reporter: Jakob Homan
>
> When the union type was introduced, full support for it wasn't provided. For instance, when working with a union that gets passed to LazyBinarySerde:
> {noformat}Caused by: java.lang.RuntimeException: Unrecognized type: UNION
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(LazyBinarySerDe.java:468)
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serializeStruct(LazyBinarySerDe.java:230)
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(LazyBinarySerDe.java:184)
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2390) Expand support for union types
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086777#comment-13086777 ]
Amareshwari Sriramadasu commented on HIVE-2390:
-----------------------------------------------
+1. I agree that when Union type was added, complete support for it was not added. We should extend its usage in all the serdes.
bq. Part of the problem is that the term union has been overloaded.
The type is called 'uniontype' in Hive to resolve ambiguities.
> Expand support for union types
> ------------------------------
>
> Key: HIVE-2390
> URL: https://issues.apache.org/jira/browse/HIVE-2390
> Project: Hive
> Issue Type: Bug
> Reporter: Jakob Homan
>
> When the union type was introduced, full support for it wasn't provided. For instance, when working with a union that gets passed to LazyBinarySerde:
> {noformat}Caused by: java.lang.RuntimeException: Unrecognized type: UNION
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(LazyBinarySerDe.java:468)
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serializeStruct(LazyBinarySerDe.java:230)
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(LazyBinarySerDe.java:184)
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2390) Expand support for union types
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan reassigned HIVE-2390:
---------------------------------
Assignee: Jakob Homan
> Expand support for union types
> ------------------------------
>
> Key: HIVE-2390
> URL: https://issues.apache.org/jira/browse/HIVE-2390
> Project: Hive
> Issue Type: Bug
> Reporter: Jakob Homan
> Assignee: Jakob Homan
>
> When the union type was introduced, full support for it wasn't provided. For instance, when working with a union that gets passed to LazyBinarySerde:
> {noformat}Caused by: java.lang.RuntimeException: Unrecognized type: UNION
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(LazyBinarySerDe.java:468)
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serializeStruct(LazyBinarySerDe.java:230)
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(LazyBinarySerDe.java:184)
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2390) Expand support for union types
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HIVE-2390:
------------------------------
Summary: Expand support for union types (was: LazyBinarySerde (and others) don't support unions)
Changing name of JIRA to be more representative of what needs to be done. If reaction is positive, will open subtasks for individual items.
> Expand support for union types
> ------------------------------
>
> Key: HIVE-2390
> URL: https://issues.apache.org/jira/browse/HIVE-2390
> Project: Hive
> Issue Type: Bug
> Reporter: Jakob Homan
>
> When the union type was introduced, full support for it wasn't provided. For instance, when working with a union that gets passed to LazyBinarySerde:
> {noformat}Caused by: java.lang.RuntimeException: Unrecognized type: UNION
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(LazyBinarySerDe.java:468)
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serializeStruct(LazyBinarySerDe.java:230)
> at org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe.serialize(LazyBinarySerDe.java:184)
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira