You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ashutosh Chauhan (Created) (JIRA)" <ji...@apache.org> on 2011/10/18 12:10:10 UTC
[jira] [Created] (HIVE-2508) Join on union type fails
Join on union type fails
------------------------
Key: HIVE-2508
URL: https://issues.apache.org/jira/browse/HIVE-2508
Project: Hive
Issue Type: Bug
Reporter: Ashutosh Chauhan
{code}
hive> CREATE TABLE DEST1(key UNIONTYPE<STRING, STRING>, value BIGINT) STORED AS TEXTFILE;
OK
Time taken: 0.076 seconds
hive> CREATE TABLE DEST2(key UNIONTYPE<STRING, STRING>, value BIGINT) STORED AS TEXTFILE;
OK
Time taken: 0.034 seconds
hive> SELECT * FROM DEST1 JOIN DEST2 on (DEST1.key = DEST2.key);
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2508) Join on union type fails
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2508:
---------------------------------
Component/s: Query Processor
> Join on union type fails
> ------------------------
>
> Key: HIVE-2508
> URL: https://issues.apache.org/jira/browse/HIVE-2508
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Ashutosh Chauhan
> Labels: uniontype
>
> {code}
> hive> CREATE TABLE DEST1(key UNIONTYPE<STRING, STRING>, value BIGINT) STORED AS TEXTFILE;
> OK
> Time taken: 0.076 seconds
> hive> CREATE TABLE DEST2(key UNIONTYPE<STRING, STRING>, value BIGINT) STORED AS TEXTFILE;
> OK
> Time taken: 0.034 seconds
> hive> SELECT * FROM DEST1 JOIN DEST2 on (DEST1.key = DEST2.key);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2508) Join on union type fails
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132820#comment-13132820 ]
Ashutosh Chauhan commented on HIVE-2508:
----------------------------------------
Thinking more about this. I dont think we want to support join on union type, since in general real data in a union could be of any type which makes it impossible to do comparison of union types correctly. It can be argued that in some special cases that could be achieved via type widening, but I would argue selecting union type for storing such data which later need to be joined is a wrong choice. Thoughts?
If there is an agreement, I propose to disable this in semantic analysis phase itself instead of letting it fail at runtime later.
> Join on union type fails
> ------------------------
>
> Key: HIVE-2508
> URL: https://issues.apache.org/jira/browse/HIVE-2508
> Project: Hive
> Issue Type: Bug
> Reporter: Ashutosh Chauhan
>
> {code}
> hive> CREATE TABLE DEST1(key UNIONTYPE<STRING, STRING>, value BIGINT) STORED AS TEXTFILE;
> OK
> Time taken: 0.076 seconds
> hive> CREATE TABLE DEST2(key UNIONTYPE<STRING, STRING>, value BIGINT) STORED AS TEXTFILE;
> OK
> Time taken: 0.034 seconds
> hive> SELECT * FROM DEST1 JOIN DEST2 on (DEST1.key = DEST2.key);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2508) Join on union type fails
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129635#comment-13129635 ]
Ashutosh Chauhan commented on HIVE-2508:
----------------------------------------
NPE is thrown in setup task of Reducer:
{code}
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:484)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:419)
at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Reduce operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:157)
... 14 more
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardObjectInspector(ObjectInspectorUtils.java:105)
at org.apache.hadoop.hive.ql.exec.JoinUtil.getStandardObjectInspectors(JoinUtil.java:97)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:255)
at org.apache.hadoop.hive.ql.exec.JoinOperator.initializeOp(JoinOperator.java:59)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:150)
... 14 more
{code}
> Join on union type fails
> ------------------------
>
> Key: HIVE-2508
> URL: https://issues.apache.org/jira/browse/HIVE-2508
> Project: Hive
> Issue Type: Bug
> Reporter: Ashutosh Chauhan
>
> {code}
> hive> CREATE TABLE DEST1(key UNIONTYPE<STRING, STRING>, value BIGINT) STORED AS TEXTFILE;
> OK
> Time taken: 0.076 seconds
> hive> CREATE TABLE DEST2(key UNIONTYPE<STRING, STRING>, value BIGINT) STORED AS TEXTFILE;
> OK
> Time taken: 0.034 seconds
> hive> SELECT * FROM DEST1 JOIN DEST2 on (DEST1.key = DEST2.key);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira