You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Franklin Hu (JIRA)" <ji...@apache.org> on 2011/07/26 02:31:09 UTC
[jira] [Created] (HIVE-2305) UNION ALL on different types throws
runtime exception
UNION ALL on different types throws runtime exception
-----------------------------------------------------
Key: HIVE-2305
URL: https://issues.apache.org/jira/browse/HIVE-2305
Project: Hive
Issue Type: Bug
Affects Versions: 0.7.1
Reporter: Franklin Hu
Assignee: Franklin Hu
Ex:
SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
Unioning columns of different types currently does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2305) UNION ALL on different types throws
runtime exception
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072961#comment-13072961 ]
jiraposter@reviews.apache.org commented on HIVE-2305:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1223/
-----------------------------------------------------------
Review request for hive.
Summary
-------
UNION ALL on columns of different types is semantically allowed but throws a runtime exception due to bugs in getting the correct object inspector after the parent MR input file is change and in setting the TypeInfo for the union row schema.
This addresses bug hive-2305.
https://issues.apache.org/jira/browse/hive-2305
Diffs
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnInfo.java 1151937
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1151937
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java 1151937
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/UnionOperator.java 1151937
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1151937
trunk/ql/src/test/queries/clientpositive/union24.q PRE-CREATION
trunk/ql/src/test/results/clientpositive/union24.q.out PRE-CREATION
trunk/ql/src/test/results/compiler/plan/union.q.xml 1151937
Diff: https://reviews.apache.org/r/1223/diff
Testing
-------
unit tests pass
Thanks,
Franklin
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Attachments: hive-2305.1.patch, hive-2305.2.patch, hive-2305.3.patch
>
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently throws runtime exceptions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2305) UNION ALL on different types throws
runtime exception
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13078504#comment-13078504 ]
jiraposter@reviews.apache.org commented on HIVE-2305:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1223/
-----------------------------------------------------------
(Updated 2011-08-02 23:28:01.105364)
Review request for hive, Yongqiang He, namit jain, and Siying Dong.
Summary
-------
UNION ALL on columns of different types is semantically allowed but throws a runtime exception due to bugs in getting the correct object inspector after the parent MR input file is change and in setting the TypeInfo for the union row schema.
This addresses bug hive-2305.
https://issues.apache.org/jira/browse/hive-2305
Diffs
-----
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnInfo.java 1151937
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 1151937
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java 1151937
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/UnionOperator.java 1151937
trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1151937
trunk/ql/src/test/queries/clientpositive/union24.q PRE-CREATION
trunk/ql/src/test/results/clientpositive/union24.q.out PRE-CREATION
trunk/ql/src/test/results/compiler/plan/union.q.xml 1151937
Diff: https://reviews.apache.org/r/1223/diff
Testing
-------
unit tests pass
Thanks,
Franklin
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Fix For: 0.8.0
>
> Attachments: hive-2305.1.patch, hive-2305.2.patch, hive-2305.3.patch
>
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently throws runtime exceptions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2305) UNION ALL on different types throws
runtime exception
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2305:
------------------------------
Description:
Ex:
SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
Unioning columns of different types currently throws runtime exceptions.
was:
Ex:
SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
Unioning columns of different types currently does not work.
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Attachments: hive-2305.1.patch
>
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently throws runtime exceptions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2305) UNION ALL on different types throws
runtime exception
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2305:
---------------------------------
Fix Version/s: (was: 0.8.0)
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Attachments: hive-2305.1.patch, hive-2305.2.patch, hive-2305.3.patch
>
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently throws runtime exceptions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2305) UNION ALL on different types throws
runtime exception
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2305:
------------------------------
Attachment: hive-2305.3.patch
-Added getCommonClassForUnion (needed since getCommonClass returns null for complex types)
-Added unit text
-Fixed changed existing unit test output
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Attachments: hive-2305.1.patch, hive-2305.2.patch, hive-2305.3.patch
>
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently throws runtime exceptions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HIVE-2305) UNION ALL on different types
throws runtime exception
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HIVE-2305 started by Franklin Hu.
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2305) UNION ALL on different types throws
runtime exception
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2305:
------------------------------
Attachment: hive-2305.2.patch
fix upstream input file change propagation
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Attachments: hive-2305.1.patch, hive-2305.2.patch
>
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently throws runtime exceptions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2305) UNION ALL on different types throws
runtime exception
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2305:
------------------------------
Fix Version/s: 0.8.0
Status: Patch Available (was: Open)
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Fix For: 0.8.0
>
> Attachments: hive-2305.1.patch, hive-2305.2.patch, hive-2305.3.patch
>
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently throws runtime exceptions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2305) UNION ALL on different types throws
runtime exception
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Franklin Hu updated HIVE-2305:
------------------------------
Attachment: hive-2305.1.patch
For multiple input directories, UnionOperator was not correctly setting the object inspector for the new row schema.
Also fixed incorrect call to get a common class.
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Attachments: hive-2305.1.patch
>
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently does not work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2305) UNION ALL on different types throws
runtime exception
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2305:
---------------------------------
Status: Open (was: Patch Available)
@Franklin: I see test failures in TestParse.union and TestCliDriver.union24. Can you please take a look? Thanks.
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Fix For: 0.8.0
>
> Attachments: hive-2305.1.patch, hive-2305.2.patch, hive-2305.3.patch
>
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently throws runtime exceptions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work stopped] (HIVE-2305) UNION ALL on different types
throws runtime exception
Posted by "Franklin Hu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HIVE-2305 stopped by Franklin Hu.
> UNION ALL on different types throws runtime exception
> -----------------------------------------------------
>
> Key: HIVE-2305
> URL: https://issues.apache.org/jira/browse/HIVE-2305
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.7.1
> Reporter: Franklin Hu
> Assignee: Franklin Hu
> Fix For: 0.8.0
>
> Attachments: hive-2305.1.patch, hive-2305.2.patch, hive-2305.3.patch
>
>
> Ex:
> SELECT * (SELECT 123 FROM ... UNION ALL SELECT '123' FROM ..) t;
> Unioning columns of different types currently throws runtime exceptions.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira