You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Zheng Shao (JIRA)" <ji...@apache.org> on 2010/02/02 09:28:20 UTC
[jira] Created: (HIVE-1124) create view should expand the query
text consistently
create view should expand the query text consistently
-----------------------------------------------------
Key: HIVE-1124
URL: https://issues.apache.org/jira/browse/HIVE-1124
Project: Hadoop Hive
Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Zheng Shao
Assignee: John Sichi
We should expand the omitted alias in the same way in "select" and in "group by".
Hive "Group By" recognize "group by" expressions by comparing the literal string.
{code}
hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
OK
Time taken: 0.286 seconds
hive> select * from zshao_view;
FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
] used as zshao_view at line 1:14
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1124) create view should expand the query
text consistently
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829471#action_12829471 ]
Zheng Shao commented on HIVE-1124:
----------------------------------
Can you make the query result deterministic by adding an "order by" at the end? Otherwise the tests might fail.
{code}
[junit] diff -a -I file: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I owner -I transient_lastDdlTime /data/users/zshao/hadoop_hive_trunk/.ptest_0/build/ql/test/logs\
/clientpositive/create_view.q.out /data/users/zshao/hadoop_hive_trunk/.ptest_0/ql/src/test/results/clientpositive/create_view.q.out
[junit] 869d868
[junit] < 401 5
[junit] 871,873c870
[junit] < 489 4
[junit] < 169 4
[junit] < 277 4
[junit] ---
[junit] > 401 5
[junit] 875a873,875
[junit] > 277 4
[junit] > 489 4
[junit] > 169 4
{code}
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch, HIVE-1124.2.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1124) create view should expand the query
text consistently
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1124:
-----------------------------
Attachment: HIVE-1124.2.patch
Updated patch with Zheng's suggestion. (I had left two calls to get("") unchanged because I wasn't sure how the key/value pairs in the aggregation map were being constucted, but it looks like the text and ASTNode are already 1-to-1 there.)
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch, HIVE-1124.2.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1124) create view should expand the query
text consistently
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1124:
-----------------------------
Attachment: HIVE-1124.1.patch
The problem turned out to be with the way we resolve common subexpressions for matching expressions in the SELECT list and GROUP BY. As soon as we saw the match, we were skipping over the SELECT item subtree without performing any translations, leading to the inconsistency later on when the view was referenced.
The fix in this patch is a bit involved, since I had to make the UnparseTranslator capable of copying the translations for the common subexpressions.
I have not yet run through all tests yet, but the patch is available for review now.
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Attachments: HIVE-1124.1.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1124) create view should expand the query
text consistently
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829369#action_12829369 ]
Zheng Shao commented on HIVE-1124:
----------------------------------
Patch line 993:
{code}
- ColumnInfo paraExprInfo = groupByInputRowResolver.get("", text);
+ ColumnInfo paraExprInfo = groupByInputRowResolver.get("", entry.getKey());
{code}
Shall we use getExpression?
Otherwise it looks good.
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1124) create view should expand the query
text consistently
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-1124:
-----------------------------
Status: Open (was: Patch Available)
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1124) create view should expand the query
text consistently
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-1124:
-----------------------------
Status: Patch Available (was: Open)
+1. Will test and commit.
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch, HIVE-1124.2.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1124) create view should expand the query
text consistently
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-1124:
-----------------------------
Resolution: Fixed
Release Note: HIVE-1124. Create view should expand the query text consistently. (John Sichi via zshao)
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed. Thanks John!
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch, HIVE-1124.2.patch, HIVE-1124.3.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1124) create view should expand the query
text consistently
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1124:
-----------------------------
Fix Version/s: 0.6.0
Status: Patch Available (was: Open)
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1124) create view should expand the query
text consistently
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829367#action_12829367 ]
John Sichi commented on HIVE-1124:
----------------------------------
Ran through all tests successfully.
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1124) CREATE VIEW should expand the query
text consistently
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1124:
---------------------------------
Release Note: (was: HIVE-1124. Create view should expand the query text consistently. (John Sichi via zshao))
Summary: CREATE VIEW should expand the query text consistently (was: create view should expand the query text consistently)
> CREATE VIEW should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch, HIVE-1124.2.patch, HIVE-1124.3.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1124) create view should expand the query
text consistently
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1124:
-----------------------------
Status: Patch Available (was: Open)
.3.patch fixes the ORDER BY.
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch, HIVE-1124.2.patch, HIVE-1124.3.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1124) create view should expand the query
text consistently
Posted by "John Sichi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-1124:
-----------------------------
Attachment: HIVE-1124.3.patch
Whoops, I had an ORDER BY, but only a subset of the full key.
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch, HIVE-1124.2.patch, HIVE-1124.3.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1124) create view should expand the query
text consistently
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-1124:
-----------------------------
Status: Open (was: Patch Available)
> create view should expand the query text consistently
> -----------------------------------------------------
>
> Key: HIVE-1124
> URL: https://issues.apache.org/jira/browse/HIVE-1124
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Zheng Shao
> Assignee: John Sichi
> Fix For: 0.6.0
>
> Attachments: HIVE-1124.1.patch, HIVE-1124.2.patch
>
>
> We should expand the omitted alias in the same way in "select" and in "group by".
> Hive "Group By" recognize "group by" expressions by comparing the literal string.
> {code}
> hive> create view zshao_view as select d, count(1) as cnt from zshao_tt group by d;
> OK
> Time taken: 0.286 seconds
> hive> select * from zshao_view;
> FAILED: Error in semantic analysis: line 1:7 Expression Not In Group By Key d in definition of VIEW zshao_view [
> select d, count(1) as `cnt` from `zshao_tt` group by `zshao_tt`.`d`
> ] used as zshao_view at line 1:14
> {code}
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.