You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Paul Yang (JIRA)" <ji...@apache.org> on 2010/02/20 05:21:27 UTC
[jira] Created: (HIVE-1184) Expression Not In Group By Key error is
sometimes masked
Expression Not In Group By Key error is sometimes masked
--------------------------------------------------------
Key: HIVE-1184
URL: https://issues.apache.org/jira/browse/HIVE-1184
Project: Hadoop Hive
Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Paul Yang
Assignee: Paul Yang
Depending on the order of expressions, the error message for a expression not in group key is not displayed; instead it is null.
{code}
hive> select concat(value, concat(value)) from src group by concat(value);
FAILED: Error in semantic analysis: null
hive> select concat(concat(value), value) from src group by concat(value);
FAILED: Error in semantic analysis: line 1:29 Expression Not In Group By Key value
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1184) Expression Not In Group By Key error is
sometimes masked
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1184:
----------------------------
Attachment: HIVE-1184.1.patch
The problem stems from DefaultExprProcessor:process(). During the AST walk, any error is cleared if a node matches a group by expression
{code}
...
exprNodeDesc desc = TypeCheckProcFactory.processGByExpr(nd, procCtx);
if (desc != null) {
ctx.setError(null);
return desc;
}
{code}
Clearing the error was probably an attempt to address the generation of false errors during the DFS walk of the AST. Consider the AST fragment from the query
{code}
SELECT concat(src.key) FROM src GROUP BY concat(src.key)
TOK_FUNCTION
/ \
/ \
/ \
concat .
/ \
/ \
TOK_TABLE_OR_COL key
|
|
src
{code}
During the walk, process() will be called on src before TOK_FUNCTION. Because src is not a group by expression, an error will be set in ctx. However, when process() is called on TOK_FUNCTION, it matches the group by expression 'concat(src.key)' and the error is cleared, producing the expected behavior.
A problem arises with a query like
{code}
select concat(value, concat(value)) from src group by concat(value)
{code}
as the AST is such that 'value' (1st argument of outer concat) is processed before 'concat(value)'. When process() acts on 'value', it sets an error because it is not a group by expression. But then the error is cleared when process() is called on 'concat(value)'. The error should not really be cleared as it was generated outside of the group by expression.
The proposed solution is to keep track of the ASTNode that generated the error and only clear the error when it was generated from a node within the group by expression.
> Expression Not In Group By Key error is sometimes masked
> --------------------------------------------------------
>
> Key: HIVE-1184
> URL: https://issues.apache.org/jira/browse/HIVE-1184
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Attachments: HIVE-1184.1.patch
>
>
> Depending on the order of expressions, the error message for a expression not in group key is not displayed; instead it is null.
> {code}
> hive> select concat(value, concat(value)) from src group by concat(value);
> FAILED: Error in semantic analysis: null
> hive> select concat(concat(value), value) from src group by concat(value);
> FAILED: Error in semantic analysis: line 1:29 Expression Not In Group By Key value
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1184) Expression Not In Group By Key error
is sometimes masked
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao resolved HIVE-1184.
------------------------------
Resolution: Fixed
Fix Version/s: 0.6.0
Release Note: HIVE-1184. Fix Expression Not In Group By Key error. (Paul Yang via zshao)
Hadoop Flags: [Reviewed]
Committed. Thanks Paul!
> Expression Not In Group By Key error is sometimes masked
> --------------------------------------------------------
>
> Key: HIVE-1184
> URL: https://issues.apache.org/jira/browse/HIVE-1184
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.6.0
>
> Attachments: HIVE-1184.1.patch, HIVE-1184.2.patch
>
>
> Depending on the order of expressions, the error message for a expression not in group key is not displayed; instead it is null.
> {code}
> hive> select concat(value, concat(value)) from src group by concat(value);
> FAILED: Error in semantic analysis: null
> hive> select concat(concat(value), value) from src group by concat(value);
> FAILED: Error in semantic analysis: line 1:29 Expression Not In Group By Key value
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1184) Expression Not In Group By Key error is
sometimes masked
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1184:
----------------------------
Status: Patch Available (was: Open)
> Expression Not In Group By Key error is sometimes masked
> --------------------------------------------------------
>
> Key: HIVE-1184
> URL: https://issues.apache.org/jira/browse/HIVE-1184
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Attachments: HIVE-1184.1.patch
>
>
> Depending on the order of expressions, the error message for a expression not in group key is not displayed; instead it is null.
> {code}
> hive> select concat(value, concat(value)) from src group by concat(value);
> FAILED: Error in semantic analysis: null
> hive> select concat(concat(value), value) from src group by concat(value);
> FAILED: Error in semantic analysis: line 1:29 Expression Not In Group By Key value
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1184) Expression Not In Group By Key error is
sometimes masked
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-1184:
-----------------------------
Status: Open (was: Patch Available)
> Expression Not In Group By Key error is sometimes masked
> --------------------------------------------------------
>
> Key: HIVE-1184
> URL: https://issues.apache.org/jira/browse/HIVE-1184
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Attachments: HIVE-1184.1.patch
>
>
> Depending on the order of expressions, the error message for a expression not in group key is not displayed; instead it is null.
> {code}
> hive> select concat(value, concat(value)) from src group by concat(value);
> FAILED: Error in semantic analysis: null
> hive> select concat(concat(value), value) from src group by concat(value);
> FAILED: Error in semantic analysis: line 1:29 Expression Not In Group By Key value
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1184) Expression Not In Group By Key error is
sometimes masked
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1184:
---------------------------------
Component/s: Query Processor
Release Note: (was: HIVE-1184. Fix Expression Not In Group By Key error. (Paul Yang via zshao))
> Expression Not In Group By Key error is sometimes masked
> --------------------------------------------------------
>
> Key: HIVE-1184
> URL: https://issues.apache.org/jira/browse/HIVE-1184
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Fix For: 0.6.0
>
> Attachments: HIVE-1184.1.patch, HIVE-1184.2.patch
>
>
> Depending on the order of expressions, the error message for a expression not in group key is not displayed; instead it is null.
> {code}
> hive> select concat(value, concat(value)) from src group by concat(value);
> FAILED: Error in semantic analysis: null
> hive> select concat(concat(value), value) from src group by concat(value);
> FAILED: Error in semantic analysis: line 1:29 Expression Not In Group By Key value
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1184) Expression Not In Group By Key error
is sometimes masked
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12837476#action_12837476 ]
Zheng Shao commented on HIVE-1184:
----------------------------------
The explanation looks good to me, but I am not convinced the solution will solve the problem.
When processing "concat(value, concat(value))". we will set "error" when processing the first "value", then overwrite the "error" when processing the second "value", correct?
I think the "error" should be part of the "return value" of the "process" function, instead of a global field in the "context".
Does that make sense?
> Expression Not In Group By Key error is sometimes masked
> --------------------------------------------------------
>
> Key: HIVE-1184
> URL: https://issues.apache.org/jira/browse/HIVE-1184
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Attachments: HIVE-1184.1.patch
>
>
> Depending on the order of expressions, the error message for a expression not in group key is not displayed; instead it is null.
> {code}
> hive> select concat(value, concat(value)) from src group by concat(value);
> FAILED: Error in semantic analysis: null
> hive> select concat(concat(value), value) from src group by concat(value);
> FAILED: Error in semantic analysis: line 1:29 Expression Not In Group By Key value
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1184) Expression Not In Group By Key error is
sometimes masked
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-1184:
----------------------------
Attachment: HIVE-1184.2.patch
>From an offline conversation - the solution still works for cases like "concat(value, concat(value))" because the error is only set with the first "value" - node processors terminate early when the global error is set.
* Added better comments.
> Expression Not In Group By Key error is sometimes masked
> --------------------------------------------------------
>
> Key: HIVE-1184
> URL: https://issues.apache.org/jira/browse/HIVE-1184
> Project: Hadoop Hive
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Paul Yang
> Assignee: Paul Yang
> Attachments: HIVE-1184.1.patch, HIVE-1184.2.patch
>
>
> Depending on the order of expressions, the error message for a expression not in group key is not displayed; instead it is null.
> {code}
> hive> select concat(value, concat(value)) from src group by concat(value);
> FAILED: Error in semantic analysis: null
> hive> select concat(concat(value), value) from src group by concat(value);
> FAILED: Error in semantic analysis: line 1:29 Expression Not In Group By Key value
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.