You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/06/24 22:31:00 UTC
[jira] [Commented] (IMPALA-9620) Predicates in the SELECT and GROUP-BY cause failure with CNF rewrite enabled
[ https://issues.apache.org/jira/browse/IMPALA-9620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558694#comment-17558694 ]
ASF subversion and git services commented on IMPALA-9620:
---------------------------------------------------------
Commit f561daa247d6e6bd2f898e9b637830bcd51b6cc5 in impala's branch refs/heads/master from guojingfeng
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f561daa24 ]
IMPALA-10865: Fix initialize SelectStmt's groupingExprs_ in analyzeGroupingExprs
This patch rollback some changes of IMPALA-9620. IMPALA-9620 re-
initialize SelectStmt's groupingExprs_ to ensure that group-by and
cnf exprs are analyzed. But the following patch of IMPALA-9693
explicitly analyzes exprs which is equivalent to IMPALA-9620. So this
rollback is safe here.
In general, the analyze algorithm is that:
1. Analyze the stmt tree and make copies of expressions
2. Rewrite selected expressions, **rewrite rules should ensure
rewritten exprs are analyzed**
3. Make copied expressions analyzed
4. ReAnalyze the tree
The problem is that if we change the groupingExprs_ of SelectStmt,
in re-analyze phase column alias will be substitude to Expr that
duplicate with origin column which will be removed in
`buildAggregateExprs`.
Another reason why this patch is submitted is that re-initialize
SelectStmt's groupingExprs_ will cause other problems. IMPALA-10096 is
a typical case. See jira for detail execeptions.
Beside, this patch modifies ExtractCompundVerticalBarExprRule to do a
explicit analyze to ensure expr are rewritten.
Test:
- Add new test into aggregation.test and passed
- Ran all fe tests and passed
Change-Id: I9d1779e6c282d9fd02beacf5ddfafcc5c0baf3b0
Reviewed-on: http://gerrit.cloudera.org:8080/17781
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Predicates in the SELECT and GROUP-BY cause failure with CNF rewrite enabled
> ----------------------------------------------------------------------------
>
> Key: IMPALA-9620
> URL: https://issues.apache.org/jira/browse/IMPALA-9620
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Aman Sinha
> Assignee: Aman Sinha
> Priority: Major
> Fix For: Impala 4.0.0
>
>
> Predicates can appear in the SELECT and GROUP BY list as part of IF(), CASE() clauses.
> When enable_cnf_rewrites is set to true, such queries encounter failure during planning as shown below. Queries run successfully when the flag is disabled.
> Note that the predicate does not have to be disjunctive predicate for this failure to occur..even other types of predicates repro the issue.
> {noformat}
> set enable_cnf_rewrites = true;
> select l_quantity, if(l_quantity < 5 or l_quantity > 45, 'invalid', 'valid') from lineitem group by l_quantity, if(l_quantity < 5 or l_quantity > 45, 'invalid', 'valid') limit 5
> ERROR: IllegalStateException: null
> {noformat}
> Stack trace:
> {noformat}
> I0407 17:40:40.306650 31240 jni-util.cc:288] 2741e90d2edac592:c625a35f00000000] java.lang.IllegalStateException
> at com.google.common.base.Preconditions.checkState(Preconditions.java:492)
> at org.apache.impala.analysis.SlotRef.getIdsHelper(SlotRef.java:229)
> at org.apache.impala.analysis.Expr.getIdsHelper(Expr.java:1286)
> at org.apache.impala.analysis.Expr.getIdsHelper(Expr.java:1286)
> at org.apache.impala.analysis.Expr.getIds(Expr.java:1279)
> at org.apache.impala.rewrite.ConvertToCNFRule.convertToCNF(ConvertToCNFRule.java:111)
> at org.apache.impala.rewrite.ConvertToCNFRule.apply(ConvertToCNFRule.java:86)
> at org.apache.impala.rewrite.ExprRewriter.applyRuleBottomUp(ExprRewriter.java:85)
> at org.apache.impala.rewrite.ExprRewriter.applyRuleBottomUp(ExprRewriter.java:83)
> at org.apache.impala.rewrite.ExprRewriter.applyRuleRepeatedly(ExprRewriter.java:71)
> at org.apache.impala.rewrite.ExprRewriter.rewrite(ExprRewriter.java:55)
> at org.apache.impala.analysis.SelectStmt.rewriteCheckOrdinalResult(SelectStmt.java:1043)
> at org.apache.impala.analysis.SelectStmt.rewriteExprs(SelectStmt.java:1068)
> at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:472)
> at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:415)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1530)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1497)
> {noformat}
> Other variations of the same pattern that also encounter similar failure:
> {noformat}
> explain select case when not (l_quantity = 5) then 0 else 1 end from lineitem group by case when not (l_quantity = 5) then 0 else 1 end
> ERROR: IllegalStateException: null
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org