You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/06/24 22:31:00 UTC
[jira] [Commented] (IMPALA-10865) Query with select alias the same as column name rewrite error for grouping exprs
[ https://issues.apache.org/jira/browse/IMPALA-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558693#comment-17558693 ]
ASF subversion and git services commented on IMPALA-10865:
----------------------------------------------------------
Commit f561daa247d6e6bd2f898e9b637830bcd51b6cc5 in impala's branch refs/heads/master from guojingfeng
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f561daa24 ]
IMPALA-10865: Fix initialize SelectStmt's groupingExprs_ in analyzeGroupingExprs
This patch rollback some changes of IMPALA-9620. IMPALA-9620 re-
initialize SelectStmt's groupingExprs_ to ensure that group-by and
cnf exprs are analyzed. But the following patch of IMPALA-9693
explicitly analyzes exprs which is equivalent to IMPALA-9620. So this
rollback is safe here.
In general, the analyze algorithm is that:
1. Analyze the stmt tree and make copies of expressions
2. Rewrite selected expressions, **rewrite rules should ensure
rewritten exprs are analyzed**
3. Make copied expressions analyzed
4. ReAnalyze the tree
The problem is that if we change the groupingExprs_ of SelectStmt,
in re-analyze phase column alias will be substitude to Expr that
duplicate with origin column which will be removed in
`buildAggregateExprs`.
Another reason why this patch is submitted is that re-initialize
SelectStmt's groupingExprs_ will cause other problems. IMPALA-10096 is
a typical case. See jira for detail execeptions.
Beside, this patch modifies ExtractCompundVerticalBarExprRule to do a
explicit analyze to ensure expr are rewritten.
Test:
- Add new test into aggregation.test and passed
- Ran all fe tests and passed
Change-Id: I9d1779e6c282d9fd02beacf5ddfafcc5c0baf3b0
Reviewed-on: http://gerrit.cloudera.org:8080/17781
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Query with select alias the same as column name rewrite error for grouping exprs
> --------------------------------------------------------------------------------
>
> Key: IMPALA-10865
> URL: https://issues.apache.org/jira/browse/IMPALA-10865
> Project: IMPALA
> Issue Type: Bug
> Affects Versions: Impala 4.0.0
> Reporter: guojingfeng
> Assignee: guojingfeng
> Priority: Major
>
> We hit a AnalyzeException when replay online query stmts when upgrading from 3.4.x from 4.0.
> The exception is that:
> {code:java}
> Java exception follows:
> org.apache.impala.common.AnalysisException: select list expression not produced by aggregation output (missing from GROUP BY clause?): column column_alias
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.verifyAggregation(SelectStmt.java:906)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:297)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:265)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:258)
> at org.apache.impala.analysis.AnalysisContext.reAnalyze(AnalysisContext.java:575)
> at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:549)
> at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:445)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1659)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1625)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1595)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:162)
> I0802 22:05:34.565348 6026 BaseAuthorizationChecker.java:112] 774fe9cee424b3f0:86fb11dc00000000] Authorization check took 107 ms
> I0802 22:05:34.565541 6026 jni-util.cc:286] 774fe9cee424b3f0:86fb11dc00000000] org.apache.impala.common.AnalysisException: select list expression not produced by aggregation output (missing from GROUP BY clause?): test_id test_group
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.verifyAggregation(SelectStmt.java:906)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:297)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:265)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:258)
> at org.apache.impala.analysis.AnalysisContext.reAnalyze(AnalysisContext.java:575)
> at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:549)
> at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:445)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1659)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1625)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1595)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:162)
> I0802 22:05:34.565551 6026 status.cc:129] 774fe9cee424b3f0:86fb11dc00000000] AnalysisException: select list expression not produced by aggregation output (missing from GROUP BY clause?): column column_alias
> @ 0xd6c6d3
> @ 0x159e8e9
> @ 0x133545b
> @ 0x19e704b
> @ 0x13a46b1
> @ 0x13b67d2
> @ 0x14a3089
> @ 0x14574d7
> @ 0x13c2177
> @ 0xcc9af5
> @ 0x1128176
> @ 0x111c058
> @ 0x16649e1
> @ 0x1665e5a
> @ 0x1ea21c1
> @ 0x7f20726f8e24
> @ 0x7f206f20b35c
> {code}
> This is a corner case of SelectStmt analyze.
> In general, the analyze algorithm is that:
> 1. Analyze the stmt tree and make copies of expressions
> 2. Rewrite selected expressions
> 3. Make copied expressions analyzed
> 4. ReAnalyze the tree // This is the root cause
> The problem maybe is that step 1~3 changed the state of expr tree but not keep in sync with re-analyze phase.
>
> Reproduce case:
> {code:java}
> EXPLAIN
> SELECT ss_item_sk ss_item_sk_group,
> ss_item_sk+300 ss_item_sk,
> count(ss_ticket_number)
> FROM tpcds.store_sales a
> WHERE ss_sold_date_sk > cast('245263' AS INT)
> GROUP BY ss_item_sk_group,
> ss_item_sk;
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org