You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2022/08/26 00:25:59 UTC

[Impala-ASF-CR](branch-4.1.1) IMPALA-10865: Fix initialize SelectStmt's groupingExprs in analyzeGroupingExprs

Hello Impala Public Jenkins,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/18913

to review the following change.

Change subject: IMPALA-10865: Fix initialize SelectStmt's groupingExprs_ in analyzeGroupingExprs
......................................................................

IMPALA-10865: Fix initialize SelectStmt's groupingExprs_ in analyzeGroupingExprs

This patch rollback some changes of IMPALA-9620. IMPALA-9620 re-
initialize SelectStmt's groupingExprs_ to ensure that group-by and
cnf exprs are analyzed. But the following patch of IMPALA-9693
explicitly analyzes exprs which is equivalent to IMPALA-9620. So this
rollback is safe here.

In general, the analyze algorithm is that:
1. Analyze the stmt tree and make copies of expressions
2. Rewrite selected expressions, **rewrite rules should ensure
rewritten exprs are analyzed**
3. Make copied expressions analyzed
4. ReAnalyze the tree

The problem is that if we change the groupingExprs_ of SelectStmt,
in re-analyze phase column alias will be substitude to Expr that
duplicate with origin column which will be removed in
`buildAggregateExprs`.

Another reason why this patch is submitted is that re-initialize
SelectStmt's groupingExprs_ will cause other problems. IMPALA-10096 is
a typical case. See jira for detail execeptions.

Beside, this patch modifies ExtractCompundVerticalBarExprRule to do a
explicit analyze to ensure expr are rewritten.

Test:
- Add new test into aggregation.test and passed
- Ran all fe tests and passed

Change-Id: I9d1779e6c282d9fd02beacf5ddfafcc5c0baf3b0
Reviewed-on: http://gerrit.cloudera.org:8080/17781
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/rewrite/ExtractCompoundVerticalBarExprRule.java
M testdata/workloads/functional-planner/queries/PlannerTest/aggregation.test
3 files changed, 29 insertions(+), 15 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/13/18913/1
--
To view, visit http://gerrit.cloudera.org:8080/18913
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-4.1.1
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9d1779e6c282d9fd02beacf5ddfafcc5c0baf3b0
Gerrit-Change-Number: 18913
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: guojingfeng <gu...@tencent.com>

[Impala-ASF-CR](branch-4.1.1) IMPALA-10865: Fix initialize SelectStmt's groupingExprs in analyzeGroupingExprs

Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.

Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/18913 )

Change subject: IMPALA-10865: Fix initialize SelectStmt's groupingExprs_ in analyzeGroupingExprs
......................................................................


Patch Set 1: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/18913
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-4.1.1
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d1779e6c282d9fd02beacf5ddfafcc5c0baf3b0
Gerrit-Change-Number: 18913
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: guojingfeng <gu...@tencent.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Tue, 30 Aug 2022 08:00:41 +0000
Gerrit-HasComments: No

[Impala-ASF-CR](branch-4.1.1) IMPALA-10865: Fix initialize SelectStmt's groupingExprs in analyzeGroupingExprs

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.

Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18913 )

Change subject: IMPALA-10865: Fix initialize SelectStmt's groupingExprs_ in analyzeGroupingExprs
......................................................................


Patch Set 1: Verified+1

This is a clean cherrypick. Verified in https://jenkins.impala.io/job/gerrit-verify-dryrun/8498/ (ignored the flaky test failure).


-- 
To view, visit http://gerrit.cloudera.org:8080/18913
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-4.1.1
Gerrit-MessageType: comment
Gerrit-Change-Id: I9d1779e6c282d9fd02beacf5ddfafcc5c0baf3b0
Gerrit-Change-Number: 18913
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: guojingfeng <gu...@tencent.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Fri, 26 Aug 2022 05:39:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR](branch-4.1.1) IMPALA-10865: Fix initialize SelectStmt's groupingExprs in analyzeGroupingExprs

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.

Quanlong Huang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18913 )

Change subject: IMPALA-10865: Fix initialize SelectStmt's groupingExprs_ in analyzeGroupingExprs
......................................................................

IMPALA-10865: Fix initialize SelectStmt's groupingExprs_ in analyzeGroupingExprs

Another reason why this patch is submitted is that re-initialize
SelectStmt's groupingExprs_ will cause other problems. IMPALA-10096 is
a typical case. See jira for detail execeptions.

Beside, this patch modifies ExtractCompundVerticalBarExprRule to do a
explicit analyze to ensure expr are rewritten.

Test:
- Add new test into aggregation.test and passed
- Ran all fe tests and passed

Change-Id: I9d1779e6c282d9fd02beacf5ddfafcc5c0baf3b0
Reviewed-on: http://gerrit.cloudera.org:8080/17781
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-on: http://gerrit.cloudera.org:8080/18913
Tested-by: Quanlong Huang <hu...@gmail.com>
Reviewed-by: Tamas Mate <tm...@apache.org>
---
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/rewrite/ExtractCompoundVerticalBarExprRule.java
M testdata/workloads/functional-planner/queries/PlannerTest/aggregation.test
3 files changed, 29 insertions(+), 15 deletions(-)

Approvals:
Quanlong Huang: Verified
Tamas Mate: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/18913
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: branch-4.1.1
Gerrit-MessageType: merged
Gerrit-Change-Id: I9d1779e6c282d9fd02beacf5ddfafcc5c0baf3b0
Gerrit-Change-Number: 18913
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: guojingfeng <gu...@tencent.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>