You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by st...@apache.org on 2021/12/15 08:38:37 UTC
[impala] 01/04: IMPALA-11040: Remove unnecessary reset() method in class 'UnionStmt'
This is an automated email from the ASF dual-hosted git repository.
stigahuang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit ad29ce70b3f4cfab4105be52ef664129060fa04e
Author: skyyws <sk...@163.com>
AuthorDate: Wed Dec 1 19:40:57 2021 +0800
IMPALA-11040: Remove unnecessary reset() method in class 'UnionStmt'
When query contains multiple nested union stmt, more than twenty or
thirty, and needs 'reAnalyze', such as rewrite expr. Query would
execute slowly, even failed due to 'reset' method called in class
'UnionStmt' and 'SetOperationStmt'.
'SetOperationStmt' is added in IMPALA-9943 and IMPALA-4974. Multiple
nested union stmt will lead to 'reset' called numbers to grow
exponentially. Since 'operands_' will be reset in two class' reset()
method, and handle with their children recursively. Too many nested
union stmt will cause deep nesting.
UnionStmt.reset() content is exactly same as SetOperationStmt.reset().
This patch removed this method in 'UnionStmt'. After this, the
original query would execute quickly.
An example already add in file 'union.test', without this patch, the
example query would execute slowly, or even fail.
Testing:
- Added new test case in 'union.test'
Change-Id: I408a396d40d9622f2ae6c459f49cbfcc19affd14
Reviewed-on: http://gerrit.cloudera.org:8080/18061
Reviewed-by: Qifan Chen <qc...@cloudera.com>
Reviewed-by: Zoltan Borok-Nagy <bo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
.../java/org/apache/impala/analysis/UnionStmt.java | 20 ----
.../functional-query/queries/QueryTest/union.test | 129 +++++++++++++++++++++
2 files changed, 129 insertions(+), 20 deletions(-)
diff --git a/fe/src/main/java/org/apache/impala/analysis/UnionStmt.java b/fe/src/main/java/org/apache/impala/analysis/UnionStmt.java
index 12a1e99..90ef70a 100644
--- a/fe/src/main/java/org/apache/impala/analysis/UnionStmt.java
+++ b/fe/src/main/java/org/apache/impala/analysis/UnionStmt.java
@@ -53,24 +53,4 @@ public class UnionStmt extends SetOperationStmt {
public SetOperationStmt clone() {
return new UnionStmt(this);
}
-
- /**
- * Undoes all changes made by analyze() except distinct propagation and unnesting.
- * After analysis, operands_ contains the list of unnested operands with qualifiers
- * adjusted to reflect distinct propagation. Every operand in that list is reset().
- * The unionDistinctOperands_ and unionAllOperands_ are cleared because they are
- * redundant
- * with operands_.
- */
- @Override
- public void reset() {
- super.reset();
- for (SetOperand op : operands_) op.reset();
- unionDistinctOperands_.clear();
- unionAllOperands_.clear();
- distinctAggInfo_ = null;
- tupleId_ = null;
- toSqlString_ = null;
- widestExprs_ = null;
- }
}
diff --git a/testdata/workloads/functional-query/queries/QueryTest/union.test b/testdata/workloads/functional-query/queries/QueryTest/union.test
index 18889f6..fc5927e 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/union.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/union.test
@@ -1182,3 +1182,132 @@ group by id
---- TYPES
double, int
=====
+---- QUERY
+# IMPALA-11040: Verify query performance after removed reset() method in class 'UnionStmt'
+# Without this patch, the query would execute slowly, or even fail.
+SELECT t1.dt
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT *
+FROM
+(SELECT '20210101' AS dt
+UNION ALL SELECT '20210102' AS dt) AS t
+UNION ALL SELECT '20210103' AS dt) AS t
+UNION ALL SELECT '20210104' AS dt) AS t
+UNION ALL SELECT '20210105' AS dt) AS t
+UNION ALL SELECT '20210106' AS dt) AS t
+UNION ALL SELECT '20210107' AS dt) AS t
+UNION ALL SELECT '20210108' AS dt) AS t
+UNION ALL SELECT '20210109' AS dt) AS t
+UNION ALL SELECT '20210110' AS dt) AS t
+UNION ALL SELECT '20210111' AS dt) AS t
+UNION ALL SELECT '20210112' AS dt) AS t
+UNION ALL SELECT '20210113' AS dt) AS t
+UNION ALL SELECT '20210114' AS dt) AS t
+UNION ALL SELECT '20210115' AS dt) AS t
+UNION ALL SELECT '20210116' AS dt) AS t
+UNION ALL SELECT '20210117' AS dt) AS t
+UNION ALL SELECT '20210118' AS dt) AS t
+UNION ALL SELECT '20210119' AS dt) AS t
+UNION ALL SELECT '20210120' AS dt) AS t
+UNION ALL SELECT '20210121' AS dt) AS t
+UNION ALL SELECT '20210122' AS dt) AS t
+UNION ALL SELECT '20210123' AS dt) AS t
+UNION ALL SELECT '20210124' AS dt) AS t
+UNION ALL SELECT '20210125' AS dt) AS t
+UNION ALL SELECT '20210126' AS dt) AS t
+UNION ALL SELECT '20210127' AS dt) AS t
+UNION ALL SELECT '20210128' AS dt) AS t
+UNION ALL SELECT '20210129' AS dt) AS t
+UNION ALL SELECT '20210130' AS dt) AS t1
+CROSS JOIN
+(SELECT 1000+1000) ASt2
+GROUP BY t1.dt
+ORDER BY t1.dt;
+---- RESULTS
+'20210101'
+'20210102'
+'20210103'
+'20210104'
+'20210105'
+'20210106'
+'20210107'
+'20210108'
+'20210109'
+'20210110'
+'20210111'
+'20210112'
+'20210113'
+'20210114'
+'20210115'
+'20210116'
+'20210117'
+'20210118'
+'20210119'
+'20210120'
+'20210121'
+'20210122'
+'20210123'
+'20210124'
+'20210125'
+'20210126'
+'20210127'
+'20210128'
+'20210129'
+'20210130'
+---- TYPES
+string
+=====