You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by ta...@apache.org on 2016/10/05 20:10:38 UTC
[5/6] incubator-impala git commit: IMPALA-4042: Preserve root types
when substituting grouping exprs
IMPALA-4042: Preserve root types when substituting grouping exprs
In case of count(distinct), FunctionCallExpr.analyze() changes type
for "NULL" into "BOOLEAN" to make sure that BE doesn't see any
"NULL_TYPE" exprs. In the meantime, Expr substitution, happening in
Expr.substituteImpl() reverts this change back to original type,
"NULL_TYPE".
This causes an issue when AggregateInfo.checkConsistency() performs
precondition check where slot types from
AggregateInfo.outputTupleDesc_ should be matched with the types from
AggregateInfo.groupingExpr_. The slot type shows "BOOLEAN" while type
from groupingExpr_ is "NULL_TYPE", which makes the precondition fail
and throws an exception.
To resolve the issue, preserveRootType is set to true when
Expr.substituteList() gets called in AggregateInfo.substitute()
Change-Id: Icf3b4511234e473e5b9548fbf3e97f333c9980f1
(cherry picked from commit b17785b4890bedd1c825140ce3c48cd7d9734295)
Reviewed-on: http://gerrit.cloudera.org:8080/4600
Reviewed-by: Alex Behm <al...@cloudera.com>
Tested-by: Internal Jenkins
Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/112ff68e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/112ff68e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/112ff68e
Branch: refs/heads/master
Commit: 112ff68eddde5e29bacf31170741ee7746c0cbb8
Parents: e659337
Author: Yonghyun Hwang <yo...@cloudera.com>
Authored: Fri Sep 30 16:12:46 2016 -0700
Committer: Internal Jenkins <cl...@gerrit.cloudera.org>
Committed: Wed Oct 5 03:04:17 2016 +0000
----------------------------------------------------------------------
.../apache/impala/analysis/AggregateInfo.java | 4 +++-
.../queries/PlannerTest/distinct.test | 22 ++++++++++++++++++++
2 files changed, 25 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/112ff68e/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java
----------------------------------------------------------------------
diff --git a/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java b/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java
index 29b80ef..8590872 100644
--- a/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java
+++ b/fe/src/main/java/org/apache/impala/analysis/AggregateInfo.java
@@ -335,7 +335,9 @@ public class AggregateInfo extends AggregateInfoBase {
*/
public void substitute(ExprSubstitutionMap smap, Analyzer analyzer)
throws InternalException {
- groupingExprs_ = Expr.substituteList(groupingExprs_, smap, analyzer, false);
+
+ // Preserve the root type for NULL literals.
+ groupingExprs_ = Expr.substituteList(groupingExprs_, smap, analyzer, true);
LOG.trace("AggInfo: grouping_exprs=" + Expr.debugString(groupingExprs_));
// The smap in this case should not substitute the aggs themselves, only
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/112ff68e/testdata/workloads/functional-planner/queries/PlannerTest/distinct.test
----------------------------------------------------------------------
diff --git a/testdata/workloads/functional-planner/queries/PlannerTest/distinct.test b/testdata/workloads/functional-planner/queries/PlannerTest/distinct.test
index 15f1d59..e5f3bce 100644
--- a/testdata/workloads/functional-planner/queries/PlannerTest/distinct.test
+++ b/testdata/workloads/functional-planner/queries/PlannerTest/distinct.test
@@ -473,3 +473,25 @@ select * from (select count(distinct int_col) cd from functional.alltypes group
00:SCAN HDFS [functional.alltypes]
partitions=24/24 files=24 size=478.45KB
====
+# IMPALA-4042: count(distinct NULL) fails on a view
+select count(distinct null) from functional.alltypes_view
+---- DISTRIBUTEDPLAN
+06:AGGREGATE [FINALIZE]
+| output: count:merge(NULL)
+|
+05:EXCHANGE [UNPARTITIONED]
+|
+02:AGGREGATE
+| output: count(NULL)
+|
+04:AGGREGATE
+| group by: NULL
+|
+03:EXCHANGE [HASH(NULL)]
+|
+01:AGGREGATE [STREAMING]
+| group by: NULL
+|
+00:SCAN HDFS [functional.alltypes]
+ partitions=24/24 files=24 size=478.45KB
+====