You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Xianqing He (Code Review)" <ge...@cloudera.org> on 2020/01/21 05:46:25 UTC
[Impala-ASF-CR] IMPALA-8361: Propagate predicates of outer-joined InlineView
Xianqing He has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/15047 )
Change subject: IMPALA-8361: Propagate predicates of outer-joined InlineView
......................................................................
IMPALA-8361: Propagate predicates of outer-joined InlineView
This is improvement that propagate predicates of the nullable side of
the outer join into inline view.
I refactored the code "Analyzer#canEvalPredicate()" and add
"SingleNodePlanner#getConjunctsToInlineView()" to get conjucts can be
correctly to migrate or propagate into inline view.
Before this change, we skip predicate b.upper_val='1' since
canEvalPredicate() returns false on it. Because it doesn't satisfy
isLastOjMaterializedByTupleIds(). So we can't migrate it inside the
inline view. However, we can be more aggressive. It's correct to
duplicate(not migrate) this predicate inside the inline view since
it's not evaluted to true with null slots.
This patch picks up the predicates that being skipped due to not
satisfying isLastOjMaterializedByTupleIds() and duplicates them
if they satisfy !isTrueWithNullSlots().
For example:
EXPLAIN
SELECT count(*) FROM functional.alltypessmall a
LEFT JOIN
(SELECT id, upper(string_col) AS upper_val
FROM functional.alltypestiny) b ON a.id=b.id
WHERE b.upper_val='1';
+------------------------------------------------------------+
| Explain String |
+------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=1.95MB Threads=5 |
| Per-Host Resource Estimates: Memory=86MB |
| Codegen disabled by planner |
| |
| PLAN-ROOT SINK |
| | |
| 06:AGGREGATE [FINALIZE] |
| | output: count:merge(*) |
| | row-size=8B cardinality=1 |
| | |
| 05:EXCHANGE [UNPARTITIONED] |
| | |
| 03:AGGREGATE |
| | output: count(*) |
| | row-size=8B cardinality=1 |
| | |
| 02:HASH JOIN [LEFT OUTER JOIN, BROADCAST] |
| | hash predicates: a.id = id |
| | other predicates: upper(string_col) = '1' <-eval after join
| | row-size=21B cardinality=100 |
| | |
| |--04:EXCHANGE [BROADCAST] |
| | | |
| | 01:SCAN HDFS [functional.alltypestiny] |
| | HDFS partitions=4/4 files=4 size=460B |
| | predicates: upper(string_col) = '1' <-push down |
| | row-size=17B cardinality=4 |
| | |
| 00:SCAN HDFS [functional.alltypessmall a] |
| HDFS partitions=4/4 files=4 size=6.32KB |
| row-size=4B cardinality=100 |
+------------------------------------------------------------+
Change-Id: I6c23a45aeb5dd1aa06a95c9aa8628ecbe37ef2c1
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/MultiAggregateInfo.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M testdata/workloads/functional-planner/queries/PlannerTest/inline-view.test
M testdata/workloads/functional-planner/queries/PlannerTest/nested-collections.test
M testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test
6 files changed, 146 insertions(+), 57 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/15047/5
--
To view, visit http://gerrit.cloudera.org:8080/15047
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6c23a45aeb5dd1aa06a95c9aa8628ecbe37ef2c1
Gerrit-Change-Number: 15047
Gerrit-PatchSet: 5
Gerrit-Owner: Xianqing He <he...@126.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Xianqing He <he...@126.com>