[Impala-ASF-CR] IMPALA-5003: Constant propagation in scan nodes and inline views

["Zach Amsden (Code Review)" <ge...@cloudera.org>]

Zach Amsden has uploaded a new patch set (#12).

Change subject: IMPALA-5003: Constant propagation in scan nodes and inline views
......................................................................

IMPALA-5003: Constant propagation in scan nodes and inline views

When conjuncts are pushed into table refs and inline views, they can
be considered for constant progagation within that node.  In certain
cases, we might end up with a FALSE conditional and now we can
convert ScanNodes to EmptySet nodes when that occurs.

I also added an inequality collation phase which is now partially
tested and will combine conjuncts such as a < k1, a < k2  into
a < min(k1, k2), as well as detect equivalence from a >= k, a <= k,
and determine conflicting bounds requirements to be false.

This could be expanded to do analysis against other slotrefs in the
future, but this should probably be saved for another diff.

Testing: Expanded the test cases for the planner to achieve constant
propagation.  Added Kudu, datasource, Hdfs and HBase tests to validate
we can create EmptySetNodes.

Query: explain select * from functional_hbase.widetable_250_cols a
where a.int_col1 > 1 and a.int_col1 <= 20 and a.int_col1 < 50 and
a.int_col1 > 2
+-------------------------------------------------------------------
| Explain String
+-------------------------------------------------------------------
| Estimated Per-Host Requirements: Memory=1.00GB VCores=1
| PLAN-ROOT SINK
| |
| 01:EXCHANGE [UNPARTITIONED]
| |
| 00:SCAN HBASE [functional_hbase.widetable_250_cols a]
|    predicates: a.int_col1 <= 20, a.int_col1 > 2
+-------------------------------------------------------------------
Fetched 10 row(s) in 0.08s

Right now I'm seeing some suspicious test output, specifically the
SCANRANGELOCATIONS seem to be arbitrarily modified:

section SCANRANGELOCATIONS of query:
select * from functional_hbase.alltypesagg
where bigint_col is not null and bool_col = true
Actual does not match expected result:
  HBASE KEYRANGE port=16201 <unbounded>:1
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  HBASE KEYRANGE port=16202 1:5
  HBASE KEYRANGE port=16202 9:<unbounded>
  HBASE KEYRANGE port=16203 5:9
NODE 0:

Expected:
  HBASE KEYRANGE port=16201 <unbounded>:3
  HBASE KEYRANGE port=16202 3:7
  HBASE KEYRANGE port=16203 7:<unbounded>
NODE 0:

Change-Id: I79750a8edb945effee2a519fa3b8192b77042cb4
---
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/Expr.java
M fe/src/main/java/org/apache/impala/analysis/SelectList.java
M fe/src/main/java/org/apache/impala/planner/DataSourceScanNode.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M fe/src/main/java/org/apache/impala/planner/ValueRange.java
M fe/src/main/java/org/apache/impala/rewrite/ExprRewriter.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
M testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test
M testdata/workloads/functional-planner/queries/PlannerTest/conjunct-ordering.test
A testdata/workloads/functional-planner/queries/PlannerTest/constant-propagation.test
M testdata/workloads/functional-planner/queries/PlannerTest/data-source-tables.test
M testdata/workloads/functional-planner/queries/PlannerTest/hbase.test
M testdata/workloads/functional-planner/queries/PlannerTest/hdfs.test
M testdata/workloads/functional-planner/queries/PlannerTest/inline-view-limit.test
M testdata/workloads/functional-planner/queries/PlannerTest/joins.test
M testdata/workloads/functional-planner/queries/PlannerTest/kudu.test
M testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test
M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test
21 files changed, 640 insertions(+), 128 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/6389/12
-- 
To view, visit http://gerrit.cloudera.org:8080/6389
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I79750a8edb945effee2a519fa3b8192b77042cb4
Gerrit-PatchSet: 12
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Zach Amsden <za...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <ma...@cloudera.com>
Gerrit-Reviewer: Zach Amsden <za...@cloudera.com>