[Impala-ASF-CR] IMPALA-14092: Part3: Enable predicate pushdown for Paimon tables.

["ji chen (Code Review)" <ge...@cloudera.org>]

ji chen has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/23793 )

Change subject: IMPALA-14092: Part3: Enable predicate pushdown for Paimon tables.
......................................................................

IMPALA-14092: Part3: Enable predicate pushdown for Paimon tables.

This patch mainly enable the predicate pushdown feature for paimon
table. It will analyze the filter predicates for the given paimon table,
and extract the predicates that can be pushed down to paimon.
these predicate will be applied in-advance to paimon table
to exclude unnecessary partitions,data files for the scanning.

Remap pushed predicates to the JNI projected read schema and always
re-evaluate pushed conjuncts in Impala post-scan for correctness while
using Paimon filters for split/file pruning.

Testing:
- Add test_paimon_predicate_pushdown in test_paimon.py.
- Add testPaimonPredicate in PlannerTest.java
- Add unit test PaimonImpalaExprConverterTest.java
- All query results are same for tpcds queries against paimon table
   before/after enabing paimon predicate push down.
Change-Id: Iee07fa35de8381121a20b2976d6424626d705483
---
M be/src/exec/paimon/paimon-jni-scan-node.cc
M be/src/scheduling/scheduler.cc
M common/protobuf/planner.proto
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/planner/PaimonScanNode.java
A fe/src/main/java/org/apache/impala/planner/paimon/ImpalaExprConverter.java
A fe/src/main/java/org/apache/impala/planner/paimon/ImpalaPaimonRowTypeUtils.java
A fe/src/main/java/org/apache/impala/planner/paimon/PaimonPartitionPredicateVisitor.java
A fe/src/main/java/org/apache/impala/planner/paimon/PaimonReadFilterBuilder.java
M fe/src/main/java/org/apache/impala/planner/paimon/PaimonSplit.java
A fe/src/main/java/org/apache/impala/planner/paimon/PredicateExtractor.java
M fe/src/main/java/org/apache/impala/util/paimon/PaimonJniScanner.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
A fe/src/test/java/org/apache/impala/planner/paimon/PaimonImpalaExprConverterTest.java
A fe/src/test/java/org/apache/impala/planner/paimon/PaimonReadFilterBuilderTest.java
A testdata/workloads/functional-planner/queries/PlannerTest/paimon-predicates.test
A testdata/workloads/functional-query/queries/QueryTest/paimon-predicate-push-down.test
M tests/query_test/test_paimon.py
18 files changed, 2,533 insertions(+), 44 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/23793/11
-- 
To view, visit http://gerrit.cloudera.org:8080/23793
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iee07fa35de8381121a20b2976d6424626d705483
Gerrit-Change-Number: 23793
Gerrit-PatchSet: 11
Gerrit-Owner: ji chen <ji...@163.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: ji chen <ji...@163.com>