You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Riza Suminto (Code Review)" <ge...@cloudera.org> on 2023/05/04 22:10:33 UTC
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19843
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Revert "IMPALA-11123: Optimize count(star) for ORC scans"
This reverts commit f932d78ad0a30e322d59fc39072f710f889d2135.
The commit is reverted because it cause significant regression for
non-optimized counts star query in parquet format.
Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
---
M be/src/exec/hdfs-columnar-scanner.cc
M be/src/exec/hdfs-columnar-scanner.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/hdfs-scanner.cc
M be/src/exec/orc/hdfs-orc-scanner.cc
M be/src/exec/orc/hdfs-orc-scanner.h
M be/src/exec/orc/orc-column-readers.h
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M be/src/exec/parquet/hdfs-parquet-scanner.h
M be/src/exec/parquet/parquet-column-readers.h
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
D testdata/workloads/functional-planner/queries/PlannerTest/orc-stats-agg.test
M testdata/workloads/functional-planner/queries/PlannerTest/parquet-stats-agg.test
M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-compound-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-is-null-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-upper-lower-bound-metrics.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes-orc.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes.test
M testdata/workloads/functional-query/queries/QueryTest/mixed-format.test
D testdata/workloads/functional-query/queries/QueryTest/orc-stats-agg.test
M testdata/workloads/functional-query/queries/QueryTest/parquet-stats-agg.test
M testdata/workloads/functional-query/queries/QueryTest/partition-key-scans.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_filters.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_filters_mt_dop.test
M testdata/workloads/functional-query/queries/QueryTest/scanners.test
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_query_retries.py
M tests/query_test/test_aggregation.py
M tests/query_test/test_iceberg.py
M tests/query_test/test_scanners.py
M tests/util/test_file_parser.py
39 files changed, 255 insertions(+), 1,100 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/19843/1
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 2:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19843/2/tests/query_test/test_aggregation.py
File tests/query_test/test_aggregation.py:
http://gerrit.cloudera.org:8080/#/c/19843/2/tests/query_test/test_aggregation.py@264
PS2, Line 264: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/2/tests/query_test/test_aggregation.py@273
PS2, Line 273: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/2/tests/query_test/test_aggregation.py@331
PS2, Line 331: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/2/tests/util/test_file_parser.py
File tests/util/test_file_parser.py:
http://gerrit.cloudera.org:8080/#/c/19843/2/tests/util/test_file_parser.py@269
PS2, Line 269: ,
flake8: W602 deprecated form of raising exception
http://gerrit.cloudera.org:8080/#/c/19843/2/tests/util/test_file_parser.py@273
PS2, Line 273: ,
flake8: W602 deprecated form of raising exception
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 17:15:51 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Revert "IMPALA-11123: Optimize count(star) for ORC scans"
This reverts commit f932d78ad0a30e322d59fc39072f710f889d2135.
The commit is reverted because it cause significant regression for
non-optimized counts star query in parquet format.
There are several conflicts that need to be resolved manually:
- Removed assertion against 'NumFileMetadataRead' counter that is lost
with the revert.
- Adjust the assertion in test_plain_count_star_optimization,
test_in_predicate_push_down, and test_partitioned_insert of
test_iceberg.py due to missing improvement in parquet optimized count
star code path.
- Keep the "override" specifier in hdfs-parquet-scanner.h to pass
clang-tidy
- Keep python3 style of RuntimeError instantiation in
test_file_parser.py to pass check-python-syntax.sh
Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Reviewed-on: http://gerrit.cloudera.org:8080/19843
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Wenzhe Zhou <wz...@cloudera.com>
---
M be/src/exec/hdfs-columnar-scanner.cc
M be/src/exec/hdfs-columnar-scanner.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/hdfs-scanner.cc
M be/src/exec/orc/hdfs-orc-scanner.cc
M be/src/exec/orc/hdfs-orc-scanner.h
M be/src/exec/orc/orc-column-readers.h
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M be/src/exec/parquet/hdfs-parquet-scanner.h
M be/src/exec/parquet/parquet-column-readers.h
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
D testdata/workloads/functional-planner/queries/PlannerTest/orc-stats-agg.test
M testdata/workloads/functional-planner/queries/PlannerTest/parquet-stats-agg.test
M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-compound-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-is-null-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-upper-lower-bound-metrics.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes-orc.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes.test
M testdata/workloads/functional-query/queries/QueryTest/mixed-format.test
D testdata/workloads/functional-query/queries/QueryTest/orc-stats-agg.test
M testdata/workloads/functional-query/queries/QueryTest/parquet-stats-agg.test
M testdata/workloads/functional-query/queries/QueryTest/partition-key-scans.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_filters.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_filters_mt_dop.test
M testdata/workloads/functional-query/queries/QueryTest/scanners.test
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_query_retries.py
M tests/query_test/test_aggregation.py
M tests/query_test/test_iceberg.py
M tests/query_test/test_scanners.py
M tests/util/test_file_parser.py
39 files changed, 269 insertions(+), 1,104 deletions(-)
Approvals:
Impala Public Jenkins: Verified
Wenzhe Zhou: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 2:
> Patch Set 2:
>
> Build Failed
>
> https://jenkins.impala.io/job/gerrit-code-review-checks/12951/ : Initial code review checks failed. See linked job for details on the failure.
This failed python3 compliance
15:30:35 Python 3 compilation failed for tests/util/test_file_parser.py:
15:30:51 *** Error compiling 'tests/util/test_file_parser.py'...
15:30:51 File "tests/util/test_file_parser.py", line 269
15:30:51 raise RuntimeError, 'RUNTIME_PROFILE comment (%s) must be of the form ' \
15:30:51 ^
15:30:51 SyntaxError: invalid syntax
tests/util/test_file_parser.py is another one that can't be reverted.
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 06 May 2023 04:33:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 4:
The syntax error is on patch set 3. Restart GVO now.
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 06 May 2023 16:34:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 3:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19843/3/tests/query_test/test_aggregation.py
File tests/query_test/test_aggregation.py:
http://gerrit.cloudera.org:8080/#/c/19843/3/tests/query_test/test_aggregation.py@264
PS3, Line 264: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/3/tests/query_test/test_aggregation.py@273
PS3, Line 273: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/3/tests/query_test/test_aggregation.py@331
PS3, Line 331: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/3/tests/util/test_file_parser.py
File tests/util/test_file_parser.py:
http://gerrit.cloudera.org:8080/#/c/19843/3/tests/util/test_file_parser.py@269
PS3, Line 269: ,
flake8: W602 deprecated form of raising exception
http://gerrit.cloudera.org:8080/#/c/19843/3/tests/util/test_file_parser.py@273
PS3, Line 273: ,
flake8: W602 deprecated form of raising exception
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 18:19:54 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 4:
> Patch Set 4:
>
> tests/util/test_file_parser.py failed to pass python syntax check in line 269.
Hi Wenzhe, is the syntax error still there in patch set 4?
In patch set 4, I run both ./bin/check-python-syntax.sh and ./bin/check-pylint-py3k.sh and found no error.
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 06 May 2023 14:28:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12952/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 18:39:33 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 4: Code-Review+2
carry +1 from Michael
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 06 May 2023 22:55:03 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 1:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19843/1/tests/query_test/test_aggregation.py
File tests/query_test/test_aggregation.py:
http://gerrit.cloudera.org:8080/#/c/19843/1/tests/query_test/test_aggregation.py@264
PS1, Line 264: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/1/tests/query_test/test_aggregation.py@273
PS1, Line 273: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/1/tests/query_test/test_aggregation.py@331
PS1, Line 331: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/1/tests/util/test_file_parser.py
File tests/util/test_file_parser.py:
http://gerrit.cloudera.org:8080/#/c/19843/1/tests/util/test_file_parser.py@269
PS1, Line 269: ,
flake8: W602 deprecated form of raising exception
http://gerrit.cloudera.org:8080/#/c/19843/1/tests/util/test_file_parser.py@273
PS1, Line 273: ,
flake8: W602 deprecated form of raising exception
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 04 May 2023 22:11:25 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 1:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/12939/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 04 May 2023 22:31:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 1:
Could you core test?
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Thu, 04 May 2023 22:16:47 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Kurt Deschler (Code Review)" <ge...@cloudera.org>.
Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 3:
Do we need to be careful with the description so as not to lose track of the partial revert?
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 22:54:06 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Kurt Deschler, Gabor Kaszab, Wenzhe Zhou, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19843
to look at the new patch set (#4).
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Revert "IMPALA-11123: Optimize count(star) for ORC scans"
This reverts commit f932d78ad0a30e322d59fc39072f710f889d2135.
The commit is reverted because it cause significant regression for
non-optimized counts star query in parquet format.
There are several conflicts that need to be resolved manually:
- Removed assertion against 'NumFileMetadataRead' counter that is lost
with the revert.
- Adjust the assertion in test_plain_count_star_optimization,
test_in_predicate_push_down, and test_partitioned_insert of
test_iceberg.py due to missing improvement in parquet optimized count
star code path.
- Keep the "override" specifier in hdfs-parquet-scanner.h to pass
clang-tidy
- Keep python3 style of RuntimeError instantiation in
test_file_parser.py to pass check-python-syntax.sh
Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
---
M be/src/exec/hdfs-columnar-scanner.cc
M be/src/exec/hdfs-columnar-scanner.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/hdfs-scanner.cc
M be/src/exec/orc/hdfs-orc-scanner.cc
M be/src/exec/orc/hdfs-orc-scanner.h
M be/src/exec/orc/orc-column-readers.h
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M be/src/exec/parquet/hdfs-parquet-scanner.h
M be/src/exec/parquet/parquet-column-readers.h
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
D testdata/workloads/functional-planner/queries/PlannerTest/orc-stats-agg.test
M testdata/workloads/functional-planner/queries/PlannerTest/parquet-stats-agg.test
M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-compound-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-is-null-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-upper-lower-bound-metrics.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes-orc.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes.test
M testdata/workloads/functional-query/queries/QueryTest/mixed-format.test
D testdata/workloads/functional-query/queries/QueryTest/orc-stats-agg.test
M testdata/workloads/functional-query/queries/QueryTest/parquet-stats-agg.test
M testdata/workloads/functional-query/queries/QueryTest/partition-key-scans.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_filters.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_filters_mt_dop.test
M testdata/workloads/functional-query/queries/QueryTest/scanners.test
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_query_retries.py
M tests/query_test/test_aggregation.py
M tests/query_test/test_iceberg.py
M tests/query_test/test_scanners.py
M tests/util/test_file_parser.py
39 files changed, 269 insertions(+), 1,104 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/19843/4
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 2:
Noticed some test behavior change after the revert.
Patch set 2 adjust the assertion in test_plain_count_star_optimization, test_in_predicate_push_down, and test_partitioned_insert of test_iceberg.py.
This should pass the core tests suite.
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 17:16:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12957/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 06 May 2023 05:28:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 2:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/12951/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 17:36:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9292/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 06 May 2023 16:35:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 3: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 22:18:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 4:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/19843/4/tests/query_test/test_aggregation.py
File tests/query_test/test_aggregation.py:
http://gerrit.cloudera.org:8080/#/c/19843/4/tests/query_test/test_aggregation.py@264
PS4, Line 264: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/4/tests/query_test/test_aggregation.py@273
PS4, Line 273: o
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19843/4/tests/query_test/test_aggregation.py@331
PS4, Line 331: o
flake8: W504 line break after binary operator
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 06 May 2023 05:08:31 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 3:
The original IMPALA-11123 patch add some "override" specifier in hdfs-parquet-scanner.h
clang-tidy complains when those specifier removed along with the revert. So patch set 3 put the "override" specifier back in hdfs-parquet-scanner.h
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 18:47:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 3: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/9290/
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 06 May 2023 03:47:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 3:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9290/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 22:25:01 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Gabor Kaszab, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19843
to look at the new patch set (#2).
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Revert "IMPALA-11123: Optimize count(star) for ORC scans"
This reverts commit f932d78ad0a30e322d59fc39072f710f889d2135.
The commit is reverted because it cause significant regression for
non-optimized counts star query in parquet format.
Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
---
M be/src/exec/hdfs-columnar-scanner.cc
M be/src/exec/hdfs-columnar-scanner.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/hdfs-scanner.cc
M be/src/exec/orc/hdfs-orc-scanner.cc
M be/src/exec/orc/hdfs-orc-scanner.h
M be/src/exec/orc/orc-column-readers.h
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M be/src/exec/parquet/hdfs-parquet-scanner.h
M be/src/exec/parquet/parquet-column-readers.h
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
D testdata/workloads/functional-planner/queries/PlannerTest/orc-stats-agg.test
M testdata/workloads/functional-planner/queries/PlannerTest/parquet-stats-agg.test
M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-compound-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-is-null-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-upper-lower-bound-metrics.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes-orc.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes.test
M testdata/workloads/functional-query/queries/QueryTest/mixed-format.test
D testdata/workloads/functional-query/queries/QueryTest/orc-stats-agg.test
M testdata/workloads/functional-query/queries/QueryTest/parquet-stats-agg.test
M testdata/workloads/functional-query/queries/QueryTest/partition-key-scans.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_filters.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_filters_mt_dop.test
M testdata/workloads/functional-query/queries/QueryTest/scanners.test
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_query_retries.py
M tests/query_test/test_aggregation.py
M tests/query_test/test_iceberg.py
M tests/query_test/test_scanners.py
M tests/util/test_file_parser.py
39 files changed, 275 insertions(+), 1,110 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/19843/2
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Gabor Kaszab, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19843
to look at the new patch set (#3).
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Revert "IMPALA-11123: Optimize count(star) for ORC scans"
This reverts commit f932d78ad0a30e322d59fc39072f710f889d2135.
The commit is reverted because it cause significant regression for
non-optimized counts star query in parquet format.
Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
---
M be/src/exec/hdfs-columnar-scanner.cc
M be/src/exec/hdfs-columnar-scanner.h
M be/src/exec/hdfs-scan-node-base.cc
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/hdfs-scanner.cc
M be/src/exec/orc/hdfs-orc-scanner.cc
M be/src/exec/orc/hdfs-orc-scanner.h
M be/src/exec/orc/orc-column-readers.h
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M be/src/exec/parquet/hdfs-parquet-scanner.h
M be/src/exec/parquet/parquet-column-readers.h
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
D testdata/workloads/functional-planner/queries/PlannerTest/orc-stats-agg.test
M testdata/workloads/functional-planner/queries/PlannerTest/parquet-stats-agg.test
M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-compound-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-is-null-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-upper-lower-bound-metrics.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes-orc.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes.test
M testdata/workloads/functional-query/queries/QueryTest/mixed-format.test
D testdata/workloads/functional-query/queries/QueryTest/orc-stats-agg.test
M testdata/workloads/functional-query/queries/QueryTest/parquet-stats-agg.test
M testdata/workloads/functional-query/queries/QueryTest/partition-key-scans.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_filters.test
M testdata/workloads/functional-query/queries/QueryTest/runtime_filters_mt_dop.test
M testdata/workloads/functional-query/queries/QueryTest/scanners.test
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_query_retries.py
M tests/query_test/test_aggregation.py
M tests/query_test/test_iceberg.py
M tests/query_test/test_scanners.py
M tests/util/test_file_parser.py
39 files changed, 270 insertions(+), 1,105 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/43/19843/3
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 3: Code-Review+1
It'd be helpful to note things you needed to resolve in the commit message.
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Fri, 05 May 2023 23:11:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 06 May 2023 21:49:46 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Revert "IMPALA-11123: Optimize count(star) for ORC scans"
Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19843 )
Change subject: Revert "IMPALA-11123: Optimize count(star) for ORC scans"
......................................................................
Patch Set 4:
tests/util/test_file_parser.py failed to pass python syntax check in line 269.
--
To view, visit http://gerrit.cloudera.org:8080/19843
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iefd8fd0838638f9db146f7b706e541fe2aaf01c1
Gerrit-Change-Number: 19843
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 06 May 2023 05:30:59 +0000
Gerrit-HasComments: No