You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Riza Suminto (Code Review)" <ge...@cloudera.org> on 2023/05/07 02:17:10 UTC

[Impala-ASF-CR] IMPALA-11123: Restore NumFileMetadataRead counter

Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19854


Change subject: IMPALA-11123: Restore NumFileMetadataRead counter
......................................................................

IMPALA-11123: Restore NumFileMetadataRead counter

NumFileMetadataRead counter was lost with the revert of commit
f932d78ad0a30e322d59fc39072f710f889d2135. This patch restore
NumFileMetadataRead counter and also assertions in impacted iceberg test
files. Other impacted test files wll be gradually restored with
reimplementation of optimized count star for ORC.

Testing:
- Pass core tests.

Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
---
M be/src/exec/hdfs-columnar-scanner.cc
M be/src/exec/hdfs-columnar-scanner.h
M be/src/exec/orc/hdfs-orc-scanner.cc
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M testdata/workloads/functional-query/queries/QueryTest/iceberg-compound-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-is-null-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-upper-lower-bound-metrics.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes-orc.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes.test
M tests/query_test/test_iceberg.py
14 files changed, 72 insertions(+), 2 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/19854/1
-- 
To view, visit http://gerrit.cloudera.org:8080/19854
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
Gerrit-Change-Number: 19854
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>

[Impala-ASF-CR] IMPALA-11123: Restore NumFileMetadataRead counter

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19854 )

Change subject: IMPALA-11123: Restore NumFileMetadataRead counter
......................................................................

IMPALA-11123: Restore NumFileMetadataRead counter

NumFileMetadataRead counter was lost with the revert of commit
f932d78ad0a30e322d59fc39072f710f889d2135. This patch restore
NumFileMetadataRead counter and also assertions in impacted iceberg test
files. Other impacted test files will be gradually restored with
reimplementation of optimized count star for ORC.

Testing:
- Pass core tests.

Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
Reviewed-on: http://gerrit.cloudera.org:8080/19854
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exec/hdfs-columnar-scanner.cc
M be/src/exec/hdfs-columnar-scanner.h
M be/src/exec/orc/hdfs-orc-scanner.cc
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M testdata/workloads/functional-query/queries/QueryTest/iceberg-compound-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-is-null-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-upper-lower-bound-metrics.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes-orc.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes.test
M tests/query_test/test_iceberg.py
14 files changed, 72 insertions(+), 2 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/19854
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
Gerrit-Change-Number: 19854
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-11123: Restore NumFileMetadataRead counter

Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Gabor Kaszab, Wenzhe Zhou, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19854

to look at the new patch set (#2).

Change subject: IMPALA-11123: Restore NumFileMetadataRead counter
......................................................................

IMPALA-11123: Restore NumFileMetadataRead counter

NumFileMetadataRead counter was lost with the revert of commit
f932d78ad0a30e322d59fc39072f710f889d2135. This patch restore
NumFileMetadataRead counter and also assertions in impacted iceberg test
files. Other impacted test files will be gradually restored with
reimplementation of optimized count star for ORC.

Testing:
- Pass core tests.

Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
---
M be/src/exec/hdfs-columnar-scanner.cc
M be/src/exec/hdfs-columnar-scanner.h
M be/src/exec/orc/hdfs-orc-scanner.cc
M be/src/exec/parquet/hdfs-parquet-scanner.cc
M testdata/workloads/functional-query/queries/QueryTest/iceberg-compound-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-in-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-is-null-predicate-push-down.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-upper-lower-bound-metrics.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-plain-count-star-optimization.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes-orc.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes.test
M tests/query_test/test_iceberg.py
14 files changed, 72 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/19854/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19854
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
Gerrit-Change-Number: 19854
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-11123: Restore NumFileMetadataRead counter

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19854 )

Change subject: IMPALA-11123: Restore NumFileMetadataRead counter
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/19854
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
Gerrit-Change-Number: 19854
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 May 2023 18:53:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11123: Restore NumFileMetadataRead counter

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19854 )

Change subject: IMPALA-11123: Restore NumFileMetadataRead counter
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19854
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
Gerrit-Change-Number: 19854
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 May 2023 00:05:35 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11123: Restore NumFileMetadataRead counter

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19854 )

Change subject: IMPALA-11123: Restore NumFileMetadataRead counter
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9296/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/19854
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
Gerrit-Change-Number: 19854
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 May 2023 18:53:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11123: Restore NumFileMetadataRead counter

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19854 )

Change subject: IMPALA-11123: Restore NumFileMetadataRead counter
......................................................................


Patch Set 2: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19854
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
Gerrit-Change-Number: 19854
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 May 2023 05:07:51 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11123: Restore NumFileMetadataRead counter

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19854 )

Change subject: IMPALA-11123: Restore NumFileMetadataRead counter
......................................................................


Patch Set 2: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/19854
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
Gerrit-Change-Number: 19854
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 May 2023 18:34:35 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11123: Restore NumFileMetadataRead counter

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19854 )

Change subject: IMPALA-11123: Restore NumFileMetadataRead counter
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/12961/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19854
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib14576245d978a127f688e265cab2f4ff519600c
Gerrit-Change-Number: 19854
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 07 May 2023 02:38:08 +0000
Gerrit-HasComments: No