You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Peter Rozsa (Code Review)" <ge...@cloudera.org> on 2024/01/09 15:39:36 UTC
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Peter Rozsa has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20874
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Addendum: IMPALA-12584: Enable strict data file access by default
This change sets the default value to 'true' for
'iceberg_restrict_data_file_location'.
Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
---
M be/src/common/global-flags.cc
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/20874/1
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 1
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14915/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 2
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Jan 2024 14:16:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 10: Code-Review+2
(5 comments)
Found a few nits, otherwise LGTM!
http://gerrit.cloudera.org:8080/#/c/20874/10//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20874/10//COMMIT_MSG@11
PS10, Line 11: T
nit: no space
http://gerrit.cloudera.org:8080/#/c/20874/10//COMMIT_MSG@14
PS10, Line 14:
You could mention that during data loading we disable the location check.
http://gerrit.cloudera.org:8080/#/c/20874/10/be/src/util/backend-gflag-util.cc
File be/src/util/backend-gflag-util.cc:
http://gerrit.cloudera.org:8080/#/c/20874/10/be/src/util/backend-gflag-util.cc@447
PS10, Line 447:
nit: indentation is off
http://gerrit.cloudera.org:8080/#/c/20874/10/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/20874/10/fe/src/main/java/org/apache/impala/service/BackendConfig.java@435
PS10, Line 435: public void setIcebergAllowDatafileInTableLocationOnly(boolean flag) {
nit: please add empty line before this
http://gerrit.cloudera.org:8080/#/c/20874/10/tests/custom_cluster/test_iceberg_strict_data.py
File tests/custom_cluster/test_iceberg_strict_data.py:
http://gerrit.cloudera.org:8080/#/c/20874/10/tests/custom_cluster/test_iceberg_strict_data.py@55
PS10, Line 55: vector)
nit: fits earlier line
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 10
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 15:01:59 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 8: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 8
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 15:28:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14919/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Jan 2024 16:59:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20874/6/tests/custom_cluster/test_iceberg_strict_data.py
File tests/custom_cluster/test_iceberg_strict_data.py:
http://gerrit.cloudera.org:8080/#/c/20874/6/tests/custom_cluster/test_iceberg_strict_data.py@54
PS6, Line 54:
flake8: W292 no newline at end of file
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 14:44:26 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 7:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14963/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 7
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 15:11:47 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20874/5/tests/custom_cluster/test_iceberg_strict_data.py
File tests/custom_cluster/test_iceberg_strict_data.py:
http://gerrit.cloudera.org:8080/#/c/20874/5/tests/custom_cluster/test_iceberg_strict_data.py@56
PS5, Line 56: unique_database
There are only SELECT queries in iceberg-multiple-storage-locations-table.test, so we don't need a unique database for this test.
Custom cluster tests take some time to start an Impala cluster, also these tests cannot run in parallel. So I think it would make sense to merge this test with 'test_disabled'.
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 10:00:00 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Addendum: IMPALA-12584: Enable strict data file access by default
This change sets the default value to 'true' for
'iceberg_restrict_data_file_location' and changes the flag name to
'iceberg_allow_datafiles_in_table_location_only'.Tests related to
multiple storage locations in Iceberg tables are moved out to custom
cluster tests.
Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M tests/custom_cluster/test_iceberg_strict_data.py
M tests/query_test/test_iceberg.py
7 files changed, 15 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/20874/7
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 7
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Addendum: IMPALA-12584: Enable strict data file access by default
This change sets the default value to 'true' for
'iceberg_restrict_data_file_location' and changes the flag name to
'iceberg_allow_datafiles_in_table_location_only'.Tests related to
multiple storage locations in Iceberg tables are moved out to custom
cluster tests.
Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java
M testdata/bin/create-load-data.sh
M tests/custom_cluster/test_iceberg_strict_data.py
M tests/query_test/test_iceberg.py
9 files changed, 31 insertions(+), 24 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/20874/9
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 9
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Addendum: IMPALA-12584: Enable strict data file access by default
This change sets the default value to 'true' for
'iceberg_restrict_data_file_location' and changes the flag name to
'iceberg_allow_datafiles_in_table_location_only'. Tests related to
multiple storage locations in Iceberg tables are moved out to custom
cluster tests. During test data loading, the flag is set to 'false'
to make the creation of 'iceberg_multiple_storage_locations' table
possible.
Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java
M testdata/bin/create-load-data.sh
M tests/custom_cluster/test_iceberg_strict_data.py
M tests/query_test/test_iceberg.py
9 files changed, 32 insertions(+), 24 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/20874/11
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 11
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 12: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 12
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 19:53:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20874/5/tests/custom_cluster/test_iceberg_strict_data.py
File tests/custom_cluster/test_iceberg_strict_data.py:
http://gerrit.cloudera.org:8080/#/c/20874/5/tests/custom_cluster/test_iceberg_strict_data.py@56
PS5, Line 56:
> There are only SELECT queries in iceberg-multiple-storage-locations-table.t
Done
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 14:43:25 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14962/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 15:11:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 12:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10178/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 12
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 15:15:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 9:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20874/9/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:
http://gerrit.cloudera.org:8080/#/c/20874/9/testdata/bin/create-load-data.sh@169
PS9, Line 169: START_CLUSTER_ARGS_INT+=("--catalogd_args=-iceberg_allow_datafiles_in_table_location_only=false")
line too long (99 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 9
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 08:58:44 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 7: Code-Review+2
Thanks for working on this! LGTM!
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 7
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 15:27:47 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 12: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 12
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 15:15:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 3: Code-Review+2
Thanks for fixing this, LGTM!
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Jan 2024 14:19:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20874/2/be/src/common/global-flags.cc
File be/src/common/global-flags.cc:
http://gerrit.cloudera.org:8080/#/c/20874/2/be/src/common/global-flags.cc@424
PS2, Line 424: DEFINE_bool(iceberg_allow_data_in_table_location_only, true, "If true, Impala does not allow "
line too long (94 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 2
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Jan 2024 13:50:32 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Addendum: IMPALA-12584: Enable strict data file access by default
This change sets the default value to 'true' for
'iceberg_restrict_data_file_location' and changes the flag name to
'iceberg_allow_datafiles_in_table_location_only'.Tests related to
multiple storage locations in Iceberg tables are moved out to custom
cluster tests.
Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M tests/custom_cluster/test_iceberg_strict_data.py
M tests/query_test/test_iceberg.py
7 files changed, 19 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/20874/5
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Addendum: IMPALA-12584: Enable strict data file access by default
This change sets the default value to 'true' for
'iceberg_restrict_data_file_location' and changes the flag name to
'iceberg_allow_datafiles_in_table_location_only'.Tests related to
multiple storage locations in Iceberg tables are moved out to custom
cluster tests.
Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M tests/custom_cluster/test_iceberg_strict_data.py
M tests/query_test/test_iceberg.py
7 files changed, 15 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/20874/6
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 6
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 8:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10158/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 8
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 15:28:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 11:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15002/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 11
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 15:40:53 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Addendum: IMPALA-12584: Enable strict data file access by default
This change sets the default value to 'true' for
'iceberg_restrict_data_file_location' and changes the flag name to
'iceberg_allow_datafiles_in_table_location_only'. Tests related to
multiple storage locations in Iceberg tables are moved out to custom
cluster tests. During test data loading, the flag is set to 'false'
to make the creation of 'iceberg_multiple_storage_locations' table
possible.
Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Reviewed-on: http://gerrit.cloudera.org:8080/20874
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java
M testdata/bin/create-load-data.sh
M tests/custom_cluster/test_iceberg_strict_data.py
M tests/query_test/test_iceberg.py
9 files changed, 32 insertions(+), 24 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 13
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 5:
(1 comment)
> Patch Set 4:
>
> (1 comment)
Thank you Zoltan for catching the mistake!
http://gerrit.cloudera.org:8080/#/c/20874/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20874/4//COMMIT_MSG@10
PS4, Line 10: iceberg_restrict_data_file_location
> Hive actually uses 'hive.iceberg.allow.datafiles.in.table.location.only':
Done
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 5
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Jan 2024 16:32:37 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Jan 2024 14:19:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Addendum: IMPALA-12584: Enable strict data file access by default
This change sets the default value to 'true' for
'iceberg_restrict_data_file_location' and changes the flag name to
'iceberg_allow_datafiles_in_table_location_only'.Tests related to
multiple storage locations in Iceberg tables are moved out to custom
cluster tests.
Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java
M testdata/bin/create-load-data.sh
M tests/custom_cluster/test_iceberg_strict_data.py
M tests/query_test/test_iceberg.py
9 files changed, 32 insertions(+), 24 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/20874/10
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 10
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 11: Code-Review+2
Thanks for fixing this, LGTM!
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 11
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 15:14:53 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 10:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15001/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 10
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 10:37:47 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10135/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Jan 2024 14:19:19 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20874/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20874/4//COMMIT_MSG@10
PS4, Line 10: iceberg_restrict_data_file_location
Hive actually uses 'hive.iceberg.allow.datafiles.in.table.location.only':
https://github.com/apache/hive/pull/4910/files
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Jan 2024 15:55:17 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has removed a vote on this change.
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Removed Code-Review+2 by Impala Public Jenkins <im...@cloudera.com>
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 4
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Addendum: IMPALA-12584: Enable strict data file access by default
This change sets the default value to 'true' for
'iceberg_restrict_data_file_location' and changes the flag name to
'iceberg_allow_data_in_table_location_only'. Turning on the flag by
default, tests regarding multiple storage locations in Iceberg tables
are moved out to custom cluster tests.
Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M tests/custom_cluster/test_iceberg_strict_data.py
M tests/query_test/test_iceberg.py
7 files changed, 19 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/20874/3
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 8: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10158/
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 8
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 16 Jan 2024 17:38:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 9:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15000/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 9
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 09:24:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 11:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/20874/10//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20874/10//COMMIT_MSG@11
PS10, Line 11:
> nit: no space
Done
http://gerrit.cloudera.org:8080/#/c/20874/10//COMMIT_MSG@14
PS10, Line 14: to make the creation of 'iceberg_multiple_storage_locations' table
> You could mention that during data loading we disable the location check.
Done
http://gerrit.cloudera.org:8080/#/c/20874/10/be/src/util/backend-gflag-util.cc
File be/src/util/backend-gflag-util.cc:
http://gerrit.cloudera.org:8080/#/c/20874/10/be/src/util/backend-gflag-util.cc@447
PS10, Line 447:
> nit: indentation is off
Done
http://gerrit.cloudera.org:8080/#/c/20874/10/fe/src/main/java/org/apache/impala/service/BackendConfig.java
File fe/src/main/java/org/apache/impala/service/BackendConfig.java:
http://gerrit.cloudera.org:8080/#/c/20874/10/fe/src/main/java/org/apache/impala/service/BackendConfig.java@435
PS10, Line 435:
> nit: please add empty line before this
Done
http://gerrit.cloudera.org:8080/#/c/20874/10/tests/custom_cluster/test_iceberg_strict_data.py
File tests/custom_cluster/test_iceberg_strict_data.py:
http://gerrit.cloudera.org:8080/#/c/20874/10/tests/custom_cluster/test_iceberg_strict_data.py@55
PS10, Line 55:
> nit: fits earlier line
Done
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 11
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 15:13:26 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 10:
(1 comment)
> Patch Set 9:
>
> (1 comment)
http://gerrit.cloudera.org:8080/#/c/20874/9/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:
http://gerrit.cloudera.org:8080/#/c/20874/9/testdata/bin/create-load-data.sh@169
PS9, Line 169: DATAFILE_LOCATION_CHECK="-iceberg_allow_datafiles_in_table_location_only=false"
> line too long (99 > 90)
Done
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 10
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 19 Jan 2024 10:10:45 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14904/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 1
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 09 Jan 2024 16:06:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Peter Rozsa (Code Review)" <ge...@cloudera.org>.
Peter Rozsa has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Addendum: IMPALA-12584: Enable strict data file access by default
This change sets the default value to 'true' for
'iceberg_restrict_data_file_location' and changes the flag name to
'iceberg_allow_data_in_table_location_only'. Turning on the flag by
default, tests regarding multiple storage locations in Iceberg tables
are moved out to custom cluster tests.
Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
---
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M tests/custom_cluster/test_iceberg_strict_data.py
M tests/query_test/test_iceberg.py
7 files changed, 17 insertions(+), 15 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/20874/2
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 2
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] Addendum: IMPALA-12584: Enable strict data file access by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20874 )
Change subject: Addendum: IMPALA-12584: Enable strict data file access by default
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/14916/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ifec84c86132a8a44d7e161006dcf51be2e7c7e57
Gerrit-Change-Number: 20874
Gerrit-PatchSet: 3
Gerrit-Owner: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Peter Rozsa <pr...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 10 Jan 2024 14:20:06 +0000
Gerrit-HasComments: No