You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tamas Mate (Code Review)" <ge...@cloudera.org> on 2022/10/17 07:19:32 UTC
[Impala-ASF-CR] [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Tamas Mate has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19145
Change subject: [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
[WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Extend LOAD DATA INPATH statement to support Iceberg tables.
Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
---
M be/src/service/client-request-state.cc
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M tests/query_test/test_iceberg.py
5 files changed, 191 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/19145/1
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 1
Gerrit-Owner: Tamas Mate <tm...@apache.org>
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 3:
(7 comments)
Thanks for working on this.
http://gerrit.cloudera.org:8080/#/c/19145/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19145/3//COMMIT_MSG@18
PS3, Line 18:
Please mention the current limitations:
* no single files supported
* no partition clause
Probably with Jira IDs if separate tickets are created for these.
http://gerrit.cloudera.org:8080/#/c/19145/3/be/src/service/client-request-state.cc
File be/src/service/client-request-state.cc:
http://gerrit.cloudera.org:8080/#/c/19145/3/be/src/service/client-request-state.cc@803
PS3, Line 803: __isset
If this is an Iceberg table then all these statements must be set, right? Should we rather add a DCHECK for them?
http://gerrit.cloudera.org:8080/#/c/19145/3/be/src/service/client-request-state.cc@827
PS3, Line 827: No partitions selected for incremental stats update.
Please fix message.
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
File fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java:
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@264
PS3, Line 264: PARQUET
Use the same as the inferred file format. Please also add tests for ORC.
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@301
PS3, Line 301: "Could not infer file format."
> This error message could be misleading in case of a valid but not orc nor p
If the file format cannot be inferred from the filename, then maybe we could read the magic bytes of the file. Or we could try opening the file via Parquet/ORC file readers.
http://gerrit.cloudera.org:8080/#/c/19145/3/testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test
File testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test:
http://gerrit.cloudera.org:8080/#/c/19145/3/testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test@44
PS3, Line 44: ====
Can we have tests for the case when there's a schema mismatch between table and file?
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py@799
PS3, Line 799:
nit: +2 indent needed
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 3
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 14 Nov 2022 15:38:14 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 9:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12028/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 9
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Dec 2022 10:43:43 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 6:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/19145/4/fe/src/main/java/org/apache/impala/service/Frontend.java
File fe/src/main/java/org/apache/impala/service/Frontend.java:
http://gerrit.cloudera.org:8080/#/c/19145/4/fe/src/main/java/org/apache/impala/service/Frontend.java@941
PS4, Line 941: private TLoadDataResp doLoadIcebergTableData(TLoadDataReq request)
> line too long (92 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/19145/4/fe/src/main/java/org/apache/impala/service/Frontend.java@974
PS4, Line 974:
> got it.
Whops, yes, I did not notice that I left the second parameter there. Done.
http://gerrit.cloudera.org:8080/#/c/19145/4/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:
http://gerrit.cloudera.org:8080/#/c/19145/4/tests/query_test/test_iceberg.py@825
PS4, Line 825:
> flake8: E502 the backslash is redundant between brackets
Done
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 6
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Dec 2022 13:50:38 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 3:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py@794
PS3, Line 794: @
flake8: E301 expected 1 blank line, found 0
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py@797
PS3, Line 797: ;
flake8: E703 statement ends with a semicolon
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py@799
PS3, Line 799: ;
flake8: E703 statement ends with a semicolon
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py@801
PS3, Line 801: ;
flake8: E703 statement ends with a semicolon
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py@803
PS3, Line 803: ;
flake8: E703 statement ends with a semicolon
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py@804
PS3, Line 804: +
flake8: W504 line break after binary operator
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py@806
PS3, Line 806: ;
flake8: E703 statement ends with a semicolon
http://gerrit.cloudera.org:8080/#/c/19145/3/tests/query_test/test_iceberg.py@810
PS3, Line 810: ;
flake8: E703 statement ends with a semicolon
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 3
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 01 Nov 2022 17:47:56 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Extend LOAD DATA INPATH statement to support Iceberg tables. Native
parquet tables need Iceberg field ids, therefore to add files this
change uses child queries to load and rewrite the data. The child
queries create > insert > drop the temporary table over the specified
directory.
The create part depends on LIKE PARQUET/ORC clauses to infer the file
format. This requires identifying a file in the directory and using that
to create the temporary table.
The target file or directory is moved to a staging directory before
ingestion similar to native file formats. In case of a query failure the
files are moved back to the original location. Child query executor will
return the error message of the failing query and the child query
profiles will be available through the WebUI.
At this point the PARTITION clause it not supported because it would
require analysis of the PartitionSpec (IMPALA-11750).
Testing:
- Added e2e tests
- Added fe unit tests
Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
---
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
A fe/src/main/java/org/apache/impala/analysis/QueryStringBuilder.java
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
A testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test
M tests/query_test/test_iceberg.py
10 files changed, 607 insertions(+), 14 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/19145/9
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 9
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
lipenglin@sensorsdata.cn has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 5: Code-Review+1
(1 comment)
Thanks for fixing this!
LGTM.
http://gerrit.cloudera.org:8080/#/c/19145/4/fe/src/main/java/org/apache/impala/service/Frontend.java
File fe/src/main/java/org/apache/impala/service/Frontend.java:
http://gerrit.cloudera.org:8080/#/c/19145/4/fe/src/main/java/org/apache/impala/service/Frontend.java@974
PS4, Line 974:
> I think this is a copy paste error too, FileSystemUtil.getTotalNumVisibleFi
got it.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 5
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Dec 2022 13:39:26 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/19145 )
Change subject: [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
[WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Extend LOAD DATA INPATH statement to support Iceberg tables.
Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
---
M be/src/service/client-request-state.cc
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
A testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test
M tests/query_test/test_iceberg.py
6 files changed, 216 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/19145/2
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 2
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 6: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8891/
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 6
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 09 Dec 2022 14:29:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 10: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8911/
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 10
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Dec 2022 15:54:26 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 10:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8911/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 10
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Dec 2022 10:45:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 10: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 10
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Dec 2022 10:45:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Gergely Fürnstáhl (Code Review)" <ge...@cloudera.org>.
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 5: Code-Review+1
(1 comment)
Thanks for implementing this, LGMT!
http://gerrit.cloudera.org:8080/#/c/19145/4/fe/src/main/java/org/apache/impala/service/Frontend.java
File fe/src/main/java/org/apache/impala/service/Frontend.java:
http://gerrit.cloudera.org:8080/#/c/19145/4/fe/src/main/java/org/apache/impala/service/Frontend.java@974
PS4, Line 974:
> This is just to signal that the number of loaded files can be plural.
I think this is a copy paste error too, FileSystemUtil.getTotalNumVisibleFiles(destPath) is unused (but String.format() does not tell you that unfortunately), that's the source of the confusion.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 5
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Dec 2022 13:02:43 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 6:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/11992/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 6
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Dec 2022 14:03:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Extend LOAD DATA INPATH statement to support Iceberg tables. Native
parquet tables need Iceberg field ids, therefore to add files this
change uses child queries to load and rewrite the data. The child
queries create > insert > drop the temporary table over the specified
directory.
The create part depends on LIKE PARQUET/ORC clauses to infer the file
format. This requires identifying a file in the directory and using that
to create the temporary table.
The target file or directory is moved to a staging directory before
ingestion similar to native file formats. In case of a query failure the
files are moved back to the original location. Child query executor will
return the error message of the failing query and the child query
profiles will be available through the WebUI.
At this point the PARTITION clause it not supported because it would
require analysis of the PartitionSpec (IMPALA-11750).
Testing:
- Added e2e tests
- Added fe unit tests
Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
---
M be/src/service/client-request-state.cc
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
A fe/src/main/java/org/apache/impala/analysis/QueryStringBuilder.java
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
A testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test
M tests/query_test/test_iceberg.py
9 files changed, 578 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/19145/4
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 4
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 6:
(11 comments)
Left a couple of comments, but the change looks great!
http://gerrit.cloudera.org:8080/#/c/19145/6/be/src/service/client-request-state.cc
File be/src/service/client-request-state.cc:
http://gerrit.cloudera.org:8080/#/c/19145/6/be/src/service/client-request-state.cc@794
PS6, Line 794: {
nit: could you please move the body of this statement to a separate function?
http://gerrit.cloudera.org:8080/#/c/19145/6/common/thrift/Frontend.thrift
File common/thrift/Frontend.thrift:
http://gerrit.cloudera.org:8080/#/c/19145/6/common/thrift/Frontend.thrift@376
PS6, Line 376: the new files has to be added to
: // the Iceberg catalog.
nit: "in this case we need to insert data to the Iceberg table based on the given files."
http://gerrit.cloudera.org:8080/#/c/19145/6/common/thrift/Frontend.thrift@380
PS6, Line 380: whith
nit: with
http://gerrit.cloudera.org:8080/#/c/19145/6/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
File fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java:
http://gerrit.cloudera.org:8080/#/c/19145/6/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@60
PS6, Line 60: ad
nit: indendation is off
http://gerrit.cloudera.org:8080/#/c/19145/6/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@146
PS6, Line 146: IcebergTable
Should be FeIcebergTable, so it works in local catalog mode as well.
This could also be the reason thy the dockerised tests fail.
http://gerrit.cloudera.org:8080/#/c/19145/6/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@227
PS6, Line 227: !(table_ instanceof IcebergTable)
nit: when partitionSpec_ != null then it is never an Iceberg table, right?
http://gerrit.cloudera.org:8080/#/c/19145/6/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@270
PS6, Line 270: LOAD formats
nit: file format? Including 'filePathForLike' in the error message can be useful.
http://gerrit.cloudera.org:8080/#/c/19145/6/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@296
PS6, Line 296: IcebergTable
FeIcebergTable
http://gerrit.cloudera.org:8080/#/c/19145/6/fe/src/main/java/org/apache/impala/analysis/QueryStringBuilder.java
File fe/src/main/java/org/apache/impala/analysis/QueryStringBuilder.java:
http://gerrit.cloudera.org:8080/#/c/19145/6/fe/src/main/java/org/apache/impala/analysis/QueryStringBuilder.java@24
PS6, Line 24: The inner classes
nit: The methods of the inner classes
http://gerrit.cloudera.org:8080/#/c/19145/6/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
File fe/src/main/java/org/apache/impala/common/FileSystemUtil.java:
http://gerrit.cloudera.org:8080/#/c/19145/6/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java@257
PS6, Line 257: Path
nit: please add some comments about the returned object
http://gerrit.cloudera.org:8080/#/c/19145/6/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:
http://gerrit.cloudera.org:8080/#/c/19145/6/tests/query_test/test_iceberg.py@795
PS6, Line 795: @SkipIf.not_hdfs
Maybe this could be substituted to @SkipIfLocal.hdfs_client
So the test would be executed on e.g. S3
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 6
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 09 Dec 2022 19:36:32 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Extend LOAD DATA INPATH statement to support Iceberg tables. Native
parquet tables need Iceberg field ids, therefore to add files this
change uses child queries to load and rewrite the data. The child
queries create > insert > drop the temporary table over the specified
directory.
The create part depends on LIKE PARQUET/ORC clauses to infer the file
format. This requires identifying a file in the directory and using that
to create the temporary table.
The target file or directory is moved to a staging directory before
ingestion similar to native file formats. In case of a query failure the
files are moved back to the original location. Child query executor will
return the error message of the failing query and the child query
profiles will be available through the WebUI.
At this point the PARTITION clause it not supported because it would
require analysis of the PartitionSpec (IMPALA-11750).
Testing:
- Added e2e tests
- Added fe unit tests
Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Reviewed-on: http://gerrit.cloudera.org:8080/19145
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
A fe/src/main/java/org/apache/impala/analysis/QueryStringBuilder.java
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
A testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test
M tests/query_test/test_iceberg.py
10 files changed, 607 insertions(+), 14 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 13
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/19145/8/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
File fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java:
http://gerrit.cloudera.org:8080/#/c/19145/8/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@271
PS8, Line 271: format
> I mentioned the file path but this outputs the magic string as 'format'. Ma
Done
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 8
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Dec 2022 10:21:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Gergely Fürnstáhl (Code Review)" <ge...@cloudera.org>.
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 3:
(5 comments)
The design generally looks good, added a few improvement possibilities
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
File fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java:
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@225
PS3, Line 225: able staetement
typo: table statement
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@233
PS3, Line 233: StringBuilder createTmpTblQueryBuilder = new StringBuilder();
Building the queries manually with string appends is a bit hard to follow/maintain/extend.
I think the Java way is to use a builder class. If we don't want to pull in a dependency, we could at least delegate the logic to a small compact class to support something like this:
QueryBuilder.CreateTableLike(...).InputFileFormat(...).InPath(...).FileFormat(...).Location(...)
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@240
PS3, Line 240: throw new AnalysisException("Only directories can be loaded into Iceberg "
: + "tables.");
IMPALA-11339 proposes a solution for single files as well, are we planning to cut that out of the scope and create a following JIRA?
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@289
PS3, Line 289: deped ont he
typo: depend on the
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@301
PS3, Line 301: "Could not infer file format."
This error message could be misleading in case of a valid but not orc nor parquet file.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 3
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 02 Nov 2022 10:00:55 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11759/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 3
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 01 Nov 2022 18:10:00 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 11:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8915/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 11
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Dec 2022 15:58:49 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 8: Code-Review+2
(1 comment)
One tiny nit, otherwise looks good to me.
Feel free the carry the +2.
http://gerrit.cloudera.org:8080/#/c/19145/8/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
File fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java:
http://gerrit.cloudera.org:8080/#/c/19145/8/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@271
PS8, Line 271: format
I mentioned the file path but this outputs the magic string as 'format'. Maybe include both the file name and the magic string and make it very clear what we show?
I.e. "file is 'filePathForLike', magic string is 'magicString'"
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 8
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Dec 2022 09:38:15 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19145/4/fe/src/main/java/org/apache/impala/service/Frontend.java
File fe/src/main/java/org/apache/impala/service/Frontend.java:
http://gerrit.cloudera.org:8080/#/c/19145/4/fe/src/main/java/org/apache/impala/service/Frontend.java@941
PS4, Line 941: private TLoadDataResp doLoadIcebergTableData(TLoadDataReq request) throws ImpalaException,
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/19145/4/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:
http://gerrit.cloudera.org:8080/#/c/19145/4/tests/query_test/test_iceberg.py@825
PS4, Line 825: \
flake8: E502 the backslash is redundant between brackets
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 4
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Dec 2022 12:55:34 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 8: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 8
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 12 Dec 2022 16:54:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 12: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 12
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Dec 2022 13:34:49 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 11: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 11
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Dec 2022 15:58:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 12: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 12
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Dec 2022 08:22:26 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Extend LOAD DATA INPATH statement to support Iceberg tables. Native
parquet tables need Iceberg field ids, therefore to add files this
change uses child queries to load and rewrite the data. The child
queries create > insert > drop the temporary table over the specified
directory.
The create part depends on LIKE PARQUET/ORC clauses to infer the file
format. This requires identifying a file in the directory and using that
to create the temporary table.
The target file or directory is moved to a staging directory before
ingestion similar to native file formats. In case of a query failure the
files are moved back to the original location. Child query executor will
return the error message of the failing query and the child query
profiles will be available through the WebUI.
At this point the PARTITION clause it not supported because it would
require analysis of the PartitionSpec (IMPALA-11750).
Testing:
- Added e2e tests
- Added fe unit tests
Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
---
M be/src/service/client-request-state.cc
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
A fe/src/main/java/org/apache/impala/analysis/QueryStringBuilder.java
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
A testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test
M tests/query_test/test_iceberg.py
9 files changed, 603 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/19145/5
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 5
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11990/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 4
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Dec 2022 13:13:19 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11638/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 2
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 08:41:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 12:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8919/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 12
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 15 Dec 2022 08:22:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 11: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8915/
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 11
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Dec 2022 21:03:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 6:
The build fail is not related:
14:01:06 [ERROR] Failed to execute goal on project impala-executor-deps: Could not resolve dependencies for project org.apache.impala:impala-executor-deps:pom:4.2.0-SNAPSHOT: Failed to collect dependencies at org.apache.hbase:hbase-client:jar:2.4.6.7.2.16.0-164 -> org.apache.hbase:hbase-common:jar:2.4.6.7.2.16.0-164 -> org.apache.hbase.thirdparty:hbase-unsafe:jar:4.1.1.7.2.16.0-164: Failed to read artifact descriptor for org.apache.hbase.thirdparty:hbase-unsafe:jar:4.1.1.7.2.16.0-164: Could not transfer artifact org.apache.hbase.thirdparty:hbase-unsafe:pom:4.1.1.7.2.16.0-164 from/to impala.cdp.repo (https://native-toolchain.s3.amazonaws.com/build/cdp_components/31397203/maven): Failed to transfer file: https://native-toolchain.s3.amazonaws.com/build/cdp_components/31397203/maven/org/apache/hbase/thirdparty/hbase-unsafe/4.1.1.7.2.16.0-164/hbase-unsafe-4.1.1.7.2.16.0-164.pom. Return code is: 503 , ReasonPhrase:Service Unavailable. -> [Help 1]
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 6
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Dec 2022 14:08:53 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Gergely Fürnstáhl (Code Review)" <ge...@cloudera.org>.
Gergely Fürnstáhl has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 6: Code-Review+1
LGTM!
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 6
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Dec 2022 13:53:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11991/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 5
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Dec 2022 13:19:07 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 8:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8896/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 8
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 12 Dec 2022 11:47:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 5:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19145/5/fe/src/main/java/org/apache/impala/service/Frontend.java
File fe/src/main/java/org/apache/impala/service/Frontend.java:
http://gerrit.cloudera.org:8080/#/c/19145/5/fe/src/main/java/org/apache/impala/service/Frontend.java@941
PS5, Line 941: private TLoadDataResp doLoadIcebergTableData(TLoadDataReq request) throws ImpalaException,
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/19145/5/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:
http://gerrit.cloudera.org:8080/#/c/19145/5/tests/query_test/test_iceberg.py@825
PS5, Line 825: \
flake8: E502 the backslash is redundant between brackets
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 5
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Dec 2022 12:56:23 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Extend LOAD DATA INPATH statement to support Iceberg tables. Native
parquet tables need Iceberg field ids, therefore to add files this
change uses child queries to load and rewrite the data. The child
queries create > insert > drop the temporary table over the specified
directory.
The create part depends on LIKE PARQUET/ORC clauses to infer the file
format. This requires identifying a file in the directory and using that
to create the temporary table.
The target file or directory is moved to a staging directory before
ingestion similar to native file formats. In case of a query failure the
files are moved back to the original location. Child query executor will
return the error message of the failing query and the child query
profiles will be available through the WebUI.
At this point the PARTITION clause it not supported because it would
require analysis of the PartitionSpec (IMPALA-11750).
Testing:
- Added e2e tests
- Added fe unit tests
Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
---
M be/src/service/client-request-state.cc
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
A fe/src/main/java/org/apache/impala/analysis/QueryStringBuilder.java
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
A testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test
M tests/query_test/test_iceberg.py
9 files changed, 601 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/19145/6
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 6
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 1:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19145/1/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
File fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java:
http://gerrit.cloudera.org:8080/#/c/19145/1/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@215
PS1, Line 215: partitionSpec_ != null && !(table_ instanceof IcebergTable) ? partitionSpec_.getPartitionSpecKeyValues() : null,
line too long (122 > 90)
http://gerrit.cloudera.org:8080/#/c/19145/1/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@279
PS1, Line 279: throw new AnalysisException("PARTITION clause is not supported for Iceberg tables.");
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/19145/1/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@287
PS1, Line 287: *
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/19145/1/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@295
PS1, Line 295: } else if (fileFormat.toUpperCase().equals("PARQ") || fileFormat.toUpperCase().equals("PARQUET")) {
line too long (103 > 90)
http://gerrit.cloudera.org:8080/#/c/19145/1/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:
http://gerrit.cloudera.org:8080/#/c/19145/1/tests/query_test/test_iceberg.py@792
PS1, Line 792: @
flake8: E301 expected 1 blank line, found 0
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 1
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 07:20:22 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 2:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/19145/2/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
File fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java:
http://gerrit.cloudera.org:8080/#/c/19145/2/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@215
PS2, Line 215: partitionSpec_ != null && !(table_ instanceof IcebergTable) ? partitionSpec_.getPartitionSpecKeyValues() : null,
line too long (122 > 90)
http://gerrit.cloudera.org:8080/#/c/19145/2/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@279
PS2, Line 279: throw new AnalysisException("PARTITION clause is not supported for Iceberg tables.");
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/19145/2/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@287
PS2, Line 287: *
line has trailing whitespace
http://gerrit.cloudera.org:8080/#/c/19145/2/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@295
PS2, Line 295: } else if (fileFormat.toUpperCase().equals("PARQ") || fileFormat.toUpperCase().equals("PARQUET")) {
line too long (103 > 90)
http://gerrit.cloudera.org:8080/#/c/19145/2/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:
http://gerrit.cloudera.org:8080/#/c/19145/2/tests/query_test/test_iceberg.py@792
PS2, Line 792: @
flake8: E301 expected 1 blank line, found 0
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 2
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 08:21:18 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Extend LOAD DATA INPATH statement to support Iceberg tables. Native
parquet tables need Iceberg field ids, therefore to add files this
change uses child queries to load and rewrite the data. The child
queries create > insert > drop the temporary table over the specified
directory.
The create part depends on LIKE PARQUET/ORC clauses to infer the file
format. This requires identifying a file in the directory and using that
to create the temporary table.
Testing:
- Added e2e tests
- Added fe unit tests
Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
---
M be/src/service/client-request-state.cc
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
A testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test
M tests/query_test/test_iceberg.py
6 files changed, 249 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/19145/3
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 3
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
lipenglin@sensorsdata.cn has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 3:
(4 comments)
Thanks for implementing this feature!
I left a few comments.
http://gerrit.cloudera.org:8080/#/c/19145/3/common/thrift/Frontend.thrift
File common/thrift/Frontend.thrift:
http://gerrit.cloudera.org:8080/#/c/19145/3/common/thrift/Frontend.thrift@380
PS3, Line 380: whith
nit: with?
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
File fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java:
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@66
PS3, Line 66: // Set during analysis
nit: Maybe it can be omitted because of the same comment above(line 63)
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@233
PS3, Line 233: StringBuilder createTmpTblQueryBuilder = new StringBuilder();
> Building the queries manually with string appends is a bit hard to follow/m
Yes, I agree.
http://gerrit.cloudera.org:8080/#/c/19145/3/fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java@316
PS3, Line 316: insertTblQuery_
If insert_tbl_query fails, will the temporary table created above be cleaned up?
If not, although it will not affect the user's re execution (because the suffix of the temporary table name is random uuid string), the temporary table should be manually cleaned up by the user. Should we consider how to remind the user about it?
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 3
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 08 Nov 2022 05:51:34 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
lipenglin@sensorsdata.cn has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 6: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 6
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 08 Dec 2022 02:26:28 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 6:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8891/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 6
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 09 Dec 2022 09:25:02 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Extend LOAD DATA INPATH statement to support Iceberg tables. Native
parquet tables need Iceberg field ids, therefore to add files this
change uses child queries to load and rewrite the data. The child
queries create > insert > drop the temporary table over the specified
directory.
The create part depends on LIKE PARQUET/ORC clauses to infer the file
format. This requires identifying a file in the directory and using that
to create the temporary table.
The target file or directory is moved to a staging directory before
ingestion similar to native file formats. In case of a query failure the
files are moved back to the original location. Child query executor will
return the error message of the failing query and the child query
profiles will be available through the WebUI.
At this point the PARTITION clause it not supported because it would
require analysis of the PartitionSpec (IMPALA-11750).
Testing:
- Added e2e tests
- Added fe unit tests
Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
---
M be/src/service/client-request-state.cc
M be/src/service/client-request-state.h
M common/thrift/Frontend.thrift
M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java
A fe/src/main/java/org/apache/impala/analysis/QueryStringBuilder.java
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
A testdata/workloads/functional-query/queries/QueryTest/iceberg-load.test
M tests/query_test/test_iceberg.py
10 files changed, 607 insertions(+), 14 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/19145/8
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 8
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 8:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/12015/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 8
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Anonymous Coward <li...@sensorsdata.cn>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 12 Dec 2022 11:42:17 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19145 )
Change subject: [WIP] IMPALA-11339: Add Iceberg LOAD DATA INPATH statement
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/11637/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/19145
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8499945fa57ea0499f65b455976141dcd6d789eb
Gerrit-Change-Number: 19145
Gerrit-PatchSet: 1
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 07:40:29 +0000
Gerrit-HasComments: No