You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org> on 2021/06/14 16:01:36 UTC
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17593
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
IMPALA-10166 (part 2): Enable DROP and CHANGE column
IMPALA-10166 (part 1) already added the necessary code for
DROP and CHANGE COLUMN, but disabled those stmts because to correctly
support schema evolution we had to wait for column resolution
by Iceberg field id.
Since then IMPALA-10361 and IMPALA-10485 added support for field-id
based column resolution for Parquet and ORC as well.
Hence this patch enables DROP and CHANGE column ALTER TABLE
statements. We still disallow REPLACE COLUMNS because it doesn't
really make sense for Iceberg tables as it basically makes all
existing data inaccessible.
Testing:
* added e2e tests
Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableReplaceColsStmt.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test
4 files changed, 112 insertions(+), 37 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/17593/1
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8921/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Tue, 15 Jun 2021 14:12:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 5:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7236/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Thu, 17 Jun 2021 08:12:40 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 1: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17593/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
File testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test:
http://gerrit.cloudera.org:8080/#/c/17593/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test@172
PS1, Line 172: float -> double is allowed by Iceberg.
Can you add some tests for other types? According to https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java#L119
only 3 type conversions are supported.
I am particularly curious about the decimal -> decimal change, which looks hard to implement correctly.
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Tue, 15 Jun 2021 06:18:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 6: Verified+1
The GVO failed due to IMPALA-10754.
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Tue, 22 Jun 2021 10:41:02 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17593/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
File testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test:
http://gerrit.cloudera.org:8080/#/c/17593/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test@172
PS1, Line 172: float -> double is allowed by Iceberg.
> I added tests for them. Btw, we just let Iceberg throw exceptions when it d
Unfortunately Impala cannot read decimals with lower precision due to IMPALA-7087.
I'm disabling changing DECIMAL columns until IMPALA-7087 is fixed.
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Wed, 16 Jun 2021 13:57:15 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has removed a vote on this change.
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Removed Verified-1 by Impala Public Jenkins <im...@cloudera.com>
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8956/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 21 Jun 2021 12:34:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Thu, 17 Jun 2021 08:12:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello wangsheng, Attila Jeges, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17593
to look at the new patch set (#3).
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
IMPALA-10166 (part 2): Enable DROP and CHANGE column
IMPALA-10166 (part 1) already added the necessary code for
DROP and CHANGE COLUMN, but disabled those stmts because to correctly
support schema evolution we had to wait for column resolution
by Iceberg field id.
Since then IMPALA-10361 and IMPALA-10485 added support for field-id
based column resolution for Parquet and ORC as well.
Hence this patch enables DROP and CHANGE column ALTER TABLE
statements. We still disallow REPLACE COLUMNS because it doesn't
really make sense for Iceberg tables as it basically makes all
existing data inaccessible.
Changing DECIMAL columns are still disabled due to IMPALA-7087.
Testing:
* added e2e tests
Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableReplaceColsStmt.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test
5 files changed, 156 insertions(+), 38 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/17593/3
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 5: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7236/
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Thu, 17 Jun 2021 14:17:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello wangsheng, Attila Jeges, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17593
to look at the new patch set (#6).
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
IMPALA-10166 (part 2): Enable DROP and CHANGE column
IMPALA-10166 (part 1) already added the necessary code for
DROP and CHANGE COLUMN, but disabled those stmts because to correctly
support schema evolution we had to wait for column resolution
by Iceberg field id.
Since then IMPALA-10361 and IMPALA-10485 added support for field-id
based column resolution for Parquet and ORC as well.
Hence this patch enables DROP and CHANGE column ALTER TABLE
statements. We still disallow REPLACE COLUMNS because it doesn't
really make sense for Iceberg tables as it basically makes all
existing data inaccessible.
Changing DECIMAL columns are still disabled due to IMPALA-7087.
Testing:
* added e2e tests
Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableReplaceColsStmt.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test
5 files changed, 156 insertions(+), 38 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/17593/6
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Wed, 16 Jun 2021 15:06:59 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 6:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7242/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 21 Jun 2021 12:13:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 4: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7233/
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Wed, 16 Jun 2021 21:10:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8936/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Wed, 16 Jun 2021 14:18:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 5:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7239/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Sun, 20 Jun 2021 10:40:53 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 3: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17593/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
File testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test:
http://gerrit.cloudera.org:8080/#/c/17593/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test@172
PS1, Line 172: float -> double is allowed by Iceberg.
> Unfortunately Impala cannot read decimals with lower precision due to IMPAL
Thanks for looking into this!
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Wed, 16 Jun 2021 14:37:14 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
IMPALA-10166 (part 2): Enable DROP and CHANGE column
IMPALA-10166 (part 1) already added the necessary code for
DROP and CHANGE COLUMN, but disabled those stmts because to correctly
support schema evolution we had to wait for column resolution
by Iceberg field id.
Since then IMPALA-10361 and IMPALA-10485 added support for field-id
based column resolution for Parquet and ORC as well.
Hence this patch enables DROP and CHANGE column ALTER TABLE
statements. We still disallow REPLACE COLUMNS because it doesn't
really make sense for Iceberg tables as it basically makes all
existing data inaccessible.
Changing DECIMAL columns are still disabled due to IMPALA-7087.
Testing:
* added e2e tests
Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Reviewed-on: http://gerrit.cloudera.org:8080/17593
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Zoltan Borok-Nagy <bo...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableReplaceColsStmt.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test
5 files changed, 156 insertions(+), 38 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved
Zoltan Borok-Nagy: Verified
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 7
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 5:
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7239/
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Sun, 20 Jun 2021 16:35:58 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7233/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Wed, 16 Jun 2021 15:07:00 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello wangsheng, Attila Jeges, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17593
to look at the new patch set (#2).
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
IMPALA-10166 (part 2): Enable DROP and CHANGE column
IMPALA-10166 (part 1) already added the necessary code for
DROP and CHANGE COLUMN, but disabled those stmts because to correctly
support schema evolution we had to wait for column resolution
by Iceberg field id.
Since then IMPALA-10361 and IMPALA-10485 added support for field-id
based column resolution for Parquet and ORC as well.
Hence this patch enables DROP and CHANGE column ALTER TABLE
statements. We still disallow REPLACE COLUMNS because it doesn't
really make sense for Iceberg tables as it basically makes all
existing data inaccessible.
Testing:
* added e2e tests
Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableReplaceColsStmt.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test
5 files changed, 170 insertions(+), 38 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/17593/2
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8904/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 14 Jun 2021 16:24:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 1:
(1 comment)
Thanks for the comment!
http://gerrit.cloudera.org:8080/#/c/17593/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test
File testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test:
http://gerrit.cloudera.org:8080/#/c/17593/1/testdata/workloads/functional-query/queries/QueryTest/iceberg-alter.test@172
PS1, Line 172: float -> double is allowed by Iceberg.
> Can you add some tests for other types? According to https://github.com/apa
I added tests for them. Btw, we just let Iceberg throw exceptions when it doesn't like the schema changes.
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Tue, 15 Jun 2021 13:49:33 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10166 (part 2): Enable DROP and CHANGE column
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17593 )
Change subject: IMPALA-10166 (part 2): Enable DROP and CHANGE column
......................................................................
Patch Set 6:
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/7242/
--
To view, visit http://gerrit.cloudera.org:8080/17593
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I9b0d1a55bf0ed718724a69b51392ed53680ffa90
Gerrit-Change-Number: 17593
Gerrit-PatchSet: 6
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Attila Jeges <at...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 21 Jun 2021 18:08:23 +0000
Gerrit-HasComments: No