You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org> on 2022/03/10 20:37:11 UTC
[Impala-ASF-CR] IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18307
Change subject: IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
......................................................................
IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
When Impala created the metadata objects about the Iceberg data files it
tried to convert the partition values to strings. But the partition
values can be NULLs as well. The code didn't expect this, so we got a
NullPointerException.
With this patch we pass the table's null partition key value in case
of NULLs.
Testing:
* added e2e tests
Change-Id: I88c4f7a2c2db4f6390c8ee5c08baddc96b04602e
---
M be/src/exec/file-metadata-utils.cc
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partition-transform-insert.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert.test
6 files changed, 112 insertions(+), 32 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/18307/1
--
To view, visit http://gerrit.cloudera.org:8080/18307
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I88c4f7a2c2db4f6390c8ee5c08baddc96b04602e
Gerrit-Change-Number: 18307
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18307 )
Change subject: IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
......................................................................
Patch Set 1:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7927/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/18307
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I88c4f7a2c2db4f6390c8ee5c08baddc96b04602e
Gerrit-Change-Number: 18307
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Comment-Date: Fri, 11 Mar 2022 08:24:46 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18307 )
Change subject: IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
......................................................................
IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
When Impala created the metadata objects about the Iceberg data files it
tried to convert the partition values to strings. But the partition
values can be NULLs as well. The code didn't expect this, so we got a
NullPointerException.
With this patch we pass the table's null partition key value in case
of NULLs.
Testing:
* added e2e tests
Change-Id: I88c4f7a2c2db4f6390c8ee5c08baddc96b04602e
Reviewed-on: http://gerrit.cloudera.org:8080/18307
Reviewed-by: Tamas Mate <tm...@apache.org>
Reviewed-by: Gabor Kaszab <ga...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exec/file-metadata-utils.cc
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partition-transform-insert.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-partitioned-insert.test
6 files changed, 112 insertions(+), 32 deletions(-)
Approvals:
Tamas Mate: Looks good to me, but someone else must approve
Gabor Kaszab: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/18307
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I88c4f7a2c2db4f6390c8ee5c08baddc96b04602e
Gerrit-Change-Number: 18307
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18307 )
Change subject: IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
......................................................................
Patch Set 1: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/18307
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I88c4f7a2c2db4f6390c8ee5c08baddc96b04602e
Gerrit-Change-Number: 18307
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Comment-Date: Fri, 11 Mar 2022 13:21:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
Posted by "Gabor Kaszab (Code Review)" <ge...@cloudera.org>.
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/18307 )
Change subject: IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/18307
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I88c4f7a2c2db4f6390c8ee5c08baddc96b04602e
Gerrit-Change-Number: 18307
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Comment-Date: Fri, 11 Mar 2022 09:53:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18307 )
Change subject: IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/10281/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18307
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I88c4f7a2c2db4f6390c8ee5c08baddc96b04602e
Gerrit-Change-Number: 18307
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Comment-Date: Thu, 10 Mar 2022 20:57:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
Posted by "Tamas Mate (Code Review)" <ge...@cloudera.org>.
Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/18307 )
Change subject: IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
......................................................................
Patch Set 1: Code-Review+1
Thanks for the fix Zoli, LGTM!
--
To view, visit http://gerrit.cloudera.org:8080/18307
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I88c4f7a2c2db4f6390c8ee5c08baddc96b04602e
Gerrit-Change-Number: 18307
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Comment-Date: Fri, 11 Mar 2022 08:14:34 +0000
Gerrit-HasComments: No