You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/03/11 13:26:00 UTC
[jira] [Commented] (IMPALA-11175) Iceberg table cannot be loaded when partition value is NULL
[ https://issues.apache.org/jira/browse/IMPALA-11175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504919#comment-17504919 ]
ASF subversion and git services commented on IMPALA-11175:
----------------------------------------------------------
Commit 2fffac3bad3b86d8aa6d6c1c31400578373de505 in impala's branch refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=2fffac3 ]
IMPALA-11175: Iceberg table cannot be loaded when partition value is NULL
When Impala created the metadata objects about the Iceberg data files it
tried to convert the partition values to strings. But the partition
values can be NULLs as well. The code didn't expect this, so we got a
NullPointerException.
With this patch we pass the table's null partition key value in case
of NULLs.
Testing:
* added e2e tests
Change-Id: I88c4f7a2c2db4f6390c8ee5c08baddc96b04602e
Reviewed-on: http://gerrit.cloudera.org:8080/18307
Reviewed-by: Tamas Mate <tm...@apache.org>
Reviewed-by: Gabor Kaszab <ga...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Iceberg table cannot be loaded when partition value is NULL
> -----------------------------------------------------------
>
> Key: IMPALA-11175
> URL: https://issues.apache.org/jira/browse/IMPALA-11175
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
>
> We get NullPointerException when we try to load an Iceberg table with a NULL partition:
> {noformat}
> Thread-11[1] where
> [1] org.apache.impala.util.IcebergUtil.createPartitionTransformValue (IcebergUtil.java:891)
> [2] org.apache.impala.util.IcebergUtil.createPartitionKeys (IcebergUtil.java:879)
> [3] org.apache.impala.util.IcebergUtil.createIcebergMetadata (IcebergUtil.java:857)
> [4] org.apache.impala.util.IcebergUtil.createIcebergMetadata (IcebergUtil.java:844)
> [5] org.apache.impala.catalog.FeIcebergTable$Utils.loadAllPartition (FeIcebergTable.java:583)
> [6] org.apache.impala.catalog.IcebergTable.load (IcebergTable.java:348)
> [7] org.apache.impala.service.CatalogOpExecutor.loadTableMetadata (CatalogOpExecutor.java:1,430)
> [8] org.apache.impala.service.CatalogOpExecutor.updateCatalog (CatalogOpExecutor.java:6,386)
> [9] org.apache.impala.service.JniCatalog.updateCatalog (JniCatalog.java:431)
> {noformat}
> The issue is reproducible on current master by:
> {noformat}
> create table store_sales partitioned by spec (ss_sold_date_sk) stored as iceberg as select * from tpcds_parquet.store_sales;
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org