You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Wenzhe Zhou (Jira)" <ji...@apache.org> on 2023/04/25 19:24:00 UTC
[jira] [Assigned] (IMPALA-12099) Catalogd NPE when refreshing a partitioned table
[ https://issues.apache.org/jira/browse/IMPALA-12099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenzhe Zhou reassigned IMPALA-12099:
------------------------------------
Assignee: Wenzhe Zhou
> Catalogd NPE when refreshing a partitioned table
> ------------------------------------------------
>
> Key: IMPALA-12099
> URL: https://issues.apache.org/jira/browse/IMPALA-12099
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Major
>
> The following NPE occurs in the catalogd when it tries to refresh certain partitioned tables
> {code:java}
> I0331 13:33:04.345813 256 TAcceptQueueServer.cpp:355] New connection to server CatalogService from client <Host: 127.0.0.6 Port: 50285>
> E0331 13:33:04.372188 31256 JniCatalog.java:424] Error in getting PartialCatalogObject of TABLE:ml_foodforecast.presence_history_archive_ahdp_5584. Time spent: 15ms.
> I0331 13:33:04.372326 31256 jni-util.cc:286] java.lang.NullPointerException
> at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:878)
> at org.apache.impala.compat.MetastoreShim.getPartitionsForRefreshingFileMetadata(MetastoreShim.java:602)
> at org.apache.impala.util.AcidUtils.getPartitionsForRefreshingFileMetadata(AcidUtils.java:778)
> at org.apache.impala.catalog.CatalogServiceCatalog.getOrLoadTable(CatalogServiceCatalog.java:2252)
> at org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:3528)
> at org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:3454)
> at org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:3421)
> at org.apache.impala.service.JniCatalog.getPartialCatalogObject(JniCatalog.java:419)
> I0331 13:33:04.372351 31256 status.cc:129] NullPointerException: null
> @ 0xdbeae3
> @ 0x166ade9
> @ 0xd88ec3
> @ 0xd6e013
> @ 0xd44e32
> @ 0xcae4e3
> @ 0xd19ab5
> @ 0x11d1b69
> @ 0x11c51b8
> @ 0x1748f74
> @ 0x174a3ba
> @ 0x1fec2d0
> @ 0x7fad37596ea4
> @ 0x7fad33fd7b0c
> E0331 13:33:04.372615 31256 catalog-server.cc:239] NullPointerException: null
> {code}
> According to [1], Impala assumes that if a table is partitioned then all CompactionInfoStruct entries should not have null partitionname or id. While id is required, partitionname is defined as optional by Hive [2].
> This assumption is false, however. As according to [3], it is possible for a compaction queue entry for a partitioned table to have null partitionname.
> [1]https://github.com/apache/impala/blob/master/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java#L692-L694
> [2]https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift#L1303-L1323
> [3]https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java#L345
> Function checkLatestCompaction() has similar code. It should also handle the null case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org