[jira] [Created] (IMPALA-12257) createInsertEvents failed by NullPointerException: Invalid partition name

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-12257:
---------------------------------------

             Summary: createInsertEvents failed by NullPointerException: Invalid partition name
                 Key: IMPALA-12257
                 URL: https://issues.apache.org/jira/browse/IMPALA-12257
             Project: IMPALA
          Issue Type: Bug
          Components: Catalog
            Reporter: Quanlong Huang
            Assignee: Quanlong Huang


INSERT on partitioned table could fail in createInsertEvents() if some updated partitions are missing in catalogd but actually exists in HMS. The failure is a NullPointerException, e.g.
{code:java}
I0630 14:44:29.779798 30287 jni-util.cc:288] java.lang.NullPointerException: Invalid partition name: p=0
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:907)
        at org.apache.impala.catalog.HdfsTable.getPartitionsForNames(HdfsTable.java:1758)
        at org.apache.impala.service.CatalogOpExecutor.createInsertEvents(CatalogOpExecutor.java:6935)
        at org.apache.impala.service.CatalogOpExecutor.updateCatalog(CatalogOpExecutor.java:6830)
        at org.apache.impala.service.JniCatalog.lambda$updateCatalog$16(JniCatalog.java:471)
        at org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
        at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
        at org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
        at org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
        at org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:230)
        at org.apache.impala.service.JniCatalog.updateCatalog(JniCatalog.java:470) {code}

We've seen this in an issue caused by IMPALA-12256 in which the stale DROP_PARTITION event incorrectly drops the partition, causing inconsistency between catalogd and HMS on the partition list.

To steadily reproduce the issue, we can disable HMS event-processing and manually make the partition list differs between catalogd and HMS.
{code}
impala> create table my_part (id int) partitioned by (p int) stored as textfile;
# Add one partition in Hive
hive> alter table my_part add partition(p=0);
{code}
Then run INSERT on the same partition in Impala:
{code}
impala> insert into my_part partition(p=0) values (0);
ERROR: NullPointerException: Invalid partition name: p=0
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org