You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Weizisheng (Jira)" <ji...@apache.org> on 2023/06/02 08:08:00 UTC
[jira] [Created] (IMPALA-12181) Catalogd can't process hms event once adding an already exists partition
Weizisheng created IMPALA-12181:
-----------------------------------
Summary: Catalogd can't process hms event once adding an already exists partition
Key: IMPALA-12181
URL: https://issues.apache.org/jira/browse/IMPALA-12181
Project: IMPALA
Issue Type: Bug
Components: Catalog, Frontend
Affects Versions: Impala 3.4.0
Reporter: Weizisheng
Add a already exists parttion from Hive using
{code:java}
alter table xxx add if not exists partition (ds="***"){code}
Impala will received 2 events, the 1st one contains no partition and the 2nd one contains the existing partition but with null parameters which will cause NPE， then event process status turns to ERROR...
{code:java}
I0526 17:14:57.637362 2146 MetastoreEventsProcessor.java:482] Received 2 events. Start event id : 12
I0526 17:14:57.652266 2146 MetastoreEvents.java:233] Total number of events received: 2 Total number of events filtered out: 0
I0526 17:14:57.652372 2146 MetastoreEvents.java:366] EventId: 13 EventType: ADD_PARTITION Partition list is empty. Ignoring this event.
E0526 17:14:57.654448 2146 MetastoreEventsProcessor.java:527] Unexpected exception received while processing event
Java exception follows:
java.lang.NullPointerException
at java.util.HashMap.putMapEntries(HashMap.java:501)
at java.util.HashMap.putAll(HashMap.java:785)
at org.apache.impala.catalog.events.MetastoreEvents$AddPartitionEvent.getSelfEventContext(MetastoreEvents.java:1320)
at org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.isSelfEvent(MetastoreEvents.java:425)
at org.apache.impala.catalog.events.MetastoreEvents$AddPartitionEvent.process(MetastoreEvents.java:1333)
at org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:311)
at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:615)
at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:513)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
E0526 17:14:57.654541 2146 MetastoreEventsProcessor.java:637] Notification event is null
W0526 17:14:58.654865 2146 MetastoreEventsProcessor.java:506] Event processing is skipped since status is ERROR. Last synced event id is 13 {code}
PS:
before reporting this issue i've already found IMPALA-8847 and seems it's been fixed in Impala-3.3.0. I'm using Impala-3.4.0 with Hive-3.1.0
From [https://github.com/apache/impala/commit/0ff4f45#diff-531be00fa82d527a36363426497585e5cfcfaabba18766ef875d609c8151975fR1372]
, IMPALA-8847 Ignore such events when {{addedPartitions_.isEmpty()}} is ture. But as i said, hms will generate 2 events. The 1st event can be ignore, the 2nd one will be handled...
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org