You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Weizisheng (Jira)" <ji...@apache.org> on 2023/06/02 08:13:00 UTC
[jira] [Updated] (IMPALA-12181) Catalogd can't process hms event once adding an already exists partition
[ https://issues.apache.org/jira/browse/IMPALA-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Weizisheng updated IMPALA-12181:
--------------------------------
Component/s: (was: Frontend)
> Catalogd can't process hms event once adding an already exists partition
> ------------------------------------------------------------------------
>
> Key: IMPALA-12181
> URL: https://issues.apache.org/jira/browse/IMPALA-12181
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.4.0
> Reporter: Weizisheng
> Priority: Major
>
> Add a already exists parttion from Hive using
> {code:java}
> alter table xxx add if not exists partition (ds="***"){code}
> Impala will received 2 events, the 1st one contains no partition and the 2nd one contains the existing partition but with null parameters which will cause NPE， then event process status turns to ERROR...
> {code:java}
> I0526 17:14:57.637362 2146 MetastoreEventsProcessor.java:482] Received 2 events. Start event id : 12
> I0526 17:14:57.652266 2146 MetastoreEvents.java:233] Total number of events received: 2 Total number of events filtered out: 0
> I0526 17:14:57.652372 2146 MetastoreEvents.java:366] EventId: 13 EventType: ADD_PARTITION Partition list is empty. Ignoring this event.
> E0526 17:14:57.654448 2146 MetastoreEventsProcessor.java:527] Unexpected exception received while processing event
> Java exception follows:
> java.lang.NullPointerException
> at java.util.HashMap.putMapEntries(HashMap.java:501)
> at java.util.HashMap.putAll(HashMap.java:785)
> at org.apache.impala.catalog.events.MetastoreEvents$AddPartitionEvent.getSelfEventContext(MetastoreEvents.java:1320)
> at org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.isSelfEvent(MetastoreEvents.java:425)
> at org.apache.impala.catalog.events.MetastoreEvents$AddPartitionEvent.process(MetastoreEvents.java:1333)
> at org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:311)
> at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:615)
> at org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:513)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> E0526 17:14:57.654541 2146 MetastoreEventsProcessor.java:637] Notification event is null
> W0526 17:14:58.654865 2146 MetastoreEventsProcessor.java:506] Event processing is skipped since status is ERROR. Last synced event id is 13 {code}
> PS:
> before reporting this issue i've already found IMPALA-8847 and seems it's been fixed in Impala-3.3.0. I'm using Impala-3.4.0 with Hive-3.1.0
> From [https://github.com/apache/impala/commit/0ff4f45#diff-531be00fa82d527a36363426497585e5cfcfaabba18766ef875d609c8151975fR1372]
> , IMPALA-8847 Ignore such events when {{addedPartitions_.isEmpty()}} is ture. But as i said, hms will generate 2 events. The 1st event can be ignore, the 2nd one will be handled...
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org