You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2022/10/09 02:02:00 UTC
[jira] [Commented] (IMPALA-11644) updateLatestEventId should handle cases of empty events
[ https://issues.apache.org/jira/browse/IMPALA-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614557#comment-17614557 ]
Quanlong Huang commented on IMPALA-11644:
-----------------------------------------
I add a log line to print the currentEventId and latestEventId_ in this check:
{code:java}
// no new events since we last polled
if (currentEventId <= latestEventId_.get()) {
return;
}
{code}
https://github.com/apache/impala/blob/276759271d6a75b6de1b565e7bb7843c382fbb31/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L882
When the notification events are empty in HMS, currentEventId is 0. However, at catalogd startup, latestEventId_ is -1. Then the code goes down to fetch the events and fail. We should handle this special case.
> updateLatestEventId should handle cases of empty events
> -------------------------------------------------------
>
> Key: IMPALA-11644
> URL: https://issues.apache.org/jira/browse/IMPALA-11644
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 4.2.0
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> IMPALA-11490 adds a catalog metrics of latest event id in HMS. The retention duration of notification events is 24 hours by default. If they are cleaned up and there are no more events, the thread for updating latest event id will encounter the following exception:
> {noformat}
> E1009 09:29:13.774916 11395 MetastoreEventsProcessor.java:898] Unable to update current notification event id. Last value: -1
> Java exception follows:
> java.util.NoSuchElementException
> at java.util.ArrayList$Itr.next(ArrayList.java:864)
> at org.apache.impala.catalog.events.MetastoreEventsProcessor.updateLatestEventId(MetastoreEventsProcessor.java:891)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org