You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2023/05/23 09:17:00 UTC

[jira] [Commented] (IMPALA-11588) lastProcessedEvent is not updated in MetastoreEventsProcessor.processEvents()

    [ https://issues.apache.org/jira/browse/IMPALA-11588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725313#comment-17725313 ] 

Quanlong Huang commented on IMPALA-11588:
-----------------------------------------

Note that this bug causes the event not being printed in logs. Instead, a log of "Notification event is  null" is printed.

> lastProcessedEvent is not updated in MetastoreEventsProcessor.processEvents()
> -----------------------------------------------------------------------------
>
>                 Key: IMPALA-11588
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11588
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Catalog
>            Reporter: Quanlong Huang
>            Assignee: Quanlong Huang
>            Priority: Major
>
> Code snipper:
> {code:java}
>   public void processEvents() {
>     NotificationEvent lastProcessedEvent = null;
>     try {
>       EventProcessorStatus currentStatus = eventProcessorStatus_;
>       if (currentStatus != EventProcessorStatus.ACTIVE) {
>         LOG.warn(String.format(
>             "Event processing is skipped since status is %s. Last synced event id is %d",
>             currentStatus, lastSyncedEventId_.get()));
>         return;
>       }
>       List<NotificationEvent> events = getNextMetastoreEvents();
>       processEvents(events);
>     } catch (MetastoreNotificationFetchException ex) {
>       // No need to change the EventProcessor state to error since we want the
>       // EventProcessor to continue getting new events after HMS is back up.
>       LOG.error("Unable to fetch the next batch of metastore events. Hive Metastore " +
>         "may be unavailable. Will retry.", ex);
>     } catch(MetastoreNotificationNeedsInvalidateException ex) {
>       updateStatus(EventProcessorStatus.NEEDS_INVALIDATE);
>       LOG.error("Event processing needs a invalidate command to resolve the state", ex);
>     } catch (Exception ex) {
>       // There are lot of Preconditions which can throw RuntimeExceptions when we
>       // process events this catch all exception block is needed so that the scheduler
>       // thread does not die silently
>       updateStatus(EventProcessorStatus.ERROR);
>       LOG.error("Unexpected exception received while processing event", ex);
>       dumpEventInfoToLog(lastProcessedEvent);
>     }
>   }
> {code}
> https://github.com/apache/impala/blob/cfd79b40beab86f08ad72e0bea41eabf736d0a99/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L838-L865
> "lastProcessedEvent" is a local variable and never updated so it's always null.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org