You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@impala.apache.org by "Csaba Ringhofer (Jira)" <ji...@apache.org> on 2023/09/29 08:48:00 UTC

[jira] [Created] (IMPALA-12474) Latest metastore event id polling can be blocked

Csaba Ringhofer created IMPALA-12474:
----------------------------------------

             Summary: Latest metastore event id polling can be blocked
                 Key: IMPALA-12474
                 URL: https://issues.apache.org/jira/browse/IMPALA-12474
             Project: IMPALA
          Issue Type: Bug
          Components: Catalog
            Reporter: Csaba Ringhofer


Latest event id polling is supposed to work independently from event batch polling:
https://github.com/apache/impala/blob/2d3289027c2ffdd245d13b60e6fa3f9b3e7bf833/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L684

This doesn't work in reality as the scheduler is single threaded:
https://github.com/apache/impala/blob/2d3289027c2ffdd245d13b60e6fa3f9b3e7bf833/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L533C44-L533C44

A related potential improvement: scheduleWithFixedDelay adds a delay after batch processing, even it was slow - if Impala know that it is lagging behind then it should start to work on the next batch immediately.





--
This message was sent by Atlassian Jira
(v8.20.10#820010)