You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@impala.apache.org by "Venugopal Reddy K (Jira)" <ji...@apache.org> on 2023/12/01 17:20:00 UTC

[jira] [Resolved] (IMPALA-12474) Latest metastore event id polling can be blocked

     [ https://issues.apache.org/jira/browse/IMPALA-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Venugopal Reddy K resolved IMPALA-12474.
----------------------------------------
    Resolution: Fixed

> Latest metastore event id polling can be blocked
> ------------------------------------------------
>
>                 Key: IMPALA-12474
>                 URL: https://issues.apache.org/jira/browse/IMPALA-12474
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Catalog
>            Reporter: Csaba Ringhofer
>            Assignee: Venugopal Reddy K
>            Priority: Critical
>
> Latest event id polling is supposed to work independently from event batch polling:
> https://github.com/apache/impala/blob/2d3289027c2ffdd245d13b60e6fa3f9b3e7bf833/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L684
> This doesn't work in reality as the scheduler is single threaded:
> https://github.com/apache/impala/blob/2d3289027c2ffdd245d13b60e6fa3f9b3e7bf833/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L533C44-L533C44
> A related potential improvement: scheduleWithFixedDelay adds a delay after batch processing, even it was slow - if Impala know that it is lagging behind then it should start to work on the next batch immediately.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)