[jira] [Commented] (IMPALA-11535) Skip events happen before manual REFRESH

["Sai Hemanth Gantasala (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698825#comment-17698825 ] 

Sai Hemanth Gantasala commented on IMPALA-11535:
------------------------------------------------

With the Implementation of IMPALA-11822, We can skip some of the events by comparing the current event's id with the latestRefreshEventId of that table/partition. 

> Skip events happen before manual REFRESH
> ----------------------------------------
>
>                 Key: IMPALA-11535
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11535
>             Project: IMPALA
>          Issue Type: Improvement
>            Reporter: Quanlong Huang
>            Assignee: Sai Hemanth Gantasala
>            Priority: Critical
>
> If the table has been manually refreshed, all its events happen before the manual REFRESH can be skipped.
>  
> This happens when catalogd is lagging behind in processing events. When processing an event, we can check whether there are manual REFRESH executed after its eventTime. In such case, we don't need to process the event to refresh anything. This helps catalogd to catch up HMS events quickly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org