[jira] [Resolved] (IMPALA-10926) Sync db/table to latest event id for ddl operations in catalog metastore server

["Sourabh Goyal (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sourabh Goyal resolved IMPALA-10926.
------------------------------------
    Resolution: Fixed

Patch already merged to master.

Thank you [~vihangk1]  for the review.

> Sync db/table to latest event id for ddl operations in catalog metastore server
> -------------------------------------------------------------------------------
>
>                 Key: IMPALA-10926
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10926
>             Project: IMPALA
>          Issue Type: Task
>            Reporter: Sourabh Goyal
>            Assignee: Sourabh Goyal
>            Priority: Major
>
> Whenever DDL operations like add/remove partitions, alter table etc are performed from catalog HMS endpoints, we will do the following: 
>  # Acquire lock on db/table 
>  # Perform HMS operation
>  # For db/table in step 1, fetch all events from HMS
>  # Apply those events on the db/table in catalogd cache 
>  # Set the last processed event id for db/table 
> In addition to the above, the following needs to be handled as well: 
>  # Event processor should skip processing an event on a db/table if the db/table is already synced till that event id. 
>  # If last synced event id in table/db is less than the event being processed by event processor, it should set the last synced db/table event id after successfully processing the event. 
>  # Full table refresh should set the last event processed. 
> cc - [~kishendas] [~vihangk1]



--
This message was sent by Atlassian Jira
(v8.20.1#820001)