You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Sourabh Goyal (Jira)" <ji...@apache.org> on 2021/09/20 23:25:00 UTC
[jira] [Updated] (IMPALA-10926) Sync db/table to latest event id
for ddl operations in catalog metastore server
[ https://issues.apache.org/jira/browse/IMPALA-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sourabh Goyal updated IMPALA-10926:
-----------------------------------
Summary: Sync db/table to latest event id for ddl operations in catalog metastore server (was: Sync db/table to latest event id for ddl operations in catalog megastore server)
> Sync db/table to latest event id for ddl operations in catalog metastore server
> -------------------------------------------------------------------------------
>
> Key: IMPALA-10926
> URL: https://issues.apache.org/jira/browse/IMPALA-10926
> Project: IMPALA
> Issue Type: Task
> Reporter: Sourabh Goyal
> Assignee: Sourabh Goyal
> Priority: Major
>
> Whenever DDL operations like add/remove partitions, alter table etc are performed from catalog HMS endpoints, we will do the following:
> # Acquire lock on db/table
> # Perform HMS operation
> # For db/table in step 1, fetch all events from HMS
> # Apply those events on the db/table in catalogd cache
> # Set the last processed event id for db/table
> In addition to the above, the following needs to be handled as well:
> # Event processor should skip processing an event on a db/table if the db/table is already synced till that event id.
> # If last synced event id in table/db is less than the event being processed by event processor, it should set the last synced db/table event id after successfully processing the event.
> # Full table refresh should set the last event processed.
> cc - [~kishendas] [~vihangk1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org