[jira] [Created] (IMPALA-12884) Add GC for the inflight event list

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-12884:
---------------------------------------

             Summary: Add GC for the inflight event list
                 Key: IMPALA-12884
                 URL: https://issues.apache.org/jira/browse/IMPALA-12884
             Project: IMPALA
          Issue Type: Bug
          Components: Catalog
            Reporter: Quanlong Huang


The inflight event list for catalog versions could have dangling items, e.g. in some catalog operations we don't trigger any HMS APIs but still add the version into the list.

To be more robust, we should add GC to InFlightEvents:
* For idsForInflightDmlEvents_, if the event id is smaller than the lastSycnedEventId of the event-processor, it won't be matched anymore.
* For versionsForInflightEvents_, it's a bit complex. We don't know when the catalog version is stale. Maybe we should refactor the self-event detection to only check the catalog service id and ignore any event that have older catalog version than the current table. The possibility of two Impala clusters having the same catalog service id is pretty small.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org