[jira] [Created] (IMPALA-10949) Improve batching logic of events

["Vihang Karajgaonkar (Jira)" <ji...@apache.org>]

Vihang Karajgaonkar created IMPALA-10949:
--------------------------------------------

             Summary: Improve batching logic of events
                 Key: IMPALA-10949
                 URL: https://issues.apache.org/jira/browse/IMPALA-10949
             Project: IMPALA
          Issue Type: Improvement
            Reporter: Vihang Karajgaonkar


This is a followup based on the review comment https://gerrit.cloudera.org/#/c/17848/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1641

Current approach of batching batches together the events from a single operation so that self-event check is done per-batch. However, it looks like there is a considerable scope of improving the batching logic by clubbing together accross the various sources of the events on a table when IMPALA-10926 is merged. After IMPALA-10926 each table will track the last_synced_event and then the events processor can simply ignore a event which <= the last_synced_event. This simplification of self-events logic will enable easier batching for all the events of a type on a table.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)