[jira] [Created] (IMPALA-8632) Add support for self-event detection for insert events

["Vihang Karajgaonkar (JIRA)" <ji...@apache.org>]

Vihang Karajgaonkar created IMPALA-8632:
-------------------------------------------

             Summary: Add support for self-event detection for insert events
                 Key: IMPALA-8632
                 URL: https://issues.apache.org/jira/browse/IMPALA-8632
             Project: IMPALA
          Issue Type: Sub-task
            Reporter: Vihang Karajgaonkar
            Assignee: Vihang Karajgaonkar


In case of {{INSERT_EVENTS}} if Impala inserts into a table it causes a refresh to the underlying table/partition. This could be unnecessary when there is only one Impala cluster in the system. The existing self-event detection framework cannot identify such events because they are not sending HMS objects like tables and partitions to the HMS. Instead in case of {{INSERT_EVENT}} HMS API only asks for a table name or partition value to fire a insert event on it. 

We can detect a self-event in such cases if the HMS API to fire a listener event is improved to return the event id. This would be used by EventProcessor to ignore the event when it is fetched later in the next polling cycle. In order to support this, we will need to make a change to Hive as well so that the enhanced API can be used.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)