[jira] [Created] (RANGER-4130) Improve performance of event processing in agsync by optimizing number of commits to Kafka broker

["Abhay Kulkarni (Jira)" <ji...@apache.org>]

Abhay Kulkarni created RANGER-4130:
--------------------------------------

             Summary: Improve performance of event processing in agsync by optimizing number of commits to Kafka broker 
                 Key: RANGER-4130
                 URL: https://issues.apache.org/jira/browse/RANGER-4130
             Project: Ranger
          Issue Type: Improvement
          Components: tagsync
            Reporter: Abhay Kulkarni
            Assignee: Abhay Kulkarni


tagsync, currently, commits to Kafka every handled event after successfully uploading to Ranger admin. The commits are synchronous and may impose performance penalty.

This Jira is to update event handling in tagsync by optmizing on the commits to Kafka while reducing the chances of Kafka delivering same message multiple times to tagsync.

This Jira also fixes a potential issue if the Kafka queue contains only a large number of notifications not handled by tagsync. In such cases, tagsync does not do any commit to kafka, and that may cause back-pressure on Kafka queue.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)