[jira] [Created] (GOBBLIN-1766) Define metric to measure lag from producing to consume change stream events

["Urmi Mustafi (Jira)" <ji...@apache.org>]

Urmi Mustafi created GOBBLIN-1766:
-------------------------------------

             Summary: Define metric to measure lag from producing to consume change stream events
                 Key: GOBBLIN-1766
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1766
             Project: Apache Gobblin
          Issue Type: Bug
          Components: gobblin-service
            Reporter: Urmi Mustafi
            Assignee: Abhishek Tiwari


Modify event for change stream events to add the transaction identifier and produceTimestamp. Use the transaction identifier instead of timestamp to dedup events as "at least once" delivery may result in multiple change stream events with different timestamps for the same row update in mysql. Instead use the timestamp to emit a metric measuring the time between producing the event and consuming on our monitor. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)