[jira] [Work logged] (GOBBLIN-1766) Define metric to measure lag from producing to consume change stream events

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GOBBLIN-1766?focusedWorklogId=839209&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-839209 ]

ASF GitHub Bot logged work on GOBBLIN-1766:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 14/Jan/23 02:29
            Start Date: 14/Jan/23 02:29
    Worklog Time Spent: 10m 
      Work Description: umustafi commented on PR #3625:
URL: https://github.com/apache/gobblin/pull/3625#issuecomment-1382638995

   @ZihanLi58 @phet can you review?




Issue Time Tracking
-------------------

    Worklog Id:     (was: 839209)
    Time Spent: 20m  (was: 10m)

> Define metric to measure lag from producing to consume change stream events
> ---------------------------------------------------------------------------
>
>                 Key: GOBBLIN-1766
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1766
>             Project: Apache Gobblin
>          Issue Type: Bug
>          Components: gobblin-service
>            Reporter: Urmi Mustafi
>            Assignee: Abhishek Tiwari
>            Priority: Major
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Modify event for change stream events to add the transaction identifier and produceTimestamp. Use the transaction identifier instead of timestamp to dedup events as "at least once" delivery may result in multiple change stream events with different timestamps for the same row update in mysql. Instead use the timestamp to emit a metric measuring the time between producing the event and consuming on our monitor. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)