You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Sudarshan Vasudevan (Jira)" <ji...@apache.org> on 2020/04/04 00:20:00 UTC

[jira] [Closed] (GOBBLIN-1087) Track and report histogram of observed lag from Gobblin Kafka pipeline

     [ https://issues.apache.org/jira/browse/GOBBLIN-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sudarshan Vasudevan closed GOBBLIN-1087.
----------------------------------------

> Track and report histogram of observed lag from Gobblin Kafka pipeline
> ----------------------------------------------------------------------
>
>                 Key: GOBBLIN-1087
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1087
>             Project: Apache Gobblin
>          Issue Type: Improvement
>          Components: gobblin-kafka
>    Affects Versions: 0.15.0
>            Reporter: Sudarshan Vasudevan
>            Assignee: Shirshanka Das
>            Priority: Major
>             Fix For: 0.15.0
>
>          Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> In this PR, we instrument the KafkaExtractor to track the observed latency of Kafka consumer records processed by the pipeline. Here, observed latency is measured as the time difference between processing time of the record and the original creation time. The latency distribution is tracked in an HdrHistogram, which is serialized into a string when emitted as part of a GobblinTrackingEvent.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)