[GitHub] [incubator-pinot] kishoreg commented on issue #5550: Expose lag metrics when consuming from kafka

[GitBox <gi...@apache.org>]

kishoreg commented on issue #5550:
URL: https://github.com/apache/incubator-pinot/issues/5550#issuecomment-643399886


   Relevant links
   - https://twitter.com/KishoreBytes/status/1250557828500090880
   - https://github.com/lightbend/kafka-lag-exporter
   - https://github.com/linkedin/Burrow/wiki
   
   All existing methods are flawed in one way or the other. Setting alerts on them will trigger false alarms in edge cases.
   The right solution is to some 
   
   @pradeepgv42 offsets might not be comparable across different implementations. Maybe, we can ask each implementation to provide the lag metric.
   
   At a high level, there are two things 
   - kafka offset
   - event time
   
   purely comparing offset will not during bootstrap time but is a good start
    
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org