[GitHub] [incubator-pinot] mcvsubbu commented on issue #5948: Missing Consumer lag metrics

[GitBox <gi...@apache.org>]

mcvsubbu commented on issue #5948:
URL: https://github.com/apache/incubator-pinot/issues/5948#issuecomment-683512998


   This thing keeps making its appearance every once in a while.
   
   I assume that by `lag` you mean the difference between the highest offset available vs the offset consumed. This used to be exposed by Kafka in 0.9, but not anymore. We need to make a separate API call to get the highest offset, and that is something that is sub-optimal when it comes to high ingestion use cases.
   
   We could add some calls to our existing StreamConsumer interface and some implementations may choose to return the highest offset while others may not return it. If it is returned, a metric may be emitted.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org