[jira] [Commented] (KAFKA-4429) records-lag should be zero if FetchResponse is empty

["Kevin Conaway (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-4429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15791620#comment-15791620 ] 

Kevin Conaway commented on KAFKA-4429:
--------------------------------------

I think it may be worth rethinking how the metrics are collected and represented.  Currently, the return type for the`Measurable#measure` method is a primitive `double` which means that a value must be present.  For stat values that use Min and Max (like records-lag), this means that they default to positive or negative infinity as the case may be.

In these two cases, the infinity values are used to represent the absence of a real value.  IMO, when there is no value we should return `Null` as most reporting / metrics systems can handle null.  I know that Graphite cannot properly handle infinity values but I can't speak for other systems.

[~junrao] who is the right person to talk to about this?

This is also related to KAFKA-3241

> records-lag should be zero if FetchResponse is empty
> ----------------------------------------------------
>
>                 Key: KAFKA-4429
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4429
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Dong Lin
>            Assignee: Dong Lin
>
> In Fetcher we record records-lag in terms of number of records for any partition. Currently this metric value is updated only if number of parsed records is not empty. This means that if consumer has already fully caught up and there is no new data into the topic, this metric's value will be negative infinity and users can not rely on this metric to know if their consumer has caught up.
> We can fix this problem by assuming the lag is zero is FetchResponse is empty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)