You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Nikki Thean (JIRA)" <ji...@apache.org> on 2017/04/21 21:39:04 UTC

[jira] [Comment Edited] (KAFKA-5055) Kafka Streams skipped-records-rate sensor producing nonzero values even when FailOnInvalidTimestamp is used as extractor

    [ https://issues.apache.org/jira/browse/KAFKA-5055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979433#comment-15979433 ] 

Nikki Thean edited comment on KAFKA-5055 at 4/21/17 9:38 PM:
-------------------------------------------------------------

[~guozhang] Sure. Streams config properties were [https://gist.github.com/nixsticks/ccbfdbb6141e0f7362cef53d9e39d39b]. 

The nonzero value -- it depends. I have a topic that receives billions of records per day, not sure how many per second. The nonzero value tends to be very low when the lag is high (when I start reading the topic at earliest offset), and then rises to about 400 once I am consuming recent records.


was (Author: nthean):
[~guozhang] Sure. Streams config properties were (these)[https://gist.github.com/nixsticks/ccbfdbb6141e0f7362cef53d9e39d39b]. The nonzero value -- it depends. I have a topic that receives billions of records per day, not sure how many per second. The nonzero value tends to be very low when the lag is high (when I start reading the topic at earliest offset), and then rises to about 400 once I am consuming recent records.

> Kafka Streams skipped-records-rate sensor producing nonzero values even when FailOnInvalidTimestamp is used as extractor
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-5055
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5055
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>            Reporter: Nikki Thean
>
> According to the code and the documentation for this metric, the only reason for a skipped record is an invalid timestamp, except that a) I am reading from a topic that is populated solely by Kafka Connect and b) I am using `FailOnInvalidTimestamp` as the timestamp extractor.
> Either I'm missing something in the documentation (i.e. another reason for skipped records) or there is a bug in the code that calculates this metric.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)