[jira] [Commented] (KAFKA-10441) Connect consumers are missing the fetch metrics

["Pradyumna (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-10441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185897#comment-17185897 ] 

Pradyumna commented on KAFKA-10441:
-----------------------------------

Interestingly, the MBean is available for these metrics, and it is reporting correct numbers, but it is not available via the configured metric.reporters

 

jmxterm output:

$>get -b kafka.consumer:client-id=connector-consumer-s3connector-kptest1-0,topic=kptest1,type=consumer-fetch-manager-metrics *
#mbean = kafka.consumer:client-id=connector-consumer-s3connector-kptest1-0,topic=kptest1,type=consumer-fetch-manager-metrics:
bytes-consumed-total = 167.0;

bytes-consumed-rate = 0.0;

records-consumed-rate = 0.0;

fetch-size-max = NaN;

records-consumed-total = 1.0;

fetch-size-avg = NaN;

records-per-request-avg = NaN;

$>

 

> Connect consumers are missing the fetch metrics
> -----------------------------------------------
>
>                 Key: KAFKA-10441
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10441
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer, KafkaConnect
>    Affects Versions: 2.3.0
>            Reporter: Pradyumna
>            Priority: Minor
>
> Kafka connect workers create KafkaConsumers and consume from Kafka topics.
>  
> However, unlike regular KafkaConsumers which have a good set of metrics in the group consumer-fetch-manager-metrics (such as records-lag, records-consumed-total etc), these KafkaConsumers created by Kafka Connect do not have any of these consumer metrics.
>  
> I took a heap dump and saw that the "sensors" field in the metrics field of the KafkaConsumer objects created by Kafka Connect had only the connect metrics, and not the consumer fetch metrics.
>  
> I tried briefly to find out where in the Kafka code is the metrics object being reset or the sensors field of the metrics object overridden by kafka connect, but I could not locate anything yet.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)