[jira] [Commented] (KAFKA-9066) Kafka Connect JMX : source & sink task metrics missing for tasks in failed state

["Javier Holguera (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17061693#comment-17061693 ] 

Javier Holguera commented on KAFKA-9066:
----------------------------------------

Dealing with this problem as well. Seems to be related to the fact that the task is "gone" when it goes into Failed state. Not only "status", but all the other metrics associated with the task are gone.

> Kafka Connect JMX : source & sink task metrics missing for tasks in failed state
> --------------------------------------------------------------------------------
>
>                 Key: KAFKA-9066
>                 URL: https://issues.apache.org/jira/browse/KAFKA-9066
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 2.1.1
>            Reporter: Mikołaj Stefaniak
>            Priority: Major
>
> h2. Overview
> Kafka Connect exposes various metrics via JMX. Those metrics can be exported i.e. by _Prometheus JMX Exporter_ for further processing.
> One of crucial attributes is connector's *task status.*
> According to official Kafka docs, status is available as +status+ attribute of following MBean:
> {quote}kafka.connect:type=connector-task-metrics,connector="\{connector}",task="\{task}"status - The status of the connector task. One of 'unassigned', 'running', 'paused', 'failed', or 'destroyed'.
> {quote}
> h2. Issue
> Generally +connector-task-metrics+ are exposed propery for tasks in +running+ status but not exposed at all if task is +failed+.
> Failed Task *appears* properly with failed status when queried via *REST API*:
>  
> {code:java}
> $ curl -X GET -u 'user:pass' http://kafka-connect.mydomain.com/connectors/customerconnector/status
> {"name":"customerconnector","connector":{"state":"RUNNING","worker_id":"kafka-connect.mydomain.com:8080"},"tasks":[{"id":0,"state":"FAILED","worker_id":"kafka-connect.mydomain.com:8080","trace":"org.apache.kafka.connect.errors.ConnectException: Received DML 'DELETE FROM mysql.rds_sysinfo ......"}],"type":"source"}
> $ {code}
>  
> Failed Task *doesn't appear* as bean with +connector-task-metrics+ type when queried via *JMX*:
>  
> {code:java}
> $ echo "beans -d kafka.connect" | java -jar target/jmxterm-1.1.0-SNAPSHOT-uber.jar -l localhost:8081 -n -v silent | grep connector=customerconnector
> kafka.connect:connector=customerconnector,task=0,type=task-error-metricskafka.connect:connector=customerconnector,type=connector-metrics
> $
> {code}
> h2. Expected result
> It is expected, that bean with +connector-task-metrics+ type will appear also for tasks that failed.
> Below is example of how beans are properly registered for tasks in Running state:
>  
> {code:java}
> $ echo "get -b kafka.connect:connector=sinkConsentSubscription-1000,task=0,type=connector-task-metrics status" | java -jar target/jmxterm-1.1.0-SNAPSHOT-ube r.jar -l localhost:8081 -n -v silent
> status = running;
> $
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)