[jira] [Created] (KAFKA-6779) Kafka connect JMX metrics returns inconsistent metrics while running multiple connectors

["Sairam Polavarapu (JIRA)" <ji...@apache.org>]

Sairam Polavarapu created KAFKA-6779:
----------------------------------------

             Summary: Kafka connect JMX metrics returns inconsistent metrics while running multiple connectors
                 Key: KAFKA-6779
                 URL: https://issues.apache.org/jira/browse/KAFKA-6779
             Project: Kafka
          Issue Type: Bug
          Components: KafkaConnect
    Affects Versions: 1.0.1, 1.1.0, 1.0.0
         Environment: Linux
            Reporter: Sairam Polavarapu


JMX Kafka Connect metrics is returning different values for different runs. We are running the Kafka connect in distributed mode on a 3 node cluster which allowed us the submit the connectors using rest API calls. One of the use cases, where we are using MySQL Source Connector (Confluent) and Amazon s3 Sink connector (Confluent) as pipeline and need to get the counts of data received by source and delivered to sink. Both the connectors are submitted one after the other and the scenario what we have observed is when a new connector is submitted using the rest call the existing connector metrics are zeroes and restarted. This is a known case when the actual connectors are restarted but in our case the source connector was not restarted. We have observed the similar behavior when using Kafka 1.0.0, 1.0.1 and even after the fixes released in Kafka 1.1.0 still the issue persists. And also on a side note when the Metrics were tested on the local machine with Kafka connect in distributed mode on a single instance the counts are matching between the source and sink. This issue is only on how JMX is fetching the data. The physical data that is being handled through the connectors don't have any issues.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)