[jira] [Commented] (KAFKA-5061) client.id should be set for Connect producers/consumers

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840813#comment-16840813 ] 

ASF GitHub Bot commented on KAFKA-5061:
---------------------------------------

rhauch commented on pull request #6097: KAFKA-5061 - Make default Worker Task client IDs distinct
URL: https://github.com/apache/kafka/pull/6097
 
 
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> client.id should be set for Connect producers/consumers
> -------------------------------------------------------
>
>                 Key: KAFKA-5061
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5061
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 0.10.2.1
>            Reporter: Ewen Cheslack-Postava
>            Priority: Major
>              Labels: needs-kip, newbie++
>
> In order to properly monitor individual tasks using the producer and consumer metrics, we need to have the framework disambiguate them. Currently when we create producers (https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L362) and create consumers (https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L371-L394) the client ID is not being set. You can override it for the entire worker via worker-level producer/consumer overrides, but you can't get per-task metrics.
> There are a couple of things we might want to consider doing here:
> 1. Provide default client IDs based on the worker group ID + task ID (providing uniqueness for multiple connect clusters up to the scope of the Kafka cluster they are operating on). This seems ideal since it's a good default; however it is a public-facing change and may need a KIP. Normally I would be less worried about this, but some folks may be relying on picking up metrics without this being set, in which case such a change would break their monitoring.
> 2. Allow overriding client.id on a per-connector basis. I'm not sure if this will really be useful or not -- it lets you differentiate between metrics for different connectors' tasks, but within a connector, all metrics would go to a single client.id. On the other hand, this makes the tasks act as a single group from the perspective of broker handling of client IDs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)