You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Konstantine Karantasis (Jira)" <ji...@apache.org> on 2020/02/14 00:25:00 UTC

[jira] [Commented] (KAFKA-7644) Worker Re balance enhancements

    [ https://issues.apache.org/jira/browse/KAFKA-7644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036603#comment-17036603 ] 

Konstantine Karantasis commented on KAFKA-7644:
-----------------------------------------------

This ticket is similar to [#KAFKA-5505]
 and has now been fixed after [https://cwiki.apache.org/confluence/display/KAFKA/KIP-415:+Incremental+Cooperative+Rebalancing+in+Kafka+Connect] was implemented.

Closing as duplicate of [#KAFKA-5505] that is now fixed.

> Worker Re balance enhancements
> ------------------------------
>
>                 Key: KAFKA-7644
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7644
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>            Reporter: satya
>            Priority: Major
>
> Currently Kafka Connect distributed worker triggers a re balance any time there is a new connector/task is added irrespective of whether the connector added is a source connector or sink connector. 
> My understanding has been the worker re balance should be identical to consumer group re balance. That said, should not source connectors be immune to the re balance ?
> Are we not supposed to use source connectors with distributed workers ?
> It does appear to me there is some caveat in the way the worker re balance is working and it needs enhancement to not trigger unwanted re balances causing restarts of all tasks etc.
> Kafka connectors should have a way to not restart and stay with existing partition assignment if the re balance trigger is related to a different connector
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)