[jira] [Commented] (KAFKA-10413) rebalancing leads to unevenly balanced connectors

["yazgoo (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477980#comment-17477980 ] 

yazgoo commented on KAFKA-10413:
--------------------------------

Hi, any news on this issue ?
I'm suspecting it is linked with https://issues.apache.org/jira/browse/KAFKA-12495
The current way we found to mitigate this is to destroy / re-create the connectors.

> rebalancing leads to unevenly balanced connectors
> -------------------------------------------------
>
>                 Key: KAFKA-10413
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10413
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 2.5.1
>            Reporter: yazgoo
>            Assignee: rameshkrishnan muthusamy
>            Priority: Major
>             Fix For: 2.4.2, 2.5.2, 2.8.0, 2.7.1, 2.6.2
>
>         Attachments: connect_worker_balanced.png
>
>
> GHi,
> With CP 5.5, running kafka connect s3 sink on EC2 whith autoscaling enabled, if a connect instance disappear, or a new one appear, we're seeing unbalanced consumption, much like mentionned in this post:
> [https://stackoverflow.com/questions/58644622/incremental-cooperative-rebalancing-leads-to-unevenly-balanced-connectors]
> This usually leads to one kafka connect instance taking most of the load and consumption not being able to keep on.
> Currently, we're "fixing" this by deleting the connector and re-creating it, but this is far from ideal.
> Any suggestion on what we could do to mitigate this ?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)