[jira] [Commented] (KAFKA-15693) Disabling scheduled rebalance delay in Connect can lead to indefinitely unassigned connectors and tasks

["Chris Egerton (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17779977#comment-17779977 ] 

Chris Egerton commented on KAFKA-15693:
---------------------------------------

For all affected versions, the most graceful workaround is to instead set the {{scheduled.rebalance.max.delay.ms}} property to an extremely low value (such as 1) instead of 0.

> Disabling scheduled rebalance delay in Connect can lead to indefinitely unassigned connectors and tasks
> -------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-15693
>                 URL: https://issues.apache.org/jira/browse/KAFKA-15693
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 2.3.0, 2.4.0, 2.3.1, 2.5.0, 2.4.1, 2.6.0, 2.5.1, 2.7.0, 2.6.1, 2.8.0, 2.7.1, 2.6.2, 3.1.0, 2.6.3, 2.7.2, 2.8.1, 3.0.0, 3.0.1, 2.8.2, 3.2.0, 3.1.1, 3.3.0, 3.0.2, 3.1.2, 3.2.1, 3.4.0, 3.2.2, 3.2.3, 3.3.1, 3.3.2, 3.5.0, 3.4.1, 3.6.0, 3.5.1, 3.7.0
>            Reporter: Chris Egerton
>            Assignee: Chris Egerton
>            Priority: Major
>
> Kafka Connect supports deferred resolution of imbalances when using the incremental rebalancing algorithm introduced in [KIP-415|https://cwiki.apache.org/confluence/display/KAFKA/KIP-415%3A+Incremental+Cooperative+Rebalancing+in+Kafka+Connect]. When enabled, this feature introduces a configurable delay period between when "lost" assignments (i.e., connectors and tasks that were assigned to a worker in the previous round of rebalance but are not assigned to a worker during the current round of rebalance) are detected and when they are reassigned to a worker. The delay can be configured with the {{scheduled.rebalance.max.delay.ms}} property.
> If this property is set to 0, then there should be no delay between when lost assignments are detected and when they are reassigned. Instead, however, this configuration can cause lost assignments to be withheld during a rebalance, remaining unassigned until the next rebalance, which, because scheduled delays are disabled, will not happen on its own and will only take place when unrelated conditions warrant it (such as the creation or deletion of a connector, a worker joining or leaving the cluster, new task configs being generated for a connector, etc.).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)