[jira] [Commented] (KAFKA-9792) Improve sticky task assignment for previous active tasks

["Matthias J. Sax (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-9792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072995#comment-17072995 ] 

Matthias J. Sax commented on KAFKA-9792:
----------------------------------------

Running the system test `stream_eos_test#test_failure_and_recovery()` I observed heavy task movement and multiple rebalanced after a client as added until the group stabilized. Attaching the relevant extracted logs for reference.

> Improve sticky task assignment for previous active tasks
> --------------------------------------------------------
>
>                 Key: KAFKA-9792
>                 URL: https://issues.apache.org/jira/browse/KAFKA-9792
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Sophie Blee-Goldman
>            Assignee: Sophie Blee-Goldman
>            Priority: Major
>         Attachments: rebalance.txt
>
>
> Due to the way the client quota is computed, we may end up passing over a client that is not yet full. This can cause us to “miss” the client while attempting to assign active tasks to their previous owners, and ultimately forcing an active task _away_ from its previous client.
> It’s not quite a bug, but it’s definitely sub-optimal behavior and can cause unnecessary task shuffling. It also makes the assignment harder to predict/debug
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)