[jira] [Created] (KAFKA-12464) Enhance constrained sticky Assign algorithm

["Luke Chen (Jira)" <ji...@apache.org>]

Luke Chen created KAFKA-12464:
---------------------------------

             Summary: Enhance constrained sticky Assign algorithm
                 Key: KAFKA-12464
                 URL: https://issues.apache.org/jira/browse/KAFKA-12464
             Project: Kafka
          Issue Type: Improvement
    Affects Versions: 2.7.0
            Reporter: Luke Chen
            Assignee: Luke Chen


In KAFKA-9987, we did a great improvement for the case when all consumers were subscribed to same set of topics. The algorithm contains 4 phases:
 # Reassign as many previously owned partitions as possible, up to the maxQuota
 # Fill remaining members up to minQuota
 # If we ran out of unassigned partitions before filling all consumers, we need to start stealing partitions from the over-full consumers at max capacity
 # Otherwise we may have run out of unfilled consumers before assigning all partitions, in which case we should just distribute one partition each to all consumers at min capacity

 

Take an example for better understanding:

Current status: 2 consumers (C0, C1), and 10 topic partitions: t1p0, t1p1, ... t1p9

Suppose, current assignment is:

_C0: t1p0, t1p1, t1p2, t1p3, t1p4_

_C1: t1p5, t1p6, t1p7, t1p8, t1p9_

 

New, new consumer added: C2, so we'll do:
 # Reassign as many previously owned partitions as possible, up to the maxQuota
 # Fill remaining members up to minQuota
 # If we ran out of unassigned partitions before filling all consumers, we need to start stealing partitions from the over-full consumers at max capacity
 # Otherwise we may have run out of unfilled consumers before assigning all partitions, in which case we should just distribute one partition each to all consumers at min capacity

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)