[jira] [Created] (KAFKA-21194) Speed up the balanced task assignor and register it as a built-in assignor

["Gavin Wang (Jira)" <ji...@apache.org>]

Gavin Wang created KAFKA-21194:
----------------------------------

             Summary: Speed up the balanced task assignor and register it as a built-in assignor
                 Key: KAFKA-21194
                 URL: https://issues.apache.org/jira/browse/KAFKA-21194
             Project: Kafka
          Issue Type: Sub-task
          Components: streams
    Affects Versions: 4.5.0
            Reporter: Gavin Wang
            Assignee: Gavin Wang


KAFKA-20998 added \{{BalancedTaskAssignor}}, a one-to-one translation of the classic \{{HighAvailabilityTaskAssignor}} placement, not yet registered as a built-in. Its fix-up loop tests every pair of processes on each pass, so it costs Θ(P²) per pass even when nothing moves: up to 26x the sticky assignor's time in JMH (12.4 ms for 1000 single-member processes).

Plan:
# Index-based implementation and an O(P) early exit for passes that cannot move a task. Same assignment.
# Place active tasks with the Sainte-Laguë divisor method (next task to the process with the smallest (2n+1)/c), read from a precomputed period, instead of round-robin plus fix-up. Same placement for equal member counts; for unequal ones the same per-member balance and a more even spread of each subtopology. Separate PR, as it changes placement.
# Register \{{balanced}} in \{{GroupCoordinatorConfig.STREAMS_GROUP_BUILTIN_ASSIGNORS}} once KIP-1071 is amended from \{{highly_available}}.

Prototypes: 1 is 6-10x faster, 1+2 is 12-19x, about a quarter of the sticky assignor's time.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)