You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Michael Bingham (Jira)" <ji...@apache.org> on 2020/05/06 23:45:00 UTC

[jira] [Created] (KAFKA-9965) Uneven distribution with RoundRobinPartitioner in AK 2.4+

Michael Bingham created KAFKA-9965:
--------------------------------------

             Summary: Uneven distribution with RoundRobinPartitioner in AK 2.4+
                 Key: KAFKA-9965
                 URL: https://issues.apache.org/jira/browse/KAFKA-9965
             Project: Kafka
          Issue Type: Bug
          Components: producer 
    Affects Versions: 2.4.1, 2.5.0, 2.4.0
            Reporter: Michael Bingham


{{RoundRobinPartitioner}} states that it will provide equal distribution of records across partitions. However with the enhancements made in KIP-480, it may not. In some cases, when a new batch is started, the partitioner may be called a second time for the same record:

[https://github.com/apache/kafka/blob/2.4/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L909]

[https://github.com/apache/kafka/blob/2.4/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L934]

Each time the partitioner is called, it increments a counter in {{RoundRobinPartitioner}}, so this can result in unequal distribution.

Easiest fix might be to decrement the counter in {{RoundRobinPartitioner#onNewBatch}}.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)