You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Artem Livshits (Jira)" <ji...@apache.org> on 2022/05/06 00:44:00 UTC

[jira] [Created] (KAFKA-13880) DefaultStreamPartitioner may get "stuck" to one partition for unkeyed messages

Artem Livshits created KAFKA-13880:
--------------------------------------

             Summary: DefaultStreamPartitioner may get "stuck" to one partition for unkeyed messages
                 Key: KAFKA-13880
                 URL: https://issues.apache.org/jira/browse/KAFKA-13880
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 2.4.0
            Reporter: Artem Livshits


While working on KIP-794, I noticed that DefaultStreamPartitioner does not call .onNewBatch.  The "sticky" DefaultStreamPartitioner introduced as a result of https://issues.apache.org/jira/browse/KAFKA-8601 requires .onNewBatch call in order to switch to a new partitions for unkeyed messages, just calling .partition would return the same "sticky" partition chosen during the first call to .partition.  The partition doesn't change even if the partition leader is unavailable.

Ideally, for unkeyed messages the DefaultStreamPartitioner should take advantage of the new built-in partitioning logic introduced in [https://github.com/apache/kafka/pull/12049.]  Perhaps, it could return null partition for unkeyed message, so that KafkaProducer could run built-in partitioning logic.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)