You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Andrew Olson (JIRA)" <ji...@apache.org> on 2019/07/01 15:11:01 UTC

[jira] [Commented] (KAFKA-8601) Producer Improvement: Sticky Partitioner

    [ https://issues.apache.org/jira/browse/KAFKA-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876259#comment-16876259 ] 

Andrew Olson commented on KAFKA-8601:
-------------------------------------

I think this is a worthwhile enhancement. Netflix did something similar,
https://medium.com/netflix-techblog/kafka-inside-keystone-pipeline-dd5aeabaf6bb (see "Producer sticky partitioner" section)

Our open source common-kafka project provides a partitioner implementation that follows this approach.
https://github.com/cerner/common-kafka/blob/2.0/common-kafka/src/main/java/com/cerner/common/kafka/producer/partitioners/FairPartitioner.java

> Producer Improvement: Sticky Partitioner
> ----------------------------------------
>
>                 Key: KAFKA-8601
>                 URL: https://issues.apache.org/jira/browse/KAFKA-8601
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Justine Olshan
>            Assignee: Justine Olshan
>            Priority: Major
>
> Currently the default partitioner uses a round-robin strategy to partition non-keyed values. The idea is to implement a "sticky partitioner" that chooses a partition for a topic and sends all records to that partition until the batch is sent. Then a new partition is chosen. This new partitioner will increase batching and decrease latency. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)