You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Andrew Olson (JIRA)" <ji...@apache.org> on 2019/07/01 15:11:01 UTC
[jira] [Commented] (KAFKA-8601) Producer Improvement: Sticky
Partitioner
[ https://issues.apache.org/jira/browse/KAFKA-8601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876259#comment-16876259 ]
Andrew Olson commented on KAFKA-8601:
-------------------------------------
I think this is a worthwhile enhancement. Netflix did something similar,
https://medium.com/netflix-techblog/kafka-inside-keystone-pipeline-dd5aeabaf6bb (see "Producer sticky partitioner" section)
Our open source common-kafka project provides a partitioner implementation that follows this approach.
https://github.com/cerner/common-kafka/blob/2.0/common-kafka/src/main/java/com/cerner/common/kafka/producer/partitioners/FairPartitioner.java
> Producer Improvement: Sticky Partitioner
> ----------------------------------------
>
> Key: KAFKA-8601
> URL: https://issues.apache.org/jira/browse/KAFKA-8601
> Project: Kafka
> Issue Type: Improvement
> Reporter: Justine Olshan
> Assignee: Justine Olshan
> Priority: Major
>
> Currently the default partitioner uses a round-robin strategy to partition non-keyed values. The idea is to implement a "sticky partitioner" that chooses a partition for a topic and sends all records to that partition until the batch is sent. Then a new partition is chosen. This new partitioner will increase batching and decrease latency.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)