You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "YunlongLou (Jira)" <ji...@apache.org> on 2019/12/07 09:41:00 UTC

[jira] [Created] (KAFKA-9286) Improve producer performance

YunlongLou created KAFKA-9286:
---------------------------------

             Summary: Improve producer performance 
                 Key: KAFKA-9286
                 URL: https://issues.apache.org/jira/browse/KAFKA-9286
             Project: Kafka
          Issue Type: Improvement
          Components: clients
    Affects Versions: 2.2.2
            Reporter: YunlongLou


Pull request: [https://github.com/apache/kafka/pull/7797]

For producer, the `Sender` [call](https://github.com/apache/kafka/blob/ba365bbb8deb16ec133e0e7983c1c56cef0152aa/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java#L342) `RecordAccumulator#ready` to collect all ready nodes. Then it [call](https://github.com/apache/kafka/blob/ba365bbb8deb16ec133e0e7983c1c56cef0152aa/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java#L369) `RecordAccumulator#drain` to iterate all ready nodes's topic partition to check if there is data to send.

This process is redundancy. When we collect all ready nodes, its ready topic partition can also be collected.

In our case, the producer only produce data to part(about 20) of all partitions(1024), with this patch, the performance can be increased ~3.4%.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)