Kafka consumer performance with large network delay

[Boris Sorochkin <sb...@gmail.com>]

Hi All,
I have Kafka setup with default settings and relatively large network delay
(up to dozens milliseconds) between a broker and consumer. This setup has
very poor performance (~30kbps per client) despite of no packet loss on the
way (the network is very reliable).
After investigation of the traffic on wire I discovered, that Kafka mostly
sends messages one per packet, waiting each ack sequentially. In my case
messages in Kafka may be pretty small (800-900 bytes).

I'm using Kafka version 8.2.2.2 and see the following setting
*channel.socket.setTcpNoDelay(true)* in
*core/src/main/scala/kafka/network/BlockingChannel.scala*

Summarizing, it seems that Kafka not only disables' Nagle's TCP
optimization but sending small packets not even utilizing whole available
MTU.

Probably I'm not getting something, so what is the best way to configure
Kafka to utilize network properly?

Thanks in advance
Regards,

Re: Kafka consumer performance with large network delay

[Ewen Cheslack-Postava <ew...@confluent.io>]

Kafka will batch messages, but if the rate of delivery is too slow it'll
fall back to delivering only one message per batch. What is the total
throughput per broker?

-Ewen

On Fri, Jul 15, 2016 at 5:21 PM, Boris Sorochkin <sb...@gmail.com> wrote:

> Hi All,
> I have Kafka setup with default settings and relatively large network delay
> (up to dozens milliseconds) between a broker and consumer. This setup has
> very poor performance (~30kbps per client) despite of no packet loss on the
> way (the network is very reliable).
> After investigation of the traffic on wire I discovered, that Kafka mostly
> sends messages one per packet, waiting each ack sequentially. In my case
> messages in Kafka may be pretty small (800-900 bytes).
>
> I'm using Kafka version 8.2.2.2 and see the following setting
> *channel.socket.setTcpNoDelay(true)* in
> *core/src/main/scala/kafka/network/BlockingChannel.scala*
>
> Summarizing, it seems that Kafka not only disables' Nagle's TCP
> optimization but sending small packets not even utilizing whole available
> MTU.
>
> Probably I'm not getting something, so what is the best way to configure
> Kafka to utilize network properly?
>
> Thanks in advance
> Regards,
>



-- 
Thanks,
Ewen