You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by David Yu <gu...@gmail.com> on 2016/08/19 23:35:08 UTC

Debug Kafka performance

Hi,

We have setup a 3-broker Kafka 0.9.0.0 cluster in Cloudera. The bokers are
ec2 d2.xlarge instances. What we are trying to do is to run some producer
perf tests on this cluster.

From the client host, we did a bandwidth test to the brokers using iperf3
and we are getting 900Mbps connections. For a 9-partition topic, the actual
test results shows:

start.time, end.time, compression, message.size, batch.size,
total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec
2016-08-19 23:22:42:203, 2016-08-19 23:24:08:330, 0, 5000, 200, 4768.37,
55.3644, 1000000, 11610.7609

The 55.36MBps throughput is definitely lower than we expected.

Does this results look about right?

Thanks,
David