Empty Partitions Impact on Performance

[Pavel Sýkora <pa...@seznam.cz>]

Hi,



I've started playing with Kafka_2.10-0.8.2.1 (a cluster of 3 VM nodes). I 
found a strange behaviour: Inbound throughput was about 60 thousand messages
per second for a partition with 50 partitions. But after adding topics with 
950 partitions total, the performance dropped under 25 thousand messages per
second, even if the 950 new partitions were unused and empty. In other 
words: a simple existence of empty unused partitions can drop Kafka 
performance to less than 50 %.




Has anybody else observed such Kafka behaviour? Does anybody have an 
explanation for this?




Regards, 




Pavel 

Re: Empty Partitions Impact on Performance

[Stevo Slavić <ss...@gmail.com>]

I noticed similar behavior on similar small 3 Kafka broker with 3 ZooKeeper
node cluster, Kafka 0.8.1.1 and ZooKeeper 3.4.6, with ~5K topics, most of
them with single partition, replication factor of 1, and most of them
unused for long time, but brokers are busy and performance especially
producer performance is slim. Plan to investigate root cause. Would be nice
if it was documented, by design how many topics/partitions can be handled
by single Kafka broker node, and then also scalability limits/constraints.
Both Kafka and ZooKeeper need to be checked for limits.

Kind regards,
Stevo Slavic.

On Wed, May 6, 2015 at 1:32 PM, Pavel Sýkora <pa...@seznam.cz> wrote:

> Hi,
>
>
>
> I've started playing with Kafka_2.10-0.8.2.1 (a cluster of 3 VM nodes). I
> found a strange behaviour: Inbound throughput was about 60 thousand
> messages
> per second for a partition with 50 partitions. But after adding topics with
> 950 partitions total, the performance dropped under 25 thousand messages
> per
> second, even if the 950 new partitions were unused and empty. In other
> words: a simple existence of empty unused partitions can drop Kafka
> performance to less than 50 %.
>
>
>
>
> Has anybody else observed such Kafka behaviour? Does anybody have an
> explanation for this?
>
>
>
>
> Regards,
>
>
>
>
> Pavel
>