blog on choosing # topics/partitions in Kafka

[Jun Rao <ju...@confluent.io>]

Since this is a commonly asked question in the mailing list, I summarized
some of the considerations in a bit more detail in the following blog.

http://blog.confluent.io/2015/03/12/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/

Thanks,

Jun

Re: blog on choosing # topics/partitions in Kafka

[Gwen Shapira <gs...@cloudera.com>]

Nice! Thank you, this is super helpful.

I'd add that not just the client can run out of memory with large
number of partitions - brokers can run out of memory too. We allocate
max.message.size * #partitions on each broker for replication.

Gwen

On Thu, Mar 12, 2015 at 8:53 AM, Jun Rao <ju...@confluent.io> wrote:
> Since this is a commonly asked question in the mailing list, I summarized
> some of the considerations in a bit more detail in the following blog.
>
> http://blog.confluent.io/2015/03/12/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/
>
> Thanks,
>
> Jun