You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "Yu, Libo " <li...@citi.com> on 2013/04/25 23:30:24 UTC

configuring kafka

Hi,

I have a few questions about configuring kafka 0.8.
Assume I have 3 brokers, 10 consumers and 4 topics.

1         How many partitions should a topic have for best performance?

Each broker has one partition? Or each producer should have a partition?

2         Batch size has big impact on performance, how to determine it optimally?

3         For multiple producers, do I need to increase num.network.threads and

num.io.threads from the default of 2?


Thanks.

Regards,

Libo


Re: configuring kafka

Posted by Jun Rao <ju...@gmail.com>.
1. Take a look at #6 in http://kafka.apache.org/faq.html

2. Typically, you want each batch to have 10s-100s KB.

3. In the latest 0.8, we have 3 network threads and 8 io threads. These
should be good enough for typical users.

Thanks,

Jun


On Thu, Apr 25, 2013 at 2:30 PM, Yu, Libo <li...@citi.com> wrote:

> Hi,
>
> I have a few questions about configuring kafka 0.8.
> Assume I have 3 brokers, 10 consumers and 4 topics.
>
> 1         How many partitions should a topic have for best performance?
>
> Each broker has one partition? Or each producer should have a partition?
>
> 2         Batch size has big impact on performance, how to determine it
> optimally?
>
> 3         For multiple producers, do I need to increase
> num.network.threads and
>
> num.io.threads from the default of 2?
>
>
> Thanks.
>
> Regards,
>
> Libo
>
>