You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "Yu, Libo " <li...@citi.com> on 2013/04/25 23:30:24 UTC
configuring kafka
Hi,
I have a few questions about configuring kafka 0.8.
Assume I have 3 brokers, 10 consumers and 4 topics.
1 How many partitions should a topic have for best performance?
Each broker has one partition? Or each producer should have a partition?
2 Batch size has big impact on performance, how to determine it optimally?
3 For multiple producers, do I need to increase num.network.threads and
num.io.threads from the default of 2?
Thanks.
Regards,
Libo
Re: configuring kafka
Posted by Jun Rao <ju...@gmail.com>.
1. Take a look at #6 in http://kafka.apache.org/faq.html
2. Typically, you want each batch to have 10s-100s KB.
3. In the latest 0.8, we have 3 network threads and 8 io threads. These
should be good enough for typical users.
Thanks,
Jun
On Thu, Apr 25, 2013 at 2:30 PM, Yu, Libo <li...@citi.com> wrote:
> Hi,
>
> I have a few questions about configuring kafka 0.8.
> Assume I have 3 brokers, 10 consumers and 4 topics.
>
> 1 How many partitions should a topic have for best performance?
>
> Each broker has one partition? Or each producer should have a partition?
>
> 2 Batch size has big impact on performance, how to determine it
> optimally?
>
> 3 For multiple producers, do I need to increase
> num.network.threads and
>
> num.io.threads from the default of 2?
>
>
> Thanks.
>
> Regards,
>
> Libo
>
>