You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by jaaz jozz <ja...@gmail.com> on 2019/01/27 14:53:51 UTC
Drawbacks for configuring many partitions for a topic
Hello,
In order to be prepared for large volume of messages i want to configure my
topic with large amount of partitions (>1000).
What are the drawbacks of this?
Should I except any pitfalls?
Best regards,
jazz
Re: Drawbacks for configuring many partitions for a topic
Posted by Sönke Liebau <so...@opencore.com.INVALID>.
Hi Jazz,
you probably already found this, but [1] is a good read and starting
point around this topic!
Best regards,
Sönke
[1] https://www.confluent.io/blog/how-choose-number-topics-partitions-kafka-cluster
On Sun, Jan 27, 2019 at 6:22 PM Ryanne Dolan <ry...@gmail.com> wrote:
>
> Jazz, the number of partitions isn't necessarily related to message volume.
> The biggest factors for max message volume would be the number of brokers
> and their write speed. You should choose the number of partitions based on
> the number of brokers and the number of consumers you expect to have.
>
> Ryanne
>
> On Sun, Jan 27, 2019, 8:55 AM jaaz jozz <jazzlofi2@gmail.com wrote:
>
> > Hello,
> > In order to be prepared for large volume of messages i want to configure my
> > topic with large amount of partitions (>1000).
> > What are the drawbacks of this?
> > Should I except any pitfalls?
> >
> > Best regards,
> > jazz
> >
--
Sönke Liebau
Partner
Tel. +49 179 7940878
OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany
Re: Drawbacks for configuring many partitions for a topic
Posted by Ryanne Dolan <ry...@gmail.com>.
Jazz, the number of partitions isn't necessarily related to message volume.
The biggest factors for max message volume would be the number of brokers
and their write speed. You should choose the number of partitions based on
the number of brokers and the number of consumers you expect to have.
Ryanne
On Sun, Jan 27, 2019, 8:55 AM jaaz jozz <jazzlofi2@gmail.com wrote:
> Hello,
> In order to be prepared for large volume of messages i want to configure my
> topic with large amount of partitions (>1000).
> What are the drawbacks of this?
> Should I except any pitfalls?
>
> Best regards,
> jazz
>