You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Thamaraikannan Subramanian <th...@gmail.com> on 2015/11/13 11:04:00 UTC
Kafka Scalability & Partition
All,
I am looking for clarifications. I want to have Highly Scalable Kafka
Environment, but I will have my topics without Partitions as my Application
mayn't send the Partition Id.
My question, will the Kafka Scale (Kafka Clusters) when there are Multiple
Topics but they doesn't have any Partition (in other words single
Partition)?
Please clarify. Thanks in Advance.
Re: Kafka Scalability & Partition
Posted by Prabhjot Bharaj <pr...@gmail.com>.
Hi,
Having such a model will not scale. I think it it's mentioned in earlier
posts as well as some wikis available out there
Kafka works very well if you have more partitions, so you can massively
parallel writes to Kafka.
Also, your application need not send partition ids/numbers, unless you've
some logic built over it. Kafka producer client will take care of that
Regards,
Prabhjot
On Nov 13, 2015 3:34 PM, "Thamaraikannan Subramanian" <
thamarai.bally@gmail.com> wrote:
> All,
>
> I am looking for clarifications. I want to have Highly Scalable Kafka
> Environment, but I will have my topics without Partitions as my Application
> mayn't send the Partition Id.
>
> My question, will the Kafka Scale (Kafka Clusters) when there are Multiple
> Topics but they doesn't have any Partition (in other words single
> Partition)?
>
> Please clarify. Thanks in Advance.
>
Re: Kafka Scalability & Partition
Posted by Prabhjot Bharaj <pr...@gmail.com>.
Hi,
Having such a model will not scale. I think it it's mentioned in earlier
posts as well as some wikis available out there
Kafka works very well if you have more partitions, so you can massively
parallel writes to Kafka.
Also, your application need not send partition ids/numbers, unless you've
some logic built over it. Kafka producer client will take care of that
Regards,
Prabhjot
On Nov 13, 2015 3:34 PM, "Thamaraikannan Subramanian" <
thamarai.bally@gmail.com> wrote:
> All,
>
> I am looking for clarifications. I want to have Highly Scalable Kafka
> Environment, but I will have my topics without Partitions as my Application
> mayn't send the Partition Id.
>
> My question, will the Kafka Scale (Kafka Clusters) when there are Multiple
> Topics but they doesn't have any Partition (in other words single
> Partition)?
>
> Please clarify. Thanks in Advance.
>