You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by adrien ruffie <ad...@hotmail.fr> on 2018/03/02 09:46:58 UTC

Choosing topic/partition formula

Hi all,


I have a difficulty to represent an example of the calculation of the following formula.


Based on throughput requirements one can pick a rough number of partitions.

  1.  Lets call the throughput from producer to a single partition is P
  2.  Throughput from a single partition to a consumer is C
  3.  Target throughput is T
  4.  Required partitions = Max (T/P, T/C)


In the part Choosing topic/partition of this link:

https://community.hortonworks.com/articles/80813/kafka-best-practices-1.html


Could someone explain to me the calculation ? By showing me step by step the calculation, plz ? I do not remember how the formula applies visually ...

Maths, are so far :-)


Example T = 20MB/S      P = 5MB/S     and  C = 3MB/S


==> Max(20/5, 20/3)  = ???   ==> 20/5 because is the maximum of both ? consequently I need 4 partitions for my topic ? But it does not also depend of the number of producers & consumers ?




Thank all.


Best regards,


Adrien