You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Anirudh P <pa...@gmail.com> on 2016/01/28 08:10:04 UTC

Question on Using many producers simultaneously

Hello,

We have a scenario where we will be having a large number of producers(
~50k instances - 1 Producer per instance) which will be sending data to a
kafka topic.
The producers could be inactive for a long time and then be asked to send a
message(all at the same time) and as quickly as possible.

What is the best way to do this? Should we keep the Producer connection
open throughout? Or do we create a new Producer each time we need to send a
message?
If the connection is not closed, I believe Kafka discards the connection
after a specified timeout. Also, Does Kafka have any limit on the number of
incoming connection requests? (I read a few blog posts that said that 300k
connections were handled without problems)

Thank you,
Anirudh