Kafka issue

[Amoli Tandon <am...@gmail.com>]

Hi Team,

I am working on a POC and i am facing issues while producing messages to
Kafka cluster.

I am getting 'topic doesn't exist in the metadata after 60000ms' error when
trying to produce the message using 500 threads. Topic has a partition of
500.

The code is working fine with 100 threads and 100 partition topic.

Please suggest the resolution steps.

Regards,
Amoli.

Re: Kafka issue

[Luke Chen <sh...@gmail.com>]

Hi Amoli,

> The code is working fine with 100 threads and 100 partition topic.
It's good to know you have a workable solution.

For 500 threads in 500 partitions, could you firstly make sure the topic is
created successfully?
If topic cannot be created successfully, no records will be sent even if
you have 1 thread.
So, as long as the topic can be created successfully, I think it won't be a
problem with 500 threads as long as your machine is powerful enough.

Thank you.
Luke


On Wed, Dec 21, 2022 at 11:12 PM Amoli Tandon <am...@gmail.com> wrote:

> Hi Team,
>
> I am working on a POC and i am facing issues while producing messages to
> Kafka cluster.
>
> I am getting 'topic doesn't exist in the metadata after 60000ms' error when
> trying to produce the message using 500 threads. Topic has a partition of
> 500.
>
> The code is working fine with 100 threads and 100 partition topic.
>
> Please suggest the resolution steps.
>
> Regards,
> Amoli.
>