You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Jakob Molander <jm...@geus.dk.INVALID> on 2025/07/21 08:53:59 UTC
Kafka Consumer Not Receiving Messages – Request for Help
Hello everyone,
I'm currently working with a single Kafka 3.9.1 instance running in Docker, and I've set up a single topic named "jupiter-events".
My Kafka producer is successfully sending messages to the topic, and I've confirmed that these messages are being received by the Kafka instance.
However, I'm facing an issue where my Kafka consumer does not receive any messages when polling the topic. According to the logs, the consumer appears to be functioning correctly, but it always receives an empty list during polling.
I've attached the configuration files and logs for both the producer and the consumer for reference.
Could anyone kindly take a look and let me know if there’s something I might be missing?
Thank you in advance for your time and assistance!
Kind regards,
Jakob
Re: Kafka Consumer Not Receiving Messages – Request for Help
Posted by "Samudrala, Ranganath [USA]" <Sa...@bah.com.INVALID>.
Hello
Recently I had similar problem and I removed setting a value to client-id in consumer properties. I was setting both group-id and client-id and now I set only group-id and started receiving messages via poll.
Note: I found the solution by trial and error, so i have no good logical explanation. May be some one else can explain why this is not a solution or it is a bug, etc.
HTH
Ranga
________________________________
From: Jakob Molander <jm...@geus.dk.INVALID>
Sent: Monday, July 21, 2025 7:19 AM
To: users@kafka.apache.org <us...@kafka.apache.org>
Subject: [External] Re: Kafka Consumer Not Receiving Messages – Request for Help
Hi Suresh, and others, Thank you for your reply. To provide some additional context: there is only one producer and one consumer running, and nothing else is currently interacting with this Kafka instance. Each time I run my test scenario, I
Hi Suresh, and others,
Thank you for your reply.
To provide some additional context: there is only one producer and one consumer running, and nothing else is currently interacting with this Kafka instance.
Each time I run my test scenario, I start from a clean slate:
1. Deploy the Kafka instance in Docker
2. Create the "jupiter-events" topic
3. Start the consumer instance
4. Start the producer instance
5. Send a few test messages from the producer
Despite this setup, the consumer still receives an empty list when polling. I’d appreciate any suggestions or insights on what might be going wrong.
Thanks again for your time and support!
Kind regards,
Jakob
________________________________
Fra: Suresh Chidambaram <ch...@gmail.com>
Sendt: 21. juli 2025 12:06
Til: users@kafka.apache.org <us...@kafka.apache.org>
Emne: Re: Kafka Consumer Not Receiving Messages – Request for Help
Hi Jakob,
The consumer is not consuming the messages because the messages might have
already been consumed using the consumer group, jupiter-workers. So, the
options below will help you read the messages.
1. Change the group.id from jupiter-workers to something else like
jupiter-workers-latest
2. Reset the consumer offset of the consumer group "jupiter-workers" of
Topic "jupiter-events" to the beginning offset, and then start the consumer
application
Please reach out if any more details are needed.
Thanks
C Suresh
+91 9042189457
On Mon, Jul 21, 2025 at 2:24 PM Jakob Molander <jm...@geus.dk.invalid> wrote:
> Hello everyone,
>
> I'm currently working with a single Kafka 3.9.1 instance running in
> Docker, and I've set up a single topic named "jupiter-events".
>
> My Kafka producer is successfully sending messages to the topic, and I've
> confirmed that these messages are being received by the Kafka instance.
>
> However, I'm facing an issue where my Kafka consumer does not receive any
> messages when polling the topic. According to the logs, the consumer
> appears to be functioning correctly, but it always receives an empty list
> during polling.
>
> I've attached the configuration files and logs for both the producer and
> the consumer for reference.
>
> Could anyone kindly take a look and let me know if there’s something I
> might be missing?
>
> Thank you in advance for your time and assistance!
>
> Kind regards,
> Jakob
>
Re: Kafka Consumer Not Receiving Messages – Request for Help
Posted by Franz van Betteraey <fr...@van-betteraey.de>.
I would suggest that you use the
command line tools located in the Kafka distribution to analyze the state of your topic and consumer when running.
Just unpack the distribution and you will find them in the bin folder.
kafka-topics and kafka-consumer-group have a describe option that will probably help you. (out of memory but should be at least something similar).
Good luck.
Am 21. Juli 2025 14:42:09 MESZ schrieb Jakob Molander <jm...@geus.dk.INVALID>:
>Hi Suresh, and others,
>
>I've attached the Java code snippets for both the producer and the consumer, along with the output from the most recent run.
>
>As shown in the producer log, the messages are being sent successfully—they are assigned to a partition and given an offset as expected. However, the consumer continues to receive only empty results when polling the topic.
>
>The topic is configured with a single partition, since there is only one consumer involved in this setup.
>
>I would really appreciate any suggestions or insights into what might be going wrong here.
>
>Thank you again for your time and support!
>
>Kind regards,
>Jakob
>
>
>________________________________
>Fra: Suresh Chidambaram <ch...@gmail.com>
>Sendt: 21. juli 2025 13:55
>Til: Jakob Molander <jm...@geus.dk>
>Emne: Re: Kafka Consumer Not Receiving Messages – Request for Help
>
>Hi Jakob,
>
>If it's possible for you to show a demo on the issue, I may get some ideas and I may help you. Please let me know if you are okay with that.
>
>Thanks
>C Suresh
>
>
>On Mon, Jul 21, 2025 at 4:49 PM Jakob Molander <jm...@geus.dk.invalid> wrote:
>Hi Suresh, and others,
>
>Thank you for your reply.
>
>To provide some additional context: there is only one producer and one consumer running, and nothing else is currently interacting with this Kafka instance.
>
>Each time I run my test scenario, I start from a clean slate:
>
>1. Deploy the Kafka instance in Docker
>2. Create the "jupiter-events" topic
>3. Start the consumer instance
>4. Start the producer instance
>5. Send a few test messages from the producer
>
>Despite this setup, the consumer still receives an empty list when polling. I’d appreciate any suggestions or insights on what might be going wrong.
>
>Thanks again for your time and support!
>
>Kind regards,
>Jakob
>________________________________
>Fra: Suresh Chidambaram <ch...@gmail.com>>
>Sendt: 21. juli 2025 12:06
>Til: users@kafka.apache.org<ma...@kafka.apache.org> <us...@kafka.apache.org>>
>Emne: Re: Kafka Consumer Not Receiving Messages – Request for Help
>
>Hi Jakob,
>
>The consumer is not consuming the messages because the messages might have
>already been consumed using the consumer group, jupiter-workers. So, the
>options below will help you read the messages.
>
>1. Change the group.id<https://url41.mailanyone.net/scanner?m=1udp63-000000007Sm-3Q5u&d=4%7Cmail%2F90%2F1753098600%2F1udp63-000000007Sm-3Q5u%7Cin41g%7C57e1b682%7C15209072%7C14343128%7C687E2A6BF4A95B9BA494CE56E4E9A2FA&o=gphto%2F%2Frt%3Adi.pu&s=JHEhn3KFBMCNF4K2oEY0X2mv058> from jupiter-workers to something else like
>jupiter-workers-latest
>2. Reset the consumer offset of the consumer group "jupiter-workers" of
>Topic "jupiter-events" to the beginning offset, and then start the consumer
>application
>
>Please reach out if any more details are needed.
>
>Thanks
>C Suresh
>+91 9042189457
>
>
>
>
>
>On Mon, Jul 21, 2025 at 2:24 PM Jakob Molander <jm...@geus.dk.invalid> wrote:
>
>> Hello everyone,
>>
>> I'm currently working with a single Kafka 3.9.1 instance running in
>> Docker, and I've set up a single topic named "jupiter-events".
>>
>> My Kafka producer is successfully sending messages to the topic, and I've
>> confirmed that these messages are being received by the Kafka instance.
>>
>> However, I'm facing an issue where my Kafka consumer does not receive any
>> messages when polling the topic. According to the logs, the consumer
>> appears to be functioning correctly, but it always receives an empty list
>> during polling.
>>
>> I've attached the configuration files and logs for both the producer and
>> the consumer for reference.
>>
>> Could anyone kindly take a look and let me know if there’s something I
>> might be missing?
>>
>> Thank you in advance for your time and assistance!
>>
>> Kind regards,
>> Jakob
>>
Re: Kafka Consumer Not Receiving Messages – Request for Help
Posted by Jakob Molander <jm...@geus.dk.INVALID>.
Hi Suresh, and others,
I've attached the Java code snippets for both the producer and the consumer, along with the output from the most recent run.
As shown in the producer log, the messages are being sent successfully—they are assigned to a partition and given an offset as expected. However, the consumer continues to receive only empty results when polling the topic.
The topic is configured with a single partition, since there is only one consumer involved in this setup.
I would really appreciate any suggestions or insights into what might be going wrong here.
Thank you again for your time and support!
Kind regards,
Jakob
________________________________
Fra: Suresh Chidambaram <ch...@gmail.com>
Sendt: 21. juli 2025 13:55
Til: Jakob Molander <jm...@geus.dk>
Emne: Re: Kafka Consumer Not Receiving Messages – Request for Help
Hi Jakob,
If it's possible for you to show a demo on the issue, I may get some ideas and I may help you. Please let me know if you are okay with that.
Thanks
C Suresh
On Mon, Jul 21, 2025 at 4:49 PM Jakob Molander <jm...@geus.dk.invalid> wrote:
Hi Suresh, and others,
Thank you for your reply.
To provide some additional context: there is only one producer and one consumer running, and nothing else is currently interacting with this Kafka instance.
Each time I run my test scenario, I start from a clean slate:
1. Deploy the Kafka instance in Docker
2. Create the "jupiter-events" topic
3. Start the consumer instance
4. Start the producer instance
5. Send a few test messages from the producer
Despite this setup, the consumer still receives an empty list when polling. I’d appreciate any suggestions or insights on what might be going wrong.
Thanks again for your time and support!
Kind regards,
Jakob
________________________________
Fra: Suresh Chidambaram <ch...@gmail.com>>
Sendt: 21. juli 2025 12:06
Til: users@kafka.apache.org<ma...@kafka.apache.org> <us...@kafka.apache.org>>
Emne: Re: Kafka Consumer Not Receiving Messages – Request for Help
Hi Jakob,
The consumer is not consuming the messages because the messages might have
already been consumed using the consumer group, jupiter-workers. So, the
options below will help you read the messages.
1. Change the group.id<https://url41.mailanyone.net/scanner?m=1udp63-000000007Sm-3Q5u&d=4%7Cmail%2F90%2F1753098600%2F1udp63-000000007Sm-3Q5u%7Cin41g%7C57e1b682%7C15209072%7C14343128%7C687E2A6BF4A95B9BA494CE56E4E9A2FA&o=gphto%2F%2Frt%3Adi.pu&s=JHEhn3KFBMCNF4K2oEY0X2mv058> from jupiter-workers to something else like
jupiter-workers-latest
2. Reset the consumer offset of the consumer group "jupiter-workers" of
Topic "jupiter-events" to the beginning offset, and then start the consumer
application
Please reach out if any more details are needed.
Thanks
C Suresh
+91 9042189457
On Mon, Jul 21, 2025 at 2:24 PM Jakob Molander <jm...@geus.dk.invalid> wrote:
> Hello everyone,
>
> I'm currently working with a single Kafka 3.9.1 instance running in
> Docker, and I've set up a single topic named "jupiter-events".
>
> My Kafka producer is successfully sending messages to the topic, and I've
> confirmed that these messages are being received by the Kafka instance.
>
> However, I'm facing an issue where my Kafka consumer does not receive any
> messages when polling the topic. According to the logs, the consumer
> appears to be functioning correctly, but it always receives an empty list
> during polling.
>
> I've attached the configuration files and logs for both the producer and
> the consumer for reference.
>
> Could anyone kindly take a look and let me know if there’s something I
> might be missing?
>
> Thank you in advance for your time and assistance!
>
> Kind regards,
> Jakob
>
Re: Kafka Consumer Not Receiving Messages – Request for Help
Posted by Jakob Molander <jm...@geus.dk.INVALID>.
Hi Suresh, and others,
Thank you for your reply.
To provide some additional context: there is only one producer and one consumer running, and nothing else is currently interacting with this Kafka instance.
Each time I run my test scenario, I start from a clean slate:
1. Deploy the Kafka instance in Docker
2. Create the "jupiter-events" topic
3. Start the consumer instance
4. Start the producer instance
5. Send a few test messages from the producer
Despite this setup, the consumer still receives an empty list when polling. I’d appreciate any suggestions or insights on what might be going wrong.
Thanks again for your time and support!
Kind regards,
Jakob
________________________________
Fra: Suresh Chidambaram <ch...@gmail.com>
Sendt: 21. juli 2025 12:06
Til: users@kafka.apache.org <us...@kafka.apache.org>
Emne: Re: Kafka Consumer Not Receiving Messages – Request for Help
Hi Jakob,
The consumer is not consuming the messages because the messages might have
already been consumed using the consumer group, jupiter-workers. So, the
options below will help you read the messages.
1. Change the group.id from jupiter-workers to something else like
jupiter-workers-latest
2. Reset the consumer offset of the consumer group "jupiter-workers" of
Topic "jupiter-events" to the beginning offset, and then start the consumer
application
Please reach out if any more details are needed.
Thanks
C Suresh
+91 9042189457
On Mon, Jul 21, 2025 at 2:24 PM Jakob Molander <jm...@geus.dk.invalid> wrote:
> Hello everyone,
>
> I'm currently working with a single Kafka 3.9.1 instance running in
> Docker, and I've set up a single topic named "jupiter-events".
>
> My Kafka producer is successfully sending messages to the topic, and I've
> confirmed that these messages are being received by the Kafka instance.
>
> However, I'm facing an issue where my Kafka consumer does not receive any
> messages when polling the topic. According to the logs, the consumer
> appears to be functioning correctly, but it always receives an empty list
> during polling.
>
> I've attached the configuration files and logs for both the producer and
> the consumer for reference.
>
> Could anyone kindly take a look and let me know if there’s something I
> might be missing?
>
> Thank you in advance for your time and assistance!
>
> Kind regards,
> Jakob
>
Re: Kafka Consumer Not Receiving Messages – Request for Help
Posted by Suresh Chidambaram <ch...@gmail.com>.
Hi Jakob,
The consumer is not consuming the messages because the messages might have
already been consumed using the consumer group, jupiter-workers. So, the
options below will help you read the messages.
1. Change the group.id from jupiter-workers to something else like
jupiter-workers-latest
2. Reset the consumer offset of the consumer group "jupiter-workers" of
Topic "jupiter-events" to the beginning offset, and then start the consumer
application
Please reach out if any more details are needed.
Thanks
C Suresh
+91 9042189457
On Mon, Jul 21, 2025 at 2:24 PM Jakob Molander <jm...@geus.dk.invalid> wrote:
> Hello everyone,
>
> I'm currently working with a single Kafka 3.9.1 instance running in
> Docker, and I've set up a single topic named "jupiter-events".
>
> My Kafka producer is successfully sending messages to the topic, and I've
> confirmed that these messages are being received by the Kafka instance.
>
> However, I'm facing an issue where my Kafka consumer does not receive any
> messages when polling the topic. According to the logs, the consumer
> appears to be functioning correctly, but it always receives an empty list
> during polling.
>
> I've attached the configuration files and logs for both the producer and
> the consumer for reference.
>
> Could anyone kindly take a look and let me know if there’s something I
> might be missing?
>
> Thank you in advance for your time and assistance!
>
> Kind regards,
> Jakob
>