You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Alberto Moio <al...@techedgegroup.com> on 2021/05/25 10:10:21 UTC
Monitoring Consumer
Hello, I have a question about monitoring.
Can I know when a consumer or consumer-group reads a message in a topic?
Is there an API REST or a way to do this monitoring?
Thank you
[image: Techedge | Inspiring Trust. Globally.]
<http://www.techedgegroup.com/>
*Alberto Moio*
*Data Analytics | Data Platform*
alberto.moio@techedgegroup.com
*Mobile +39 3499365766 <+39%20342%207460285>*
techedgegroup.com <http://www.techedgegroup.com/>
Message for the intended recipient only. If received in error, please
contact the sender and read www.techedgegroup.com/disclaimer
Re: [External] Monitoring Consumer
Posted by "Tauzell, Dave" <Da...@surescripts.com>.
If you know the offset and partition of the message then you could use the api to determine when a consumer has acknowledged the message.
From: Jason Turim <ja...@signalvine.com>
Date: Tuesday, May 25, 2021 at 7:31 AM
To: users@kafka.apache.org <us...@kafka.apache.org>
Subject: Re: [External] Monitoring Consumer
One approach is to have the consumer publish notifications about messages
it has processed to a new topic (or other storage mechanism).
You may be able to use the admin api, but I don't think it's a standard use
case.
On Tue, May 25, 2021, 8:21 AM Tauzell, Dave <Da...@surescripts.com>
wrote:
> I don’t know about monitoring when a particular message is reads but you
> can use something like https://github.com/linkedin/Burrow to monitor
> consumer lag. Basically you can see that consumer Y has not yet read X
> number of messages that are ready.
>
> -Dave
>
> From: Alberto Moio <al...@techedgegroup.com>
> Date: Tuesday, May 25, 2021 at 5:11 AM
> To: users@kafka.apache.org <us...@kafka.apache.org>
> Subject: [External] Monitoring Consumer
> Hello, I have a question about monitoring.
>
> Can I know when a consumer or consumer-group reads a message in a topic?
>
> Is there an API REST or a way to do this monitoring?
>
> Thank you
> [image: Techedge | Inspiring Trust. Globally.]
> <http://www.techedgegroup.com/>
> *Alberto Moio*
> *Data Analytics | Data Platform*
> alberto.moio@techedgegroup.com
>
> *Mobile +39 3499365766 <+39%20342%207460285>*
>
> techedgegroup.com <http://www.techedgegroup.com/>
> Message for the intended recipient only. If received in error, please
> contact the sender and read www.techedgegroup.com/disclaimer<<http://www.techedgegroup.com/disclaimer%3c>
> http://www.techedgegroup.com/disclaimer>
> This e-mail and any files transmitted with it are confidential, may
> contain sensitive information, and are intended solely for the use of the
> individual or entity to whom they are addressed. If you have received this
> e-mail in error, please notify the sender by reply e-mail immediately and
> destroy all copies of the e-mail and any attachments.
>
This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.
Re: [External] Monitoring Consumer
Posted by Jason Turim <ja...@signalvine.com>.
One approach is to have the consumer publish notifications about messages
it has processed to a new topic (or other storage mechanism).
You may be able to use the admin api, but I don't think it's a standard use
case.
On Tue, May 25, 2021, 8:21 AM Tauzell, Dave <Da...@surescripts.com>
wrote:
> I don’t know about monitoring when a particular message is reads but you
> can use something like https://github.com/linkedin/Burrow to monitor
> consumer lag. Basically you can see that consumer Y has not yet read X
> number of messages that are ready.
>
> -Dave
>
> From: Alberto Moio <al...@techedgegroup.com>
> Date: Tuesday, May 25, 2021 at 5:11 AM
> To: users@kafka.apache.org <us...@kafka.apache.org>
> Subject: [External] Monitoring Consumer
> Hello, I have a question about monitoring.
>
> Can I know when a consumer or consumer-group reads a message in a topic?
>
> Is there an API REST or a way to do this monitoring?
>
> Thank you
> [image: Techedge | Inspiring Trust. Globally.]
> <http://www.techedgegroup.com/>
> *Alberto Moio*
> *Data Analytics | Data Platform*
> alberto.moio@techedgegroup.com
>
> *Mobile +39 3499365766 <+39%20342%207460285>*
>
> techedgegroup.com <http://www.techedgegroup.com/>
> Message for the intended recipient only. If received in error, please
> contact the sender and read www.techedgegroup.com/disclaimer<
> http://www.techedgegroup.com/disclaimer>
> This e-mail and any files transmitted with it are confidential, may
> contain sensitive information, and are intended solely for the use of the
> individual or entity to whom they are addressed. If you have received this
> e-mail in error, please notify the sender by reply e-mail immediately and
> destroy all copies of the e-mail and any attachments.
>
Re: [External] Monitoring Consumer
Posted by "Tauzell, Dave" <Da...@surescripts.com>.
I don’t know about monitoring when a particular message is reads but you can use something like https://github.com/linkedin/Burrow to monitor consumer lag. Basically you can see that consumer Y has not yet read X number of messages that are ready.
-Dave
From: Alberto Moio <al...@techedgegroup.com>
Date: Tuesday, May 25, 2021 at 5:11 AM
To: users@kafka.apache.org <us...@kafka.apache.org>
Subject: [External] Monitoring Consumer
Hello, I have a question about monitoring.
Can I know when a consumer or consumer-group reads a message in a topic?
Is there an API REST or a way to do this monitoring?
Thank you
[image: Techedge | Inspiring Trust. Globally.]
<http://www.techedgegroup.com/>
*Alberto Moio*
*Data Analytics | Data Platform*
alberto.moio@techedgegroup.com
*Mobile +39 3499365766 <+39%20342%207460285>*
techedgegroup.com <http://www.techedgegroup.com/>
Message for the intended recipient only. If received in error, please
contact the sender and read www.techedgegroup.com/disclaimer<http://www.techedgegroup.com/disclaimer>
This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.