You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Amrit Gupta <gu...@gmail.com> on 2022/11/04 09:56:10 UTC
Regarding Producer Metric - "request-latency-avg"
Hi,
There is a producer metric
<https://github.com/apache/kafka/blob/14c36c7539cf5a7a54c2c7e30e115b3f4c4a9402/clients/src/main/java/org/apache/kafka/clients/producer/internals/SenderMetricsRegistry.java#L92>
( request-latency-avg ), this is not updated in producer metrics
documentation in confluent.
Also can I please get the detail of this metric that records the time spent
from which point to which point exactly?
Does it include the time it resides in the producer buffer? or it is from
the point it is removed from the producer buffer.
I mean does this include the time from application calling send() and the
record is sent to the broker?
Thanks,
Tushar
Re: Regarding Producer Metric - "request-latency-avg"
Posted by Amrit Gupta <gu...@gmail.com>.
So request-latency-avg is the time after drain?
Can you try to share a elaborated explanation of this metric calculation
that is being done.
On Tue, 8 Nov 2022 at 1:30 AM, Philip Nee <ph...@gmail.com> wrote:
> Hey Amrit,
>
> Actually, I think the confluent guide is correct because it seems like the
> request is created here
> <https://github.com/apache/kafka/blob/f7db6031b84a136ad0e257df722b20faa7c37b8a/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java#L847>
> .
> and the queue time
> <https://github.com/apache/kafka/blob/89933f21f204abf75336464d3ac24a4fdd254628/clients/src/main/java/org/apache/kafka/clients/producer/internals/ProducerBatch.java#L394>
> is actually the time difference between the batch creation and drain - each
> runOnce() in the sender loop first drains, then invoke sendProduceRequest.
> Sorry about the previous misleading reply.
>
> P
>
> On Mon, Nov 7, 2022 at 11:39 AM Amrit Gupta <gu...@gmail.com>
> wrote:
>
>> Bumping
>>
>> On Mon, 7 Nov 2022 at 12:45 PM, Amrit Gupta <gu...@gmail.com>
>> wrote:
>>
>>> Hi Philip,
>>>
>>> >> the latency measures the time difference between the *time the
>>> request was created* and the response received
>>> >> I believe it is *between the time send() was called*, and the time
>>> the producer receives the response from the broker.
>>>
>>> I believe this is not right as if we see
>>> <https://developer.confluent.io/learn-kafka/architecture/producer-hands-on/?_ga=2.143024061.907256740.1667543050-19124442.1666163656&_gac=1.54193882.1667550950.CjwKCAjw8JKbBhBYEiwAs3sxN21FphL9sHjqJWmlOYnVw9d474r-z9r8Hs1b3B5YbyYdxBOgrBxIORoCY-AQAvD_BwE#:~:text=1406.273-,request%2Dlatency%2Davg,-43.292>
>>> :
>>> [image: image.png]
>>>
>>> >> the producer doesn't immediately transmit the sends when send() is
>>> called, the requests can be batched depending on your configuration.
>>>
>>> if it was like the way you mentioned, then it implies it includes the
>>> producer buffer time also as you mentioned then *request-latency-avg*
>>> should be *always greater* than *request-queue-time-avg. *But as per
>>> the results above we can see in multiple cases, it is actually vice-versa.
>>>
>>>
>>> Please add more thoughts to this and let me know what you think.
>>>
>>> Thanks,
>>> T
>>>
>>
Re: Regarding Producer Metric - "request-latency-avg"
Posted by Philip Nee <ph...@gmail.com>.
Hey Amrit,
Actually, I think the confluent guide is correct because it seems like the
request is created here
<https://github.com/apache/kafka/blob/f7db6031b84a136ad0e257df722b20faa7c37b8a/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java#L847>
.
and the queue time
<https://github.com/apache/kafka/blob/89933f21f204abf75336464d3ac24a4fdd254628/clients/src/main/java/org/apache/kafka/clients/producer/internals/ProducerBatch.java#L394>
is actually the time difference between the batch creation and drain - each
runOnce() in the sender loop first drains, then invoke sendProduceRequest.
Sorry about the previous misleading reply.
P
On Mon, Nov 7, 2022 at 11:39 AM Amrit Gupta <gu...@gmail.com> wrote:
> Bumping
>
> On Mon, 7 Nov 2022 at 12:45 PM, Amrit Gupta <gu...@gmail.com>
> wrote:
>
>> Hi Philip,
>>
>> >> the latency measures the time difference between the *time the
>> request was created* and the response received
>> >> I believe it is *between the time send() was called*, and the time
>> the producer receives the response from the broker.
>>
>> I believe this is not right as if we see
>> <https://developer.confluent.io/learn-kafka/architecture/producer-hands-on/?_ga=2.143024061.907256740.1667543050-19124442.1666163656&_gac=1.54193882.1667550950.CjwKCAjw8JKbBhBYEiwAs3sxN21FphL9sHjqJWmlOYnVw9d474r-z9r8Hs1b3B5YbyYdxBOgrBxIORoCY-AQAvD_BwE#:~:text=1406.273-,request%2Dlatency%2Davg,-43.292>
>> :
>> [image: image.png]
>>
>> >> the producer doesn't immediately transmit the sends when send() is
>> called, the requests can be batched depending on your configuration.
>>
>> if it was like the way you mentioned, then it implies it includes the
>> producer buffer time also as you mentioned then *request-latency-avg*
>> should be *always greater* than *request-queue-time-avg. *But as per the
>> results above we can see in multiple cases, it is actually vice-versa.
>>
>>
>> Please add more thoughts to this and let me know what you think.
>>
>> Thanks,
>> T
>>
>
Re: Regarding Producer Metric - "request-latency-avg"
Posted by Amrit Gupta <gu...@gmail.com>.
Bumping
On Mon, 7 Nov 2022 at 12:45 PM, Amrit Gupta <gu...@gmail.com> wrote:
> Hi Philip,
>
> >> the latency measures the time difference between the *time the
> request was created* and the response received
> >> I believe it is *between the time send() was called*, and the time
> the producer receives the response from the broker.
>
> I believe this is not right as if we see
> <https://developer.confluent.io/learn-kafka/architecture/producer-hands-on/?_ga=2.143024061.907256740.1667543050-19124442.1666163656&_gac=1.54193882.1667550950.CjwKCAjw8JKbBhBYEiwAs3sxN21FphL9sHjqJWmlOYnVw9d474r-z9r8Hs1b3B5YbyYdxBOgrBxIORoCY-AQAvD_BwE#:~:text=1406.273-,request%2Dlatency%2Davg,-43.292>
> :
> [image: image.png]
>
> >> the producer doesn't immediately transmit the sends when send() is
> called, the requests can be batched depending on your configuration.
>
> if it was like the way you mentioned, then it implies it includes the
> producer buffer time also as you mentioned then *request-latency-avg*
> should be *always greater* than *request-queue-time-avg. *But as per the
> results above we can see in multiple cases, it is actually vice-versa.
>
>
> Please add more thoughts to this and let me know what you think.
>
> Thanks,
> T
>
Re: Regarding Producer Metric - "request-latency-avg"
Posted by Amrit Gupta <gu...@gmail.com>.
Hi Philip,
>> the latency measures the time difference between the *time the request
was created* and the response received
>> I believe it is *between the time send() was called*, and the time the
producer receives the response from the broker.
I believe this is not right as if we see
<https://developer.confluent.io/learn-kafka/architecture/producer-hands-on/?_ga=2.143024061.907256740.1667543050-19124442.1666163656&_gac=1.54193882.1667550950.CjwKCAjw8JKbBhBYEiwAs3sxN21FphL9sHjqJWmlOYnVw9d474r-z9r8Hs1b3B5YbyYdxBOgrBxIORoCY-AQAvD_BwE#:~:text=1406.273-,request%2Dlatency%2Davg,-43.292>
:
[image: image.png]
>> the producer doesn't immediately transmit the sends when send() is
called, the requests can be batched depending on your configuration.
if it was like the way you mentioned, then it implies it includes the
producer buffer time also as you mentioned then *request-latency-avg*
should be *always greater* than *request-queue-time-avg. *But as per the
results above we can see in multiple cases, it is actually vice-versa.
Please add more thoughts to this and let me know what you think.
Thanks,
T
Re: Regarding Producer Metric - "request-latency-avg"
Posted by Philip Nee <ph...@gmail.com>.
Hey Amrit,
Could you elaborate on "this is not updated in producer metrics
documentation in confluent"? The confluent documentation
<https://docs.confluent.io/platform/current/kafka/monitoring.html#consumer-global-request-metrics>states,
"The average request latency in ms", which seems right to me.
If you trace the code, the latency measures the time difference between the
time the request was created and the response received. I believe it is
between the time send() was called, and the time the producer receives the
response from the broker. And as you might know, the producer doesn't
immediately transmit the sends when send() is called, the requests can be
batched depending on your configuration.
Hope this helps,
P
On Sat, Nov 5, 2022 at 9:02 AM Amrit Gupta <gu...@gmail.com> wrote:
> Bumping this
>
> On Fri, 4 Nov 2022 at 3:26 PM, Amrit Gupta <gu...@gmail.com> wrote:
>
> > Hi,
> >
> > There is a producer metric
> > <
> https://github.com/apache/kafka/blob/14c36c7539cf5a7a54c2c7e30e115b3f4c4a9402/clients/src/main/java/org/apache/kafka/clients/producer/internals/SenderMetricsRegistry.java#L92
> >
> > ( request-latency-avg ), this is not updated in producer metrics
> > documentation in confluent.
> >
> > Also can I please get the detail of this metric that records the time
> > spent from which point to which point exactly?
> >
> > Does it include the time it resides in the producer buffer? or it is from
> > the point it is removed from the producer buffer.
> >
> > I mean does this include the time from application calling send() and the
> > record is sent to the broker?
> >
> > Thanks,
> > Tushar
> >
> >
>
Re: Regarding Producer Metric - "request-latency-avg"
Posted by Amrit Gupta <gu...@gmail.com>.
Bumping this
On Fri, 4 Nov 2022 at 3:26 PM, Amrit Gupta <gu...@gmail.com> wrote:
> Hi,
>
> There is a producer metric
> <https://github.com/apache/kafka/blob/14c36c7539cf5a7a54c2c7e30e115b3f4c4a9402/clients/src/main/java/org/apache/kafka/clients/producer/internals/SenderMetricsRegistry.java#L92>
> ( request-latency-avg ), this is not updated in producer metrics
> documentation in confluent.
>
> Also can I please get the detail of this metric that records the time
> spent from which point to which point exactly?
>
> Does it include the time it resides in the producer buffer? or it is from
> the point it is removed from the producer buffer.
>
> I mean does this include the time from application calling send() and the
> record is sent to the broker?
>
> Thanks,
> Tushar
>
>