You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by David Yu <gu...@gmail.com> on 2016/08/03 03:30:05 UTC

Understand producer metrics

I'm having a hard time finding documentation explaining the set of producer
metrics exposed by Kafka. Can anyone explain the following?


   - batch-size-avg - Is this the number of msgs or number of bytes? Does
   this only make sense for async producers?
   - incoming-byte-rate/outgoing-byte-rate - Is this the number of bytes in
   and out of the producer? If so, should they be about the same?
   - record-queue-time-avg - Is this the avg ms a record is buffered?
   - record-send-rate - In bytes or # of msgs?
   - record-size-avg - In bytes?
   - record-per-request-avg - What is a request? Is that the same as a
   "batch"?
   - request-latency-avg - In ms? How is this measured?
   - request-size-avg - In bytes? This doesn't seem to match
   batch-size-avg, which makes me think request and batch are different.


Thanks,
David

Re: Understand producer metrics

Posted by David Yu <gu...@gmail.com>.
Thx!

On Fri, Aug 19, 2016 at 12:48 AM Manikumar Reddy <ma...@gmail.com>
wrote:

> This doc link may help:
>
> http://kafka.apache.org/documentation.html#new_producer_monitoring
>
> On Fri, Aug 19, 2016 at 2:36 AM, David Yu <gu...@gmail.com> wrote:
>
> > Kafka users,
> >
> > I want to resurface this post since it becomes crucial for our team to
> > understand our recent Samza throughput issues we are facing.
> >
> > Any help is appreciated.
> >
> > Thanks,
> > David
> >
> > On Tue, Aug 2, 2016 at 10:30 PM David Yu <gu...@gmail.com> wrote:
> >
> > > I'm having a hard time finding documentation explaining the set of
> > > producer metrics exposed by Kafka. Can anyone explain the following?
> > >
> > >
> > >    - batch-size-avg - Is this the number of msgs or number of bytes?
> Does
> > >    this only make sense for async producers?
> > >    - incoming-byte-rate/outgoing-byte-rate - Is this the number of
> bytes
> > >    in and out of the producer? If so, should they be about the same?
> > >    - record-queue-time-avg - Is this the avg ms a record is buffered?
> > >    - record-send-rate - In bytes or # of msgs?
> > >    - record-size-avg - In bytes?
> > >    - record-per-request-avg - What is a request? Is that the same as a
> > >    "batch"?
> > >    - request-latency-avg - In ms? How is this measured?
> > >    - request-size-avg - In bytes? This doesn't seem to match
> > >    batch-size-avg, which makes me think request and batch are
> different.
> > >
> > >
> > > Thanks,
> > > David
> > >
> >
>

Re: Understand producer metrics

Posted by Manikumar Reddy <ma...@gmail.com>.
This doc link may help:

http://kafka.apache.org/documentation.html#new_producer_monitoring

On Fri, Aug 19, 2016 at 2:36 AM, David Yu <gu...@gmail.com> wrote:

> Kafka users,
>
> I want to resurface this post since it becomes crucial for our team to
> understand our recent Samza throughput issues we are facing.
>
> Any help is appreciated.
>
> Thanks,
> David
>
> On Tue, Aug 2, 2016 at 10:30 PM David Yu <gu...@gmail.com> wrote:
>
> > I'm having a hard time finding documentation explaining the set of
> > producer metrics exposed by Kafka. Can anyone explain the following?
> >
> >
> >    - batch-size-avg - Is this the number of msgs or number of bytes? Does
> >    this only make sense for async producers?
> >    - incoming-byte-rate/outgoing-byte-rate - Is this the number of bytes
> >    in and out of the producer? If so, should they be about the same?
> >    - record-queue-time-avg - Is this the avg ms a record is buffered?
> >    - record-send-rate - In bytes or # of msgs?
> >    - record-size-avg - In bytes?
> >    - record-per-request-avg - What is a request? Is that the same as a
> >    "batch"?
> >    - request-latency-avg - In ms? How is this measured?
> >    - request-size-avg - In bytes? This doesn't seem to match
> >    batch-size-avg, which makes me think request and batch are different.
> >
> >
> > Thanks,
> > David
> >
>

Re: Understand producer metrics

Posted by David Yu <gu...@gmail.com>.
Kafka users,

I want to resurface this post since it becomes crucial for our team to
understand our recent Samza throughput issues we are facing.

Any help is appreciated.

Thanks,
David

On Tue, Aug 2, 2016 at 10:30 PM David Yu <gu...@gmail.com> wrote:

> I'm having a hard time finding documentation explaining the set of
> producer metrics exposed by Kafka. Can anyone explain the following?
>
>
>    - batch-size-avg - Is this the number of msgs or number of bytes? Does
>    this only make sense for async producers?
>    - incoming-byte-rate/outgoing-byte-rate - Is this the number of bytes
>    in and out of the producer? If so, should they be about the same?
>    - record-queue-time-avg - Is this the avg ms a record is buffered?
>    - record-send-rate - In bytes or # of msgs?
>    - record-size-avg - In bytes?
>    - record-per-request-avg - What is a request? Is that the same as a
>    "batch"?
>    - request-latency-avg - In ms? How is this measured?
>    - request-size-avg - In bytes? This doesn't seem to match
>    batch-size-avg, which makes me think request and batch are different.
>
>
> Thanks,
> David
>