You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Jun Rao <ju...@gmail.com> on 2014/04/01 06:16:42 UTC
Re: Kafka Queue Depth Metrics
We also have a metric in the consumer that measures the # of messages a
consumer is behind.
Thanks,
Jun
On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson <ar...@velos.io>wrote:
> Hi,
>
> Does anyone know what the JMX metric(s) are for determining the depth of
> the kafka queue? I basically want to know the amount of messages that are
> on the queue at a certain point in time. Please let me know if there is a
> way to find that out. Thanks.
>
> --
>
> Arnaud Lawson
> ,
>
> Systems Operations Engineer
> *VELOS*
> Accelerating Machine Learning.
>
> 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
> C: (240) 393 - 6703
> F: 646.349.4063
> E: arnaud.lawson@velos.io
> W: www.velos.io
>
Re: Kafka Queue Depth Metrics
Posted by Arnaud Lawson <ar...@velos.io>.
Ok thank you.
Arnaud-
On Tue, Apr 1, 2014 at 10:50 AM, Jun Rao <ju...@gmail.com> wrote:
> Search for MaxLag in http://kafka.apache.org/documentation.html#monitoring
> .
> Note that this is only available in 0.8.0 and above.
>
> Thanks,
>
> Jun
>
>
> On Tue, Apr 1, 2014 at 7:21 AM, Arnaud Lawson <arnaud.lawson@velos.io
> >wrote:
>
> > Thanks Jun,
> >
> > So on the consumer side is the metric (measuring the # of messages a
> > consumer is behind) called getOffsetLag?
> >
> > Reference:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Monitoring
> >
> >
> > Arnaud-
> >
> >
> > On Tue, Apr 1, 2014 at 12:16 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > We also have a metric in the consumer that measures the # of messages a
> > > consumer is behind.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson <arnaud.lawson@velos.io
> > > >wrote:
> > >
> > > > Hi,
> > > >
> > > > Does anyone know what the JMX metric(s) are for determining the depth
> > of
> > > > the kafka queue? I basically want to know the amount of messages that
> > are
> > > > on the queue at a certain point in time. Please let me know if there
> > is a
> > > > way to find that out. Thanks.
> > > >
> > > > --
> > > >
> > > > Arnaud Lawson
> > > > ,
> > > >
> > > > Systems Operations Engineer
> > > > *VELOS*
> > > > Accelerating Machine Learning.
> > > >
> > > > 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
> > > > C: (240) 393 - 6703
> > > > F: 646.349.4063
> > > > E: arnaud.lawson@velos.io
> > > > W: www.velos.io
> > > >
> > >
> >
> >
> >
> > --
> >
> > Arnaud Lawson
> > ,
> >
> > Systems Operations Engineer
> > *VELOS*
> > Accelerating Machine Learning.
> >
> > 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
> > C: (240) 393 - 6703
> > F: 646.349.4063
> > E: arnaud.lawson@velos.io
> > W: www.velos.io
> >
>
--
Arnaud Lawson
,
Systems Operations Engineer
*VELOS*
Accelerating Machine Learning.
440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
C: (240) 393 - 6703
F: 646.349.4063
E: arnaud.lawson@velos.io
W: www.velos.io
Re: Kafka Queue Depth Metrics
Posted by Jun Rao <ju...@gmail.com>.
Search for MaxLag in http://kafka.apache.org/documentation.html#monitoring.
Note that this is only available in 0.8.0 and above.
Thanks,
Jun
On Tue, Apr 1, 2014 at 7:21 AM, Arnaud Lawson <ar...@velos.io>wrote:
> Thanks Jun,
>
> So on the consumer side is the metric (measuring the # of messages a
> consumer is behind) called getOffsetLag?
>
> Reference:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Monitoring
>
>
> Arnaud-
>
>
> On Tue, Apr 1, 2014 at 12:16 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > We also have a metric in the consumer that measures the # of messages a
> > consumer is behind.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson <arnaud.lawson@velos.io
> > >wrote:
> >
> > > Hi,
> > >
> > > Does anyone know what the JMX metric(s) are for determining the depth
> of
> > > the kafka queue? I basically want to know the amount of messages that
> are
> > > on the queue at a certain point in time. Please let me know if there
> is a
> > > way to find that out. Thanks.
> > >
> > > --
> > >
> > > Arnaud Lawson
> > > ,
> > >
> > > Systems Operations Engineer
> > > *VELOS*
> > > Accelerating Machine Learning.
> > >
> > > 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
> > > C: (240) 393 - 6703
> > > F: 646.349.4063
> > > E: arnaud.lawson@velos.io
> > > W: www.velos.io
> > >
> >
>
>
>
> --
>
> Arnaud Lawson
> ,
>
> Systems Operations Engineer
> *VELOS*
> Accelerating Machine Learning.
>
> 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
> C: (240) 393 - 6703
> F: 646.349.4063
> E: arnaud.lawson@velos.io
> W: www.velos.io
>
Re: Kafka Queue Depth Metrics
Posted by Arnaud Lawson <ar...@velos.io>.
Thanks Jun,
So on the consumer side is the metric (measuring the # of messages a
consumer is behind) called getOffsetLag?
Reference:
https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Monitoring
Arnaud-
On Tue, Apr 1, 2014 at 12:16 AM, Jun Rao <ju...@gmail.com> wrote:
> We also have a metric in the consumer that measures the # of messages a
> consumer is behind.
>
> Thanks,
>
> Jun
>
>
> On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson <arnaud.lawson@velos.io
> >wrote:
>
> > Hi,
> >
> > Does anyone know what the JMX metric(s) are for determining the depth of
> > the kafka queue? I basically want to know the amount of messages that are
> > on the queue at a certain point in time. Please let me know if there is a
> > way to find that out. Thanks.
> >
> > --
> >
> > Arnaud Lawson
> > ,
> >
> > Systems Operations Engineer
> > *VELOS*
> > Accelerating Machine Learning.
> >
> > 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
> > C: (240) 393 - 6703
> > F: 646.349.4063
> > E: arnaud.lawson@velos.io
> > W: www.velos.io
> >
>
--
Arnaud Lawson
,
Systems Operations Engineer
*VELOS*
Accelerating Machine Learning.
440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
C: (240) 393 - 6703
F: 646.349.4063
E: arnaud.lawson@velos.io
W: www.velos.io