Re: Kafka metrics: Bytesout greater than Bytesin??

[Jun Rao <ju...@gmail.com>]

In 0.8, we have a MaxLag jmx metrics to monitor the consumer lag. In 0.7,
we only have a command line tool ConsumerOffsetChecker.

Thanks,

Jun


On Wed, Apr 30, 2014 at 7:17 AM, Arnaud Lawson <ar...@velos.io>wrote:

> Jun,
>
> Thanks for the pointer to the docs. It would be great if more detail like
> this discussion were included in the Wiki. The detail of each metric is not
> always clear.
>
> For our case (trying to monitor and alert on if the consumer is falling far
> behind the producer on a specific topic), what would you recommend?
>
> Should we compare BrokerAllTopicStat.[topic].getMessagesIn to
> ConsumerTopicStat.[topic].getMessagesPerTopic?
>
>
> On Wed, Apr 30, 2014 at 1:13 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > Let's say a consumer wants to fetch 1000 bytes and there are two messages
> > in the broker each of 600 bytes. The broker will give 1000 bytes to the
> > consumer. The consumer will only be able to use the first 600 bytes since
> > the remaining 400 bytes doesn't form a full message.
> >
> > 0.7 jmx are documented in
> > https://cwiki.apache.org/confluence/display/KAFKA/Operations
> >
> > thanks,
> >
> > Jun
> >
> >
> > On Tue, Apr 29, 2014 at 9:51 PM, Arnaud Lawson <arnaud.lawson@velos.io
> > >wrote:
> >
> > > Thank you Jun,
> > >
> > > What could a partial message be? A consumer fetching for messages on an
> > > empty queue and receiving a message representing that fact?
> > >
> > > As an example below i just ran a test using the producer & consumer
> > shells
> > > and JMXTerm. NumProduceRequests went from 13, 14 to 15 as i added
> values
> > > one by one to the queue for a test topic but then NumFetchRequests
> varies
> > > greatly. I am not sure why. I know there is only one consumer (the
> shell
> > > consumer) for this test kafka 0.7 instance and replication is not
> > available
> > > for this version. *Essentially** we would like to be able to measure
> the
> > > rate of consumption for some particular topic and make sure that this
> > rate
> > > is about the same as the rate of production for that topic. Which JMX
> > > metrics would be appropriate for this for Kafka 0.7?* Any suggestions
> > would
> > > be helpful. Thanks.
> > >
> > >
> > > #mbean = kafka:type=kafka.SocketServerStats:
> > > NumProduceRequests = 13; (Attribute exposed for management)
> > >
> > > $>get -i -b kafka:type=kafka.SocketServerStats NumFetchRequests
> > > #mbean = kafka:type=kafka.SocketServerStats:
> > > NumFetchRequests = 4763; (Attribute exposed for management)
> > >
> > > $>get -i -b kafka:type=kafka.SocketServerStats NumProduceRequests
> > > #mbean = kafka:type=kafka.SocketServerStats:
> > > NumProduceRequests = 14; (Attribute exposed for management)
> > >
> > > $>get -i -b kafka:type=kafka.SocketServerStats NumFetchRequests
> > > #mbean = kafka:type=kafka.SocketServerStats:
> > > NumFetchRequests = 4776; (Attribute exposed for management)
> > >
> > > $>get -i -b kafka:type=kafka.SocketServerStats NumProduceRequests
> > > #mbean = kafka:type=kafka.SocketServerStats:
> > > NumProduceRequests = 15; (Attribute exposed for management)
> > >
> > > $>get -i -b kafka:type=kafka.SocketServerStats NumFetchRequests
> > > #mbean = kafka:type=kafka.SocketServerStats:
> > > NumFetchRequests = 4904; (Attribute exposed for management)
> > >
> > >
> > >
> > >
> > > On Wed, Apr 30, 2014 at 12:18 AM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > > > Is the data consumed more than once from different consumers? If
> that's
> > > not
> > > > the case, byteout could be a bit larger than bytein since we could
> > > return a
> > > > partial message to the consumer. The partial message will be ignored
> in
> > > the
> > > > consumer though.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Tue, Apr 29, 2014 at 2:13 PM, Arnaud Lawson <
> arnaud.lawson@velos.io
> > > > >wrote:
> > > >
> > > > > We are using Kafka 0.7.1 and i believe replication isn't available
> > for
> > > > this
> > > > > version (correct me if i am wrong)...
> > > > >
> > > > > Basically we want to be able to measure the rate of consumption for
> > > some
> > > > > particular topic and make sure that this rate is about the same as
> > the
> > > > rate
> > > > > of production for that topic. Which JMX metrics could be
> appropriate
> > > for
> > > > > this - for Kafka 0.7? Thanks.
> > > > >
> > > > >
> > > > > On Tue, Apr 29, 2014 at 4:28 PM, Bae, Jae Hyeon <
> metacret@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Are you using 0.8? BytesOut will include the traffic for
> > replication.
> > > > If
> > > > > > you have no consumer and replication factor is 2, BytesOut should
> > be
> > > > > > exactly double of BytesIn.
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 29, 2014 at 1:26 PM, Arnaud Lawson <
> > > arnaud.lawson@velos.io
> > > > > > >wrote:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > After graphing the cumulative values of Bytesin and Bytesout
> for
> > > one
> > > > of
> > > > > > our
> > > > > > > Kafka topics, we've noticed that the Bytesout values are
> > sometimes
> > > > > > greater
> > > > > > > than the Bytesin values. We are not really sure how this would
> be
> > > > > > possible
> > > > > > > if the max that a consumer can consume would be what has been
> put
> > > on
> > > > > the
> > > > > > > queue for that topic. Any explanations? Thanks.
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > ​Arnaud Lawson​
> > > > > > > ,
> > > > > > > ​
> > > > > > > Systems Operations Engineer
> > > > > > > *VELOS*
> > > > > > > Accelerating Machine Learning.
> > > > > > >
> > > > > > > 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
> > > > > > > C: (240) 393 - 6703
> > > > > > > F: 646.349.4063
> > > > > > > E: arnaud.lawson@velos.io
> > > > > > > W: www.velos.io
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > ​Arnaud Lawson​
> > > > > ,
> > > > > ​
> > > > > Systems Operations Engineer
> > > > > *VELOS*
> > > > > Accelerating Machine Learning.
> > > > >
> > > > > 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
> > > > > C: (240) 393 - 6703
> > > > > F: 646.349.4063
> > > > > E: arnaud.lawson@velos.io
> > > > > W: www.velos.io
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > ​Arnaud Lawson​
> > > ,
> > > ​
> > > Systems Operations Engineer
> > > *VELOS*
> > > Accelerating Machine Learning.
> > >
> > > 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
> > > C: (240) 393 - 6703
> > > F: 646.349.4063
> > > E: arnaud.lawson@velos.io
> > > W: www.velos.io
> > >
> >
>
>
>
> --
>
> ​Arnaud Lawson​
> ,
> ​
> Systems Operations Engineer
> *VELOS*
> Accelerating Machine Learning.
>
> 440 9th AVE, 11TH FLOOR, NEW YORK, NY 10001
> C: (240) 393 - 6703
> F: 646.349.4063
> E: arnaud.lawson@velos.io
> W: www.velos.io
>