You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Monika Garg <ga...@gmail.com> on 2013/10/21 14:42:54 UTC
Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Hi,
I am getting so many Mbeans from My JMX like kafka.server,kafka.network etc.
There are of different types and have a lot of attributes.
I am trying to get when the attribute value for any Mbean is getting
changed.
But no helpful docs I am getting for it on google.
Even the terms used like
kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
how/when the value of PurgatorySize get change.No clue for this.
Similar to this there are a lot more values like
when a topic is created,there is 554 values in kafka.network Mbean.
Please any one can share any doc that explains the Mbeans for Kafka-0.8.
It will be really helpful for me.
Thanks.
--
*Moni*
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Posted by Monika Garg <ga...@gmail.com>.
Thanks a lot Joel.
On Tue, Oct 22, 2013 at 11:27 PM, Joel Koshy <jj...@gmail.com> wrote:
> >
> (1)"kafka.network":type="RequestChannel",name="Processor-0-ResponseQueueSize",
> >
> "kafka.network":type="RequestChannel",name="Processor-1-ResponseQueueSize",
> >
> "kafka.network":type="RequestChannel",name="Processor-2-ResponseQueueSize"
> > Im not getting what these Processor in (1) are actually for?
>
> Kafka's request handling utilizes a number of processor "threads" and
> responses are added to the above queue before sending to the client.
> This diagram may help:
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals
>
>
> >
> (2)"kafka.server":type="FetcherLagMetrics",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-demo-0-ConsumerLag"
>
> This reports how far behind that replica is. The format is
>
> ReplicaFetcherThread-<fetcherid>-<leader-broker-id>-<leaderbrokerinfo>-<topic>-<partition>-ConsumerLag
>
> (Fetcher id is typically zero (since the default number of replica
> fetchers is one).)
>
> >
> >
> (3)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-RequestsPerSec"
> >
> (4)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-2-host_moni-192-168-145-176-port_9092-BytesPerSec"
> > The Mbeans (2),(3) and (4) are getting generated on my 3rd broker -
> > host_moni-192.168.145.67,kafka running on same port-9092.Any reason for
> > these.
>
> Requests per sec/byte rate for replica fetchers, but for all fetches
> to the indicated leader brokers (1 and 2).
>
>
> > I am getting some more Mbeans which are not mentioned here-
> > http://kafka.apache.org/documentation.html#monitoring
> >
> Kafka exposes a *lot* of mbeans - not all of them are listed on that
> page that you are referring to. I think the goal of the table was to
> provide a minimal set of critical metrics that people may want to set
> up monitoring/alerting for.
>
> Thanks,
>
> Joel
>
> >
> > On Tue, Oct 22, 2013 at 11:01 PM, Monika Garg <ga...@gmail.com>
> wrote:
> >
> >> Thanks Jun.
> >>
> >>
> >>
> >>
> >> On Tue, Oct 22, 2013 at 8:51 PM, Jun Rao <ju...@gmail.com> wrote:
> >>
> >>> When a broker goes down, we have an optimization in the controller that
> >>> proactively shrinks ISR. The ISR shrink rate only measures the rate
> that
> >>> the leader shrinks ISR and doesn't reflect the shrinking by the
> >>> controller.
> >>> In general, having ISR shrink rate stay at 0 is a good thing since it
> >>> means
> >>> that the followers can always keep up.
> >>>
> >>> Thanks,
> >>>
> >>> Jun
> >>>
> >>>
> >>> On Tue, Oct 22, 2013 at 12:55 AM, Monika Garg <ga...@gmail.com>
> >>> wrote:
> >>>
> >>> > Thanks a lot Jun.
> >>> >
> >>> > Now im getting the values in ISR expansion rate,whenever broker goes
> >>> down
> >>> > and then up.
> >>> >
> >>> > But still ISR shrink rate is always having value 0.
> >>> >
> >>> > what I have done is:
> >>> >
> >>> > I made 3 node cluster.Than created 5 partitions with replica=3.
> >>> > partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and
> >>> replica =
> >>> > 1,2,3
> >>> > broker -3 is leader for 2 partitions.
> >>> >
> >>> > then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3.
> >>> > In this case I waited for some time to get the ISR shrink value to
> get
> >>> > updated in my jconsole,but value remained 0 for all attributes of ISR
> >>> > shrink rate.
> >>> >
> >>> > Then I started broker(say 3) that i stopped earlier,what I saw is:
> >>> >
> >>> > ISR expansion rate has the values as:
> >>> >
> >>> > count -2 (on both other brokers 1,2)
> >>> > oneminuterate - 0.012 etc.
> >>> >
> >>> > then I again stopped broker-1,still isr shrink rate not getting any
> >>> > value.Can you please check?
> >>> >
> >>> > One more thing if you can please have a look:
> >>> > How the other attributes like FifteenMinuteRate,OneMinuteRate etc are
> >>> > continuosly changing for any Mbean.
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > On Tue, Oct 22, 2013 at 9:00 AM, Jun Rao <ju...@gmail.com> wrote:
> >>> >
> >>> > > Updated the description in
> >>> > > http://kafka.apache.org/documentation.html#monitoring. Does that
> help
> >>> > make
> >>> > > things clearer?
> >>> > >
> >>> > > Thanks,
> >>> > >
> >>> > > Jun
> >>> > >
> >>> > >
> >>> > > On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg <gargmoni03@gmail.com
> >
> >>> > wrote:
> >>> > >
> >>> > > > Thanks for replying Neha.It helped me a lot in getting the things
> >>> more
> >>> > > > clear.
> >>> > > > I just have some doubts for the value of below Mbeans,please
> have a
> >>> > look:
> >>> > > > (1)ISR expansion
> >>> > > >
> rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> >>> > > > ,non-zero only during broker startup.
> >>> > > > Does the property (1) mean that,it will be become zero when any
> >>> broker
> >>> > > (in
> >>> > > > a 2 node cluster)is started,and will be non-zero again if both
> the
> >>> > > brokers
> >>> > > > are stopped.
> >>> > > > (2)ISR shrink
> >>> > > >
> rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> >>> > > > , 0
> >>> > > > Will this property(2) be always be zero?
> >>> > > > Thanks.
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > > On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede <
> >>> > neha.narkhede@gmail.com
> >>> > > > >wrote:
> >>> > > >
> >>> > > > > You may find this useful -
> >>> > > > > http://kafka.apache.org/documentation.html#monitoring
> >>> > > > >
> >>> > > > > Let us know how we can improve the documentation further.
> >>> > > > >
> >>> > > > > Thanks,
> >>> > > > > Neha
> >>> > > > >
> >>> > > > >
> >>> > > > > On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg <
> >>> gargmoni03@gmail.com>
> >>> > > > wrote:
> >>> > > > >
> >>> > > > > > Hi,
> >>> > > > > >
> >>> > > > > > I am getting so many Mbeans from My JMX like
> >>> > > kafka.server,kafka.network
> >>> > > > > > etc.
> >>> > > > > >
> >>> > > > > > There are of different types and have a lot of attributes.
> >>> > > > > >
> >>> > > > > > I am trying to get when the attribute value for any Mbean is
> >>> > getting
> >>> > > > > > changed.
> >>> > > > > >
> >>> > > > > > But no helpful docs I am getting for it on google.
> >>> > > > > >
> >>> > > > > > Even the terms used like
> >>> > > > > >
> >>> > kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
> >>> > > > > > how/when the value of PurgatorySize get change.No clue for
> >>> this.
> >>> > > > > > Similar to this there are a lot more values like
> >>> > > > > >
> >>> > > > > > when a topic is created,there is 554 values in kafka.network
> >>> Mbean.
> >>> > > > > >
> >>> > > > > > Please any one can share any doc that explains the Mbeans for
> >>> > > > Kafka-0.8.
> >>> > > > > >
> >>> > > > > > It will be really helpful for me.
> >>> > > > > > Thanks.
> >>> > > > > >
> >>> > > > > > --
> >>> > > > > > *Moni*
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > > --
> >>> > > > *Moniii*
> >>> > > >
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > *Moniii*
> >>> >
> >>>
> >>
> >>
> >>
> >> --
> >> *Moniii*
> >>
> >
> >
> >
> > --
> > *Moniii*
>
--
*Moniii*
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Posted by Joel Koshy <jj...@gmail.com>.
> (1)"kafka.network":type="RequestChannel",name="Processor-0-ResponseQueueSize",
> "kafka.network":type="RequestChannel",name="Processor-1-ResponseQueueSize",
> "kafka.network":type="RequestChannel",name="Processor-2-ResponseQueueSize"
> Im not getting what these Processor in (1) are actually for?
Kafka's request handling utilizes a number of processor "threads" and
responses are added to the above queue before sending to the client.
This diagram may help:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals
> (2)"kafka.server":type="FetcherLagMetrics",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-demo-0-ConsumerLag"
This reports how far behind that replica is. The format is
ReplicaFetcherThread-<fetcherid>-<leader-broker-id>-<leaderbrokerinfo>-<topic>-<partition>-ConsumerLag
(Fetcher id is typically zero (since the default number of replica
fetchers is one).)
>
> (3)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-RequestsPerSec"
> (4)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-2-host_moni-192-168-145-176-port_9092-BytesPerSec"
> The Mbeans (2),(3) and (4) are getting generated on my 3rd broker -
> host_moni-192.168.145.67,kafka running on same port-9092.Any reason for
> these.
Requests per sec/byte rate for replica fetchers, but for all fetches
to the indicated leader brokers (1 and 2).
> I am getting some more Mbeans which are not mentioned here-
> http://kafka.apache.org/documentation.html#monitoring
>
Kafka exposes a *lot* of mbeans - not all of them are listed on that
page that you are referring to. I think the goal of the table was to
provide a minimal set of critical metrics that people may want to set
up monitoring/alerting for.
Thanks,
Joel
>
> On Tue, Oct 22, 2013 at 11:01 PM, Monika Garg <ga...@gmail.com> wrote:
>
>> Thanks Jun.
>>
>>
>>
>>
>> On Tue, Oct 22, 2013 at 8:51 PM, Jun Rao <ju...@gmail.com> wrote:
>>
>>> When a broker goes down, we have an optimization in the controller that
>>> proactively shrinks ISR. The ISR shrink rate only measures the rate that
>>> the leader shrinks ISR and doesn't reflect the shrinking by the
>>> controller.
>>> In general, having ISR shrink rate stay at 0 is a good thing since it
>>> means
>>> that the followers can always keep up.
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>>
>>> On Tue, Oct 22, 2013 at 12:55 AM, Monika Garg <ga...@gmail.com>
>>> wrote:
>>>
>>> > Thanks a lot Jun.
>>> >
>>> > Now im getting the values in ISR expansion rate,whenever broker goes
>>> down
>>> > and then up.
>>> >
>>> > But still ISR shrink rate is always having value 0.
>>> >
>>> > what I have done is:
>>> >
>>> > I made 3 node cluster.Than created 5 partitions with replica=3.
>>> > partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and
>>> replica =
>>> > 1,2,3
>>> > broker -3 is leader for 2 partitions.
>>> >
>>> > then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3.
>>> > In this case I waited for some time to get the ISR shrink value to get
>>> > updated in my jconsole,but value remained 0 for all attributes of ISR
>>> > shrink rate.
>>> >
>>> > Then I started broker(say 3) that i stopped earlier,what I saw is:
>>> >
>>> > ISR expansion rate has the values as:
>>> >
>>> > count -2 (on both other brokers 1,2)
>>> > oneminuterate - 0.012 etc.
>>> >
>>> > then I again stopped broker-1,still isr shrink rate not getting any
>>> > value.Can you please check?
>>> >
>>> > One more thing if you can please have a look:
>>> > How the other attributes like FifteenMinuteRate,OneMinuteRate etc are
>>> > continuosly changing for any Mbean.
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Tue, Oct 22, 2013 at 9:00 AM, Jun Rao <ju...@gmail.com> wrote:
>>> >
>>> > > Updated the description in
>>> > > http://kafka.apache.org/documentation.html#monitoring. Does that help
>>> > make
>>> > > things clearer?
>>> > >
>>> > > Thanks,
>>> > >
>>> > > Jun
>>> > >
>>> > >
>>> > > On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg <ga...@gmail.com>
>>> > wrote:
>>> > >
>>> > > > Thanks for replying Neha.It helped me a lot in getting the things
>>> more
>>> > > > clear.
>>> > > > I just have some doubts for the value of below Mbeans,please have a
>>> > look:
>>> > > > (1)ISR expansion
>>> > > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
>>> > > > ,non-zero only during broker startup.
>>> > > > Does the property (1) mean that,it will be become zero when any
>>> broker
>>> > > (in
>>> > > > a 2 node cluster)is started,and will be non-zero again if both the
>>> > > brokers
>>> > > > are stopped.
>>> > > > (2)ISR shrink
>>> > > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
>>> > > > , 0
>>> > > > Will this property(2) be always be zero?
>>> > > > Thanks.
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede <
>>> > neha.narkhede@gmail.com
>>> > > > >wrote:
>>> > > >
>>> > > > > You may find this useful -
>>> > > > > http://kafka.apache.org/documentation.html#monitoring
>>> > > > >
>>> > > > > Let us know how we can improve the documentation further.
>>> > > > >
>>> > > > > Thanks,
>>> > > > > Neha
>>> > > > >
>>> > > > >
>>> > > > > On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg <
>>> gargmoni03@gmail.com>
>>> > > > wrote:
>>> > > > >
>>> > > > > > Hi,
>>> > > > > >
>>> > > > > > I am getting so many Mbeans from My JMX like
>>> > > kafka.server,kafka.network
>>> > > > > > etc.
>>> > > > > >
>>> > > > > > There are of different types and have a lot of attributes.
>>> > > > > >
>>> > > > > > I am trying to get when the attribute value for any Mbean is
>>> > getting
>>> > > > > > changed.
>>> > > > > >
>>> > > > > > But no helpful docs I am getting for it on google.
>>> > > > > >
>>> > > > > > Even the terms used like
>>> > > > > >
>>> > kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
>>> > > > > > how/when the value of PurgatorySize get change.No clue for
>>> this.
>>> > > > > > Similar to this there are a lot more values like
>>> > > > > >
>>> > > > > > when a topic is created,there is 554 values in kafka.network
>>> Mbean.
>>> > > > > >
>>> > > > > > Please any one can share any doc that explains the Mbeans for
>>> > > > Kafka-0.8.
>>> > > > > >
>>> > > > > > It will be really helpful for me.
>>> > > > > > Thanks.
>>> > > > > >
>>> > > > > > --
>>> > > > > > *Moni*
>>> > > > > >
>>> > > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > --
>>> > > > *Moniii*
>>> > > >
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > *Moniii*
>>> >
>>>
>>
>>
>>
>> --
>> *Moniii*
>>
>
>
>
> --
> *Moniii*
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Posted by Monika Garg <ga...@gmail.com>.
Hi,
I am getting some more Mbeans which are not mentioned here-
http://kafka.apache.org/documentation.html#monitoring
These are:
(1)"kafka.network":type="RequestChannel",name="Processor-0-ResponseQueueSize",
"kafka.network":type="RequestChannel",name="Processor-1-ResponseQueueSize",
"kafka.network":type="RequestChannel",name="Processor-2-ResponseQueueSize"
Im not getting what these Processor in (1) are actually for?
(2)"kafka.server":type="FetcherLagMetrics",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-demo-0-ConsumerLag"
(3)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-RequestsPerSec"
(4)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-2-host_moni-192-168-145-176-port_9092-BytesPerSec"
The Mbeans (2),(3) and (4) are getting generated on my 3rd broker -
host_moni-192.168.145.67,kafka running on same port-9092.Any reason for
these.
Any help is really appreciable.
On Tue, Oct 22, 2013 at 11:01 PM, Monika Garg <ga...@gmail.com> wrote:
> Thanks Jun.
>
>
>
>
> On Tue, Oct 22, 2013 at 8:51 PM, Jun Rao <ju...@gmail.com> wrote:
>
>> When a broker goes down, we have an optimization in the controller that
>> proactively shrinks ISR. The ISR shrink rate only measures the rate that
>> the leader shrinks ISR and doesn't reflect the shrinking by the
>> controller.
>> In general, having ISR shrink rate stay at 0 is a good thing since it
>> means
>> that the followers can always keep up.
>>
>> Thanks,
>>
>> Jun
>>
>>
>> On Tue, Oct 22, 2013 at 12:55 AM, Monika Garg <ga...@gmail.com>
>> wrote:
>>
>> > Thanks a lot Jun.
>> >
>> > Now im getting the values in ISR expansion rate,whenever broker goes
>> down
>> > and then up.
>> >
>> > But still ISR shrink rate is always having value 0.
>> >
>> > what I have done is:
>> >
>> > I made 3 node cluster.Than created 5 partitions with replica=3.
>> > partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and
>> replica =
>> > 1,2,3
>> > broker -3 is leader for 2 partitions.
>> >
>> > then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3.
>> > In this case I waited for some time to get the ISR shrink value to get
>> > updated in my jconsole,but value remained 0 for all attributes of ISR
>> > shrink rate.
>> >
>> > Then I started broker(say 3) that i stopped earlier,what I saw is:
>> >
>> > ISR expansion rate has the values as:
>> >
>> > count -2 (on both other brokers 1,2)
>> > oneminuterate - 0.012 etc.
>> >
>> > then I again stopped broker-1,still isr shrink rate not getting any
>> > value.Can you please check?
>> >
>> > One more thing if you can please have a look:
>> > How the other attributes like FifteenMinuteRate,OneMinuteRate etc are
>> > continuosly changing for any Mbean.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Tue, Oct 22, 2013 at 9:00 AM, Jun Rao <ju...@gmail.com> wrote:
>> >
>> > > Updated the description in
>> > > http://kafka.apache.org/documentation.html#monitoring. Does that help
>> > make
>> > > things clearer?
>> > >
>> > > Thanks,
>> > >
>> > > Jun
>> > >
>> > >
>> > > On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg <ga...@gmail.com>
>> > wrote:
>> > >
>> > > > Thanks for replying Neha.It helped me a lot in getting the things
>> more
>> > > > clear.
>> > > > I just have some doubts for the value of below Mbeans,please have a
>> > look:
>> > > > (1)ISR expansion
>> > > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
>> > > > ,non-zero only during broker startup.
>> > > > Does the property (1) mean that,it will be become zero when any
>> broker
>> > > (in
>> > > > a 2 node cluster)is started,and will be non-zero again if both the
>> > > brokers
>> > > > are stopped.
>> > > > (2)ISR shrink
>> > > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
>> > > > , 0
>> > > > Will this property(2) be always be zero?
>> > > > Thanks.
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede <
>> > neha.narkhede@gmail.com
>> > > > >wrote:
>> > > >
>> > > > > You may find this useful -
>> > > > > http://kafka.apache.org/documentation.html#monitoring
>> > > > >
>> > > > > Let us know how we can improve the documentation further.
>> > > > >
>> > > > > Thanks,
>> > > > > Neha
>> > > > >
>> > > > >
>> > > > > On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg <
>> gargmoni03@gmail.com>
>> > > > wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > I am getting so many Mbeans from My JMX like
>> > > kafka.server,kafka.network
>> > > > > > etc.
>> > > > > >
>> > > > > > There are of different types and have a lot of attributes.
>> > > > > >
>> > > > > > I am trying to get when the attribute value for any Mbean is
>> > getting
>> > > > > > changed.
>> > > > > >
>> > > > > > But no helpful docs I am getting for it on google.
>> > > > > >
>> > > > > > Even the terms used like
>> > > > > >
>> > kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
>> > > > > > how/when the value of PurgatorySize get change.No clue for
>> this.
>> > > > > > Similar to this there are a lot more values like
>> > > > > >
>> > > > > > when a topic is created,there is 554 values in kafka.network
>> Mbean.
>> > > > > >
>> > > > > > Please any one can share any doc that explains the Mbeans for
>> > > > Kafka-0.8.
>> > > > > >
>> > > > > > It will be really helpful for me.
>> > > > > > Thanks.
>> > > > > >
>> > > > > > --
>> > > > > > *Moni*
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > *Moniii*
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > *Moniii*
>> >
>>
>
>
>
> --
> *Moniii*
>
--
*Moniii*
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Posted by Monika Garg <ga...@gmail.com>.
Thanks Jun.
On Tue, Oct 22, 2013 at 8:51 PM, Jun Rao <ju...@gmail.com> wrote:
> When a broker goes down, we have an optimization in the controller that
> proactively shrinks ISR. The ISR shrink rate only measures the rate that
> the leader shrinks ISR and doesn't reflect the shrinking by the controller.
> In general, having ISR shrink rate stay at 0 is a good thing since it means
> that the followers can always keep up.
>
> Thanks,
>
> Jun
>
>
> On Tue, Oct 22, 2013 at 12:55 AM, Monika Garg <ga...@gmail.com>
> wrote:
>
> > Thanks a lot Jun.
> >
> > Now im getting the values in ISR expansion rate,whenever broker goes down
> > and then up.
> >
> > But still ISR shrink rate is always having value 0.
> >
> > what I have done is:
> >
> > I made 3 node cluster.Than created 5 partitions with replica=3.
> > partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and replica
> =
> > 1,2,3
> > broker -3 is leader for 2 partitions.
> >
> > then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3.
> > In this case I waited for some time to get the ISR shrink value to get
> > updated in my jconsole,but value remained 0 for all attributes of ISR
> > shrink rate.
> >
> > Then I started broker(say 3) that i stopped earlier,what I saw is:
> >
> > ISR expansion rate has the values as:
> >
> > count -2 (on both other brokers 1,2)
> > oneminuterate - 0.012 etc.
> >
> > then I again stopped broker-1,still isr shrink rate not getting any
> > value.Can you please check?
> >
> > One more thing if you can please have a look:
> > How the other attributes like FifteenMinuteRate,OneMinuteRate etc are
> > continuosly changing for any Mbean.
> >
> >
> >
> >
> >
> >
> >
> > On Tue, Oct 22, 2013 at 9:00 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > Updated the description in
> > > http://kafka.apache.org/documentation.html#monitoring. Does that help
> > make
> > > things clearer?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > >
> > > On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg <ga...@gmail.com>
> > wrote:
> > >
> > > > Thanks for replying Neha.It helped me a lot in getting the things
> more
> > > > clear.
> > > > I just have some doubts for the value of below Mbeans,please have a
> > look:
> > > > (1)ISR expansion
> > > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> > > > ,non-zero only during broker startup.
> > > > Does the property (1) mean that,it will be become zero when any
> broker
> > > (in
> > > > a 2 node cluster)is started,and will be non-zero again if both the
> > > brokers
> > > > are stopped.
> > > > (2)ISR shrink
> > > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> > > > , 0
> > > > Will this property(2) be always be zero?
> > > > Thanks.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede <
> > neha.narkhede@gmail.com
> > > > >wrote:
> > > >
> > > > > You may find this useful -
> > > > > http://kafka.apache.org/documentation.html#monitoring
> > > > >
> > > > > Let us know how we can improve the documentation further.
> > > > >
> > > > > Thanks,
> > > > > Neha
> > > > >
> > > > >
> > > > > On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg <gargmoni03@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I am getting so many Mbeans from My JMX like
> > > kafka.server,kafka.network
> > > > > > etc.
> > > > > >
> > > > > > There are of different types and have a lot of attributes.
> > > > > >
> > > > > > I am trying to get when the attribute value for any Mbean is
> > getting
> > > > > > changed.
> > > > > >
> > > > > > But no helpful docs I am getting for it on google.
> > > > > >
> > > > > > Even the terms used like
> > > > > >
> > kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
> > > > > > how/when the value of PurgatorySize get change.No clue for this.
> > > > > > Similar to this there are a lot more values like
> > > > > >
> > > > > > when a topic is created,there is 554 values in kafka.network
> Mbean.
> > > > > >
> > > > > > Please any one can share any doc that explains the Mbeans for
> > > > Kafka-0.8.
> > > > > >
> > > > > > It will be really helpful for me.
> > > > > > Thanks.
> > > > > >
> > > > > > --
> > > > > > *Moni*
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > *Moniii*
> > > >
> > >
> >
> >
> >
> > --
> > *Moniii*
> >
>
--
*Moniii*
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Posted by Jun Rao <ju...@gmail.com>.
When a broker goes down, we have an optimization in the controller that
proactively shrinks ISR. The ISR shrink rate only measures the rate that
the leader shrinks ISR and doesn't reflect the shrinking by the controller.
In general, having ISR shrink rate stay at 0 is a good thing since it means
that the followers can always keep up.
Thanks,
Jun
On Tue, Oct 22, 2013 at 12:55 AM, Monika Garg <ga...@gmail.com> wrote:
> Thanks a lot Jun.
>
> Now im getting the values in ISR expansion rate,whenever broker goes down
> and then up.
>
> But still ISR shrink rate is always having value 0.
>
> what I have done is:
>
> I made 3 node cluster.Than created 5 partitions with replica=3.
> partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and replica =
> 1,2,3
> broker -3 is leader for 2 partitions.
>
> then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3.
> In this case I waited for some time to get the ISR shrink value to get
> updated in my jconsole,but value remained 0 for all attributes of ISR
> shrink rate.
>
> Then I started broker(say 3) that i stopped earlier,what I saw is:
>
> ISR expansion rate has the values as:
>
> count -2 (on both other brokers 1,2)
> oneminuterate - 0.012 etc.
>
> then I again stopped broker-1,still isr shrink rate not getting any
> value.Can you please check?
>
> One more thing if you can please have a look:
> How the other attributes like FifteenMinuteRate,OneMinuteRate etc are
> continuosly changing for any Mbean.
>
>
>
>
>
>
>
> On Tue, Oct 22, 2013 at 9:00 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > Updated the description in
> > http://kafka.apache.org/documentation.html#monitoring. Does that help
> make
> > things clearer?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg <ga...@gmail.com>
> wrote:
> >
> > > Thanks for replying Neha.It helped me a lot in getting the things more
> > > clear.
> > > I just have some doubts for the value of below Mbeans,please have a
> look:
> > > (1)ISR expansion
> > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> > > ,non-zero only during broker startup.
> > > Does the property (1) mean that,it will be become zero when any broker
> > (in
> > > a 2 node cluster)is started,and will be non-zero again if both the
> > brokers
> > > are stopped.
> > > (2)ISR shrink
> > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> > > , 0
> > > Will this property(2) be always be zero?
> > > Thanks.
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede <
> neha.narkhede@gmail.com
> > > >wrote:
> > >
> > > > You may find this useful -
> > > > http://kafka.apache.org/documentation.html#monitoring
> > > >
> > > > Let us know how we can improve the documentation further.
> > > >
> > > > Thanks,
> > > > Neha
> > > >
> > > >
> > > > On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg <ga...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I am getting so many Mbeans from My JMX like
> > kafka.server,kafka.network
> > > > > etc.
> > > > >
> > > > > There are of different types and have a lot of attributes.
> > > > >
> > > > > I am trying to get when the attribute value for any Mbean is
> getting
> > > > > changed.
> > > > >
> > > > > But no helpful docs I am getting for it on google.
> > > > >
> > > > > Even the terms used like
> > > > >
> kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
> > > > > how/when the value of PurgatorySize get change.No clue for this.
> > > > > Similar to this there are a lot more values like
> > > > >
> > > > > when a topic is created,there is 554 values in kafka.network Mbean.
> > > > >
> > > > > Please any one can share any doc that explains the Mbeans for
> > > Kafka-0.8.
> > > > >
> > > > > It will be really helpful for me.
> > > > > Thanks.
> > > > >
> > > > > --
> > > > > *Moni*
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > *Moniii*
> > >
> >
>
>
>
> --
> *Moniii*
>
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Posted by Monika Garg <ga...@gmail.com>.
Thanks a lot Jun.
Now im getting the values in ISR expansion rate,whenever broker goes down
and then up.
But still ISR shrink rate is always having value 0.
what I have done is:
I made 3 node cluster.Than created 5 partitions with replica=3.
partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and replica =
1,2,3
broker -3 is leader for 2 partitions.
then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3.
In this case I waited for some time to get the ISR shrink value to get
updated in my jconsole,but value remained 0 for all attributes of ISR
shrink rate.
Then I started broker(say 3) that i stopped earlier,what I saw is:
ISR expansion rate has the values as:
count -2 (on both other brokers 1,2)
oneminuterate - 0.012 etc.
then I again stopped broker-1,still isr shrink rate not getting any
value.Can you please check?
One more thing if you can please have a look:
How the other attributes like FifteenMinuteRate,
OneMinuteRate etc are continuosly changing for any Mbean.
One more thing I observed is-when two brokers(broker 1 and 3) were
stopped,broker-2 became the leader for all the partitions.After some time
when both brokers(broker 1 and 3) were started,they didn't become the
leader for any of the partition.only broker-2 remains the leader for all
the partitions of my cluster.This shouldn't happen as I am having 9
partitions for 2 topics and broker-2 is leader for all partitions,so for
any read/write broker-2 will be handling the producer/consumer
requests.what is no of partitions=100 for 10 topics.There will be time
delay,so other brokers should also be elected as leader after they are
started.
Thanks.
On Tue, Oct 22, 2013 at 1:25 PM, Monika Garg <ga...@gmail.com> wrote:
> Thanks a lot Jun.
>
> Now im getting the values in ISR expansion rate,whenever broker goes down
> and then up.
>
> But still ISR shrink rate is always having value 0.
>
> what I have done is:
>
> I made 3 node cluster.Than created 5 partitions with replica=3.
> partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and replica =
> 1,2,3
> broker -3 is leader for 2 partitions.
>
> then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3.
> In this case I waited for some time to get the ISR shrink value to get
> updated in my jconsole,but value remained 0 for all attributes of ISR
> shrink rate.
>
> Then I started broker(say 3) that i stopped earlier,what I saw is:
>
> ISR expansion rate has the values as:
>
> count -2 (on both other brokers 1,2)
> oneminuterate - 0.012 etc.
>
> then I again stopped broker-1,still isr shrink rate not getting any
> value.Can you please check?
>
> One more thing if you can please have a look:
> How the other attributes like FifteenMinuteRate,OneMinuteRate etc are
> continuosly changing for any Mbean.
>
>
>
>
>
>
>
> On Tue, Oct 22, 2013 at 9:00 AM, Jun Rao <ju...@gmail.com> wrote:
>
>> Updated the description in
>> http://kafka.apache.org/documentation.html#monitoring. Does that help
>> make
>> things clearer?
>>
>> Thanks,
>>
>> Jun
>>
>>
>> On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg <ga...@gmail.com>
>> wrote:
>>
>> > Thanks for replying Neha.It helped me a lot in getting the things more
>> > clear.
>> > I just have some doubts for the value of below Mbeans,please have a
>> look:
>> > (1)ISR expansion
>> > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
>> > ,non-zero only during broker startup.
>> > Does the property (1) mean that,it will be become zero when any broker
>> (in
>> > a 2 node cluster)is started,and will be non-zero again if both the
>> brokers
>> > are stopped.
>> > (2)ISR shrink
>> > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
>> > , 0
>> > Will this property(2) be always be zero?
>> > Thanks.
>> >
>> >
>> >
>> >
>> >
>> > On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede <neha.narkhede@gmail.com
>> > >wrote:
>> >
>> > > You may find this useful -
>> > > http://kafka.apache.org/documentation.html#monitoring
>> > >
>> > > Let us know how we can improve the documentation further.
>> > >
>> > > Thanks,
>> > > Neha
>> > >
>> > >
>> > > On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg <ga...@gmail.com>
>> > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > I am getting so many Mbeans from My JMX like
>> kafka.server,kafka.network
>> > > > etc.
>> > > >
>> > > > There are of different types and have a lot of attributes.
>> > > >
>> > > > I am trying to get when the attribute value for any Mbean is getting
>> > > > changed.
>> > > >
>> > > > But no helpful docs I am getting for it on google.
>> > > >
>> > > > Even the terms used like
>> > > > kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
>> > > > how/when the value of PurgatorySize get change.No clue for this.
>> > > > Similar to this there are a lot more values like
>> > > >
>> > > > when a topic is created,there is 554 values in kafka.network Mbean.
>> > > >
>> > > > Please any one can share any doc that explains the Mbeans for
>> > Kafka-0.8.
>> > > >
>> > > > It will be really helpful for me.
>> > > > Thanks.
>> > > >
>> > > > --
>> > > > *Moni*
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > *Moniii*
>> >
>>
>
>
>
> --
> *Moniii*
>
--
*Moniii*
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Posted by Monika Garg <ga...@gmail.com>.
Thanks a lot Jun.
Now im getting the values in ISR expansion rate,whenever broker goes down
and then up.
But still ISR shrink rate is always having value 0.
what I have done is:
I made 3 node cluster.Than created 5 partitions with replica=3.
partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and replica =
1,2,3
broker -3 is leader for 2 partitions.
then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3.
In this case I waited for some time to get the ISR shrink value to get
updated in my jconsole,but value remained 0 for all attributes of ISR
shrink rate.
Then I started broker(say 3) that i stopped earlier,what I saw is:
ISR expansion rate has the values as:
count -2 (on both other brokers 1,2)
oneminuterate - 0.012 etc.
then I again stopped broker-1,still isr shrink rate not getting any
value.Can you please check?
One more thing if you can please have a look:
How the other attributes like FifteenMinuteRate,OneMinuteRate etc are
continuosly changing for any Mbean.
On Tue, Oct 22, 2013 at 9:00 AM, Jun Rao <ju...@gmail.com> wrote:
> Updated the description in
> http://kafka.apache.org/documentation.html#monitoring. Does that help make
> things clearer?
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg <ga...@gmail.com> wrote:
>
> > Thanks for replying Neha.It helped me a lot in getting the things more
> > clear.
> > I just have some doubts for the value of below Mbeans,please have a look:
> > (1)ISR expansion
> > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> > ,non-zero only during broker startup.
> > Does the property (1) mean that,it will be become zero when any broker
> (in
> > a 2 node cluster)is started,and will be non-zero again if both the
> brokers
> > are stopped.
> > (2)ISR shrink
> > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> > , 0
> > Will this property(2) be always be zero?
> > Thanks.
> >
> >
> >
> >
> >
> > On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede <neha.narkhede@gmail.com
> > >wrote:
> >
> > > You may find this useful -
> > > http://kafka.apache.org/documentation.html#monitoring
> > >
> > > Let us know how we can improve the documentation further.
> > >
> > > Thanks,
> > > Neha
> > >
> > >
> > > On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg <ga...@gmail.com>
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > I am getting so many Mbeans from My JMX like
> kafka.server,kafka.network
> > > > etc.
> > > >
> > > > There are of different types and have a lot of attributes.
> > > >
> > > > I am trying to get when the attribute value for any Mbean is getting
> > > > changed.
> > > >
> > > > But no helpful docs I am getting for it on google.
> > > >
> > > > Even the terms used like
> > > > kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
> > > > how/when the value of PurgatorySize get change.No clue for this.
> > > > Similar to this there are a lot more values like
> > > >
> > > > when a topic is created,there is 554 values in kafka.network Mbean.
> > > >
> > > > Please any one can share any doc that explains the Mbeans for
> > Kafka-0.8.
> > > >
> > > > It will be really helpful for me.
> > > > Thanks.
> > > >
> > > > --
> > > > *Moni*
> > > >
> > >
> >
> >
> >
> > --
> > *Moniii*
> >
>
--
*Moniii*
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Posted by Jun Rao <ju...@gmail.com>.
Updated the description in
http://kafka.apache.org/documentation.html#monitoring. Does that help make
things clearer?
Thanks,
Jun
On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg <ga...@gmail.com> wrote:
> Thanks for replying Neha.It helped me a lot in getting the things more
> clear.
> I just have some doubts for the value of below Mbeans,please have a look:
> (1)ISR expansion
> rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> ,non-zero only during broker startup.
> Does the property (1) mean that,it will be become zero when any broker (in
> a 2 node cluster)is started,and will be non-zero again if both the brokers
> are stopped.
> (2)ISR shrink
> rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> , 0
> Will this property(2) be always be zero?
> Thanks.
>
>
>
>
>
> On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede <neha.narkhede@gmail.com
> >wrote:
>
> > You may find this useful -
> > http://kafka.apache.org/documentation.html#monitoring
> >
> > Let us know how we can improve the documentation further.
> >
> > Thanks,
> > Neha
> >
> >
> > On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg <ga...@gmail.com>
> wrote:
> >
> > > Hi,
> > >
> > > I am getting so many Mbeans from My JMX like kafka.server,kafka.network
> > > etc.
> > >
> > > There are of different types and have a lot of attributes.
> > >
> > > I am trying to get when the attribute value for any Mbean is getting
> > > changed.
> > >
> > > But no helpful docs I am getting for it on google.
> > >
> > > Even the terms used like
> > > kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
> > > how/when the value of PurgatorySize get change.No clue for this.
> > > Similar to this there are a lot more values like
> > >
> > > when a topic is created,there is 554 values in kafka.network Mbean.
> > >
> > > Please any one can share any doc that explains the Mbeans for
> Kafka-0.8.
> > >
> > > It will be really helpful for me.
> > > Thanks.
> > >
> > > --
> > > *Moni*
> > >
> >
>
>
>
> --
> *Moniii*
>
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Posted by Monika Garg <ga...@gmail.com>.
Thanks for replying Neha.It helped me a lot in getting the things more
clear.
I just have some doubts for the value of below Mbeans,please have a look:
(1)ISR expansion
rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
,non-zero only during broker startup.
Does the property (1) mean that,it will be become zero when any broker (in
a 2 node cluster)is started,and will be non-zero again if both the brokers
are stopped.
(2)ISR shrink rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
, 0
Will this property(2) be always be zero?
Thanks.
On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede <ne...@gmail.com>wrote:
> You may find this useful -
> http://kafka.apache.org/documentation.html#monitoring
>
> Let us know how we can improve the documentation further.
>
> Thanks,
> Neha
>
>
> On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg <ga...@gmail.com> wrote:
>
> > Hi,
> >
> > I am getting so many Mbeans from My JMX like kafka.server,kafka.network
> > etc.
> >
> > There are of different types and have a lot of attributes.
> >
> > I am trying to get when the attribute value for any Mbean is getting
> > changed.
> >
> > But no helpful docs I am getting for it on google.
> >
> > Even the terms used like
> > kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
> > how/when the value of PurgatorySize get change.No clue for this.
> > Similar to this there are a lot more values like
> >
> > when a topic is created,there is 554 values in kafka.network Mbean.
> >
> > Please any one can share any doc that explains the Mbeans for Kafka-0.8.
> >
> > It will be really helpful for me.
> > Thanks.
> >
> > --
> > *Moni*
> >
>
--
*Moniii*
Re: Any detail for Kafka Mbeans/yammer-Metrics obtained from JMX
Posted by Neha Narkhede <ne...@gmail.com>.
You may find this useful -
http://kafka.apache.org/documentation.html#monitoring
Let us know how we can improve the documentation further.
Thanks,
Neha
On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg <ga...@gmail.com> wrote:
> Hi,
>
> I am getting so many Mbeans from My JMX like kafka.server,kafka.network
> etc.
>
> There are of different types and have a lot of attributes.
>
> I am trying to get when the attribute value for any Mbean is getting
> changed.
>
> But no helpful docs I am getting for it on google.
>
> Even the terms used like
> kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here
> how/when the value of PurgatorySize get change.No clue for this.
> Similar to this there are a lot more values like
>
> when a topic is created,there is 554 values in kafka.network Mbean.
>
> Please any one can share any doc that explains the Mbeans for Kafka-0.8.
>
> It will be really helpful for me.
> Thanks.
>
> --
> *Moni*
>