You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Zakee <kz...@netzero.net> on 2015/02/19 20:54:03 UTC
NetworkProcessorAvgIdlePercent
Kafka documentation recommends <0.3 for above metric. I assume processor is
busier if this goes below 0.3 and obviously it being < 0.3 for long does
not seem to be a good sign.
What should be our criteria to raise an alert, I though it should be when
its value goes below 0.3. However, the value seems to be below 0.3 a lot of
the times, almost always if we take samples every five mins. What should be
the threshold to raise an alarm ?
What would be the impact of having this below 0.3 or even zero like most of
the times?
-Zakee
____________________________________________________________
How Old Men Tighten Skin
63 Year Old Man Shares DIY Skin Tightening Method You Can Do From Home
http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
Re: NetworkProcessorAvgIdlePercent
Posted by Zakee <kz...@netzero.net>.
Thanks, I have added them for monitoring.
-Zakee
> On Feb 27, 2015, at 9:21 AM, Jun Rao <ju...@confluent.io> wrote:
>
> Zakee,
>
> It would be useful to get the following.
>
> kafka.network:name=RequestQueueSize,type=RequestChannel
> kafka.network:name=RequestQueueTimeMs,request=Fetch,type=RequestMetrics
> kafka.network:name=RequestQueueTimeMs,request=Produce,type=RequestMetrics
>
> Thanks,
>
> Jun
>
>
> On Thu, Feb 26, 2015 at 2:17 PM, Zakee <kzakee1@netzero.net <ma...@netzero.net>> wrote:
>
>> Hi Jun,
>>
>> There are too many KPIs generated by kafka that I can’t monitor them all,
>> I have monitoring a filtered list that I can understand so far.
>>
>> Which keys are you talking of? Currently I am not monitoring any of below
>> keys but will add only those that are useful.
>>
>> kafka.network:name=RequestQueueSize,type=RequestChannel
>>
>> kafka.network:name=RequestQueueTimeMs,request=ConsumerMetadata,type=RequestMetrics
>>
>> kafka.network:name=RequestQueueTimeMs,request=ControlledShutdown,type=RequestMetrics
>> kafka.network:name=RequestQueueTimeMs,request=Fetch,type=RequestMetrics
>>
>> kafka.network:name=RequestQueueTimeMs,request=FetchConsumer,type=RequestMetrics
>>
>> kafka.network:name=RequestQueueTimeMs,request=FetchFollower,type=RequestMetrics
>> kafka.network:name=RequestQueueTimeMs,request=Heartbeat,type=RequestMetrics
>> kafka.network:name=RequestQueueTimeMs,request=JoinGroup,type=RequestMetrics
>>
>> kafka.network:name=RequestQueueTimeMs,request=LeaderAndIsr,type=RequestMetrics
>> kafka.network:name=RequestQueueTimeMs,request=Metadata,type=RequestMetrics
>>
>> kafka.network:name=RequestQueueTimeMs,request=OffsetCommit,type=RequestMetrics
>>
>> kafka.network:name=RequestQueueTimeMs,request=OffsetFetch,type=RequestMetrics
>> kafka.network:name=RequestQueueTimeMs,request=Offsets,type=RequestMetrics
>> kafka.network:name=RequestQueueTimeMs,request=Produce,type=RequestMetrics
>>
>> kafka.network:name=RequestQueueTimeMs,request=StopReplica,type=RequestMetrics
>>
>> kafka.network:name=RequestQueueTimeMs,request=UpdateMetadata,type=RequestMetrics
>>
>>
>> Thanks,
>> Zakee
>>
>>> On Feb 26, 2015, at 9:10 AM, Jun Rao <ju...@confluent.io> wrote:
>>>
>>> That may be enough. What's the RequestQueueSize and RequestQueueTimeMs?
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>> On Wed, Feb 25, 2015 at 10:24 PM, Zakee <kz...@netzero.net> wrote:
>>>
>>>> Well currently I have configured 14 thread for both io and network. Do
>> you
>>>> think we should consider more?
>>>>
>>>> Thanks
>>>> -Zakee
>>>>
>>>> On Wed, Feb 25, 2015 at 6:22 PM, Jun Rao <ju...@confluent.io> wrote:
>>>>
>>>>> Then you may want to consider increasing num.io.threads
>>>>> and num.network.threads.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Jun
>>>>>
>>>>> On Tue, Feb 24, 2015 at 7:48 PM, Zakee <kz...@netzero.net> wrote:
>>>>>
>>>>>> Similar pattern for that too. Mostly hovering below.
>>>>>>
>>>>>> -Zakee
>>>>>>
>>>>>> On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao <ju...@confluent.io> wrote:
>>>>>>
>>>>>>> What about RequestHandlerAvgIdlePercent?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Jun
>>>>>>>
>>>>>>> On Mon, Feb 23, 2015 at 8:47 PM, Zakee <kz...@netzero.net> wrote:
>>>>>>>
>>>>>>>> Hi Jun,
>>>>>>>>
>>>>>>>> With ~100G of data being pushed per hour across 35 topics
>>>>>>>> (replication-factor 3), the NetworkProcessorAvgIdlePercent is
>>>> mostly
>>>>>>>> showing below 0.5 sometimes when the producers send on a high rate.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> -Zakee
>>>>>>>>
>>>>>>>> On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <ju...@confluent.io>
>>>> wrote:
>>>>>>>>
>>>>>>>>> What kind of load do you have on the brokers? On an idle cluster
>>>>>> (just
>>>>>>>>> fetch requests from the follower replicas), I
>>>>>>>>> saw NetworkProcessorAvgIdlePercent at about 97%.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Jun
>>>>>>>>>
>>>>>>>>> On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kz...@netzero.net>
>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Jun,
>>>>>>>>>>
>>>>>>>>>> I am already using the latest release 0.8.2.1.
>>>>>>>>>>
>>>>>>>>>> -Zakee
>>>>>>>>>>
>>>>>>>>>> On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io>
>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Could you try the 0.8.2.1 release being voted on now? It
>>>> fixes
>>>>> a
>>>>>>> CPU
>>>>>>>>>> issue
>>>>>>>>>>> and should reduce the CPU load in network thread.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Jun
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kzakee1@netzero.net
>>>>>
>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Kafka documentation recommends <0.3 for above metric. I
>>>>> assume
>>>>>>>>>> processor
>>>>>>>>>>> is
>>>>>>>>>>>> busier if this goes below 0.3 and obviously it being < 0.3
>>>>> for
>>>>>>> long
>>>>>>>>>> does
>>>>>>>>>>>> not seem to be a good sign.
>>>>>>>>>>>>
>>>>>>>>>>>> What should be our criteria to raise an alert, I though it
>>>>>> should
>>>>>>>> be
>>>>>>>>>>> when
>>>>>>>>>>>> its value goes below 0.3. However, the value seems to be
>>>>> below
>>>>>>> 0.3
>>>>>>>> a
>>>>>>>>>> lot
>>>>>>>>>>> of
>>>>>>>>>>>> the times, almost always if we take samples every five
>>>> mins.
>>>>>> What
>>>>>>>>>> should
>>>>>>>>>>> be
>>>>>>>>>>>> the threshold to raise an alarm ?
>>>>>>>>>>>>
>>>>>>>>>>>> What would be the impact of having this below 0.3 or even
>>>>> zero
>>>>>>> like
>>>>>>>>>> most
>>>>>>>>>>> of
>>>>>>>>>>>> the times?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -Zakee
>>>>>>>>>>>>
>>>> ____________________________________________________________
>>>>>>>>>>>> How Old Men Tighten Skin
>>>>>>>>>>>> 63 Year Old Man Shares DIY Skin Tightening Method You Can
>>>> Do
>>>>>> From
>>>>>>>>> Home
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
>>>>>>>>>>> ____________________________________________________________
>>>>>>>>>>> 8% Annuity Return Secret
>>>>>>>>>>> Earn Guaranteed Income for Life! Compare Rates Today.
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>> http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
>>>>>>>>>>
>>>>>>>>> ____________________________________________________________
>>>>>>>>> High School Yearbooks
>>>>>>>>> View Class Yearbooks Online Free. Reminisce & Buy a Reprint
>>>> Today!
>>>>>>>>>
>>>>>>>
>>>>>
>> http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc
>>>>>>>>
>>>>>>> ____________________________________________________________
>>>>>>> Extended Stay America
>>>>>>> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
>>>>>>>
>>>>>
>> http://thirdpartyoffers.netzero.net/TGL3255/54ed04fb9538c4fb6188mp15duc
>>>>>>
>>>>> ____________________________________________________________
>>>>> Extended Stay America
>>>>> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
>>>>>
>> http://thirdpartyoffers.netzero.net/TGL3255/54ee8dd7e1cd7dd777e8mp03duc
>>>>
>>> ____________________________________________________________
>>> Protect what matters
>>> Floods can happen anywhere. Learn your risk and find an agent today.
>>> http://thirdpartyoffers.netzero.net/TGL3255/54ef551637a8b5514471fmp01duc
>>
>>
> ____________________________________________________________
> What's your flood risk?
> Find flood maps, interactive tools, FAQs, and agents in your area.
> http://thirdpartyoffers.netzero.net/TGL3255/54f0ad572e6932d5774b1mp08duc <http://thirdpartyoffers.netzero.net/TGL3255/54f0ad572e6932d5774b1mp08duc>
Re: NetworkProcessorAvgIdlePercent
Posted by Jun Rao <ju...@confluent.io>.
Zakee,
It would be useful to get the following.
kafka.network:name=RequestQueueSize,type=RequestChannel
kafka.network:name=RequestQueueTimeMs,request=Fetch,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=Produce,type=RequestMetrics
Thanks,
Jun
On Thu, Feb 26, 2015 at 2:17 PM, Zakee <kz...@netzero.net> wrote:
> Hi Jun,
>
> There are too many KPIs generated by kafka that I can’t monitor them all,
> I have monitoring a filtered list that I can understand so far.
>
> Which keys are you talking of? Currently I am not monitoring any of below
> keys but will add only those that are useful.
>
> kafka.network:name=RequestQueueSize,type=RequestChannel
>
> kafka.network:name=RequestQueueTimeMs,request=ConsumerMetadata,type=RequestMetrics
>
> kafka.network:name=RequestQueueTimeMs,request=ControlledShutdown,type=RequestMetrics
> kafka.network:name=RequestQueueTimeMs,request=Fetch,type=RequestMetrics
>
> kafka.network:name=RequestQueueTimeMs,request=FetchConsumer,type=RequestMetrics
>
> kafka.network:name=RequestQueueTimeMs,request=FetchFollower,type=RequestMetrics
> kafka.network:name=RequestQueueTimeMs,request=Heartbeat,type=RequestMetrics
> kafka.network:name=RequestQueueTimeMs,request=JoinGroup,type=RequestMetrics
>
> kafka.network:name=RequestQueueTimeMs,request=LeaderAndIsr,type=RequestMetrics
> kafka.network:name=RequestQueueTimeMs,request=Metadata,type=RequestMetrics
>
> kafka.network:name=RequestQueueTimeMs,request=OffsetCommit,type=RequestMetrics
>
> kafka.network:name=RequestQueueTimeMs,request=OffsetFetch,type=RequestMetrics
> kafka.network:name=RequestQueueTimeMs,request=Offsets,type=RequestMetrics
> kafka.network:name=RequestQueueTimeMs,request=Produce,type=RequestMetrics
>
> kafka.network:name=RequestQueueTimeMs,request=StopReplica,type=RequestMetrics
>
> kafka.network:name=RequestQueueTimeMs,request=UpdateMetadata,type=RequestMetrics
>
>
> Thanks,
> Zakee
>
> > On Feb 26, 2015, at 9:10 AM, Jun Rao <ju...@confluent.io> wrote:
> >
> > That may be enough. What's the RequestQueueSize and RequestQueueTimeMs?
> >
> > Thanks,
> >
> > Jun
> >
> > On Wed, Feb 25, 2015 at 10:24 PM, Zakee <kz...@netzero.net> wrote:
> >
> >> Well currently I have configured 14 thread for both io and network. Do
> you
> >> think we should consider more?
> >>
> >> Thanks
> >> -Zakee
> >>
> >> On Wed, Feb 25, 2015 at 6:22 PM, Jun Rao <ju...@confluent.io> wrote:
> >>
> >>> Then you may want to consider increasing num.io.threads
> >>> and num.network.threads.
> >>>
> >>> Thanks,
> >>>
> >>> Jun
> >>>
> >>> On Tue, Feb 24, 2015 at 7:48 PM, Zakee <kz...@netzero.net> wrote:
> >>>
> >>>> Similar pattern for that too. Mostly hovering below.
> >>>>
> >>>> -Zakee
> >>>>
> >>>> On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao <ju...@confluent.io> wrote:
> >>>>
> >>>>> What about RequestHandlerAvgIdlePercent?
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Jun
> >>>>>
> >>>>> On Mon, Feb 23, 2015 at 8:47 PM, Zakee <kz...@netzero.net> wrote:
> >>>>>
> >>>>>> Hi Jun,
> >>>>>>
> >>>>>> With ~100G of data being pushed per hour across 35 topics
> >>>>>> (replication-factor 3), the NetworkProcessorAvgIdlePercent is
> >> mostly
> >>>>>> showing below 0.5 sometimes when the producers send on a high rate.
> >>>>>>
> >>>>>> Thanks
> >>>>>> -Zakee
> >>>>>>
> >>>>>> On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <ju...@confluent.io>
> >> wrote:
> >>>>>>
> >>>>>>> What kind of load do you have on the brokers? On an idle cluster
> >>>> (just
> >>>>>>> fetch requests from the follower replicas), I
> >>>>>>> saw NetworkProcessorAvgIdlePercent at about 97%.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>> Jun
> >>>>>>>
> >>>>>>> On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kz...@netzero.net>
> >>> wrote:
> >>>>>>>
> >>>>>>>> Jun,
> >>>>>>>>
> >>>>>>>> I am already using the latest release 0.8.2.1.
> >>>>>>>>
> >>>>>>>> -Zakee
> >>>>>>>>
> >>>>>>>> On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io>
> >>> wrote:
> >>>>>>>>
> >>>>>>>>> Could you try the 0.8.2.1 release being voted on now? It
> >> fixes
> >>> a
> >>>>> CPU
> >>>>>>>> issue
> >>>>>>>>> and should reduce the CPU load in network thread.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>>
> >>>>>>>>> Jun
> >>>>>>>>>
> >>>>>>>>> On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kzakee1@netzero.net
> >>>
> >>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Kafka documentation recommends <0.3 for above metric. I
> >>> assume
> >>>>>>>> processor
> >>>>>>>>> is
> >>>>>>>>>> busier if this goes below 0.3 and obviously it being < 0.3
> >>> for
> >>>>> long
> >>>>>>>> does
> >>>>>>>>>> not seem to be a good sign.
> >>>>>>>>>>
> >>>>>>>>>> What should be our criteria to raise an alert, I though it
> >>>> should
> >>>>>> be
> >>>>>>>>> when
> >>>>>>>>>> its value goes below 0.3. However, the value seems to be
> >>> below
> >>>>> 0.3
> >>>>>> a
> >>>>>>>> lot
> >>>>>>>>> of
> >>>>>>>>>> the times, almost always if we take samples every five
> >> mins.
> >>>> What
> >>>>>>>> should
> >>>>>>>>> be
> >>>>>>>>>> the threshold to raise an alarm ?
> >>>>>>>>>>
> >>>>>>>>>> What would be the impact of having this below 0.3 or even
> >>> zero
> >>>>> like
> >>>>>>>> most
> >>>>>>>>> of
> >>>>>>>>>> the times?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> -Zakee
> >>>>>>>>>>
> >> ____________________________________________________________
> >>>>>>>>>> How Old Men Tighten Skin
> >>>>>>>>>> 63 Year Old Man Shares DIY Skin Tightening Method You Can
> >> Do
> >>>> From
> >>>>>>> Home
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>
> http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
> >>>>>>>>> ____________________________________________________________
> >>>>>>>>> 8% Annuity Return Secret
> >>>>>>>>> Earn Guaranteed Income for Life! Compare Rates Today.
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
> >>>>>>>>
> >>>>>>> ____________________________________________________________
> >>>>>>> High School Yearbooks
> >>>>>>> View Class Yearbooks Online Free. Reminisce & Buy a Reprint
> >> Today!
> >>>>>>>
> >>>>>
> >>>
> http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc
> >>>>>>
> >>>>> ____________________________________________________________
> >>>>> Extended Stay America
> >>>>> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
> >>>>>
> >>>
> http://thirdpartyoffers.netzero.net/TGL3255/54ed04fb9538c4fb6188mp15duc
> >>>>
> >>> ____________________________________________________________
> >>> Extended Stay America
> >>> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
> >>>
> http://thirdpartyoffers.netzero.net/TGL3255/54ee8dd7e1cd7dd777e8mp03duc
> >>
> > ____________________________________________________________
> > Protect what matters
> > Floods can happen anywhere. Learn your risk and find an agent today.
> > http://thirdpartyoffers.netzero.net/TGL3255/54ef551637a8b5514471fmp01duc
>
>
Re: NetworkProcessorAvgIdlePercent
Posted by Zakee <kz...@netzero.net>.
Hi Jun,
There are too many KPIs generated by kafka that I can’t monitor them all, I have monitoring a filtered list that I can understand so far.
Which keys are you talking of? Currently I am not monitoring any of below keys but will add only those that are useful.
kafka.network:name=RequestQueueSize,type=RequestChannel
kafka.network:name=RequestQueueTimeMs,request=ConsumerMetadata,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=ControlledShutdown,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=Fetch,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=FetchConsumer,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=FetchFollower,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=Heartbeat,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=JoinGroup,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=LeaderAndIsr,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=Metadata,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=OffsetCommit,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=OffsetFetch,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=Offsets,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=Produce,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=StopReplica,type=RequestMetrics
kafka.network:name=RequestQueueTimeMs,request=UpdateMetadata,type=RequestMetrics
Thanks,
Zakee
> On Feb 26, 2015, at 9:10 AM, Jun Rao <ju...@confluent.io> wrote:
>
> That may be enough. What's the RequestQueueSize and RequestQueueTimeMs?
>
> Thanks,
>
> Jun
>
> On Wed, Feb 25, 2015 at 10:24 PM, Zakee <kz...@netzero.net> wrote:
>
>> Well currently I have configured 14 thread for both io and network. Do you
>> think we should consider more?
>>
>> Thanks
>> -Zakee
>>
>> On Wed, Feb 25, 2015 at 6:22 PM, Jun Rao <ju...@confluent.io> wrote:
>>
>>> Then you may want to consider increasing num.io.threads
>>> and num.network.threads.
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>> On Tue, Feb 24, 2015 at 7:48 PM, Zakee <kz...@netzero.net> wrote:
>>>
>>>> Similar pattern for that too. Mostly hovering below.
>>>>
>>>> -Zakee
>>>>
>>>> On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao <ju...@confluent.io> wrote:
>>>>
>>>>> What about RequestHandlerAvgIdlePercent?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Jun
>>>>>
>>>>> On Mon, Feb 23, 2015 at 8:47 PM, Zakee <kz...@netzero.net> wrote:
>>>>>
>>>>>> Hi Jun,
>>>>>>
>>>>>> With ~100G of data being pushed per hour across 35 topics
>>>>>> (replication-factor 3), the NetworkProcessorAvgIdlePercent is
>> mostly
>>>>>> showing below 0.5 sometimes when the producers send on a high rate.
>>>>>>
>>>>>> Thanks
>>>>>> -Zakee
>>>>>>
>>>>>> On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <ju...@confluent.io>
>> wrote:
>>>>>>
>>>>>>> What kind of load do you have on the brokers? On an idle cluster
>>>> (just
>>>>>>> fetch requests from the follower replicas), I
>>>>>>> saw NetworkProcessorAvgIdlePercent at about 97%.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Jun
>>>>>>>
>>>>>>> On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kz...@netzero.net>
>>> wrote:
>>>>>>>
>>>>>>>> Jun,
>>>>>>>>
>>>>>>>> I am already using the latest release 0.8.2.1.
>>>>>>>>
>>>>>>>> -Zakee
>>>>>>>>
>>>>>>>> On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io>
>>> wrote:
>>>>>>>>
>>>>>>>>> Could you try the 0.8.2.1 release being voted on now? It
>> fixes
>>> a
>>>>> CPU
>>>>>>>> issue
>>>>>>>>> and should reduce the CPU load in network thread.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Jun
>>>>>>>>>
>>>>>>>>> On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kzakee1@netzero.net
>>>
>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Kafka documentation recommends <0.3 for above metric. I
>>> assume
>>>>>>>> processor
>>>>>>>>> is
>>>>>>>>>> busier if this goes below 0.3 and obviously it being < 0.3
>>> for
>>>>> long
>>>>>>>> does
>>>>>>>>>> not seem to be a good sign.
>>>>>>>>>>
>>>>>>>>>> What should be our criteria to raise an alert, I though it
>>>> should
>>>>>> be
>>>>>>>>> when
>>>>>>>>>> its value goes below 0.3. However, the value seems to be
>>> below
>>>>> 0.3
>>>>>> a
>>>>>>>> lot
>>>>>>>>> of
>>>>>>>>>> the times, almost always if we take samples every five
>> mins.
>>>> What
>>>>>>>> should
>>>>>>>>> be
>>>>>>>>>> the threshold to raise an alarm ?
>>>>>>>>>>
>>>>>>>>>> What would be the impact of having this below 0.3 or even
>>> zero
>>>>> like
>>>>>>>> most
>>>>>>>>> of
>>>>>>>>>> the times?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -Zakee
>>>>>>>>>>
>> ____________________________________________________________
>>>>>>>>>> How Old Men Tighten Skin
>>>>>>>>>> 63 Year Old Man Shares DIY Skin Tightening Method You Can
>> Do
>>>> From
>>>>>>> Home
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
>>>>>>>>> ____________________________________________________________
>>>>>>>>> 8% Annuity Return Secret
>>>>>>>>> Earn Guaranteed Income for Life! Compare Rates Today.
>>>>>>>>>
>>>>>>>
>>>>>
>>> http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
>>>>>>>>
>>>>>>> ____________________________________________________________
>>>>>>> High School Yearbooks
>>>>>>> View Class Yearbooks Online Free. Reminisce & Buy a Reprint
>> Today!
>>>>>>>
>>>>>
>>> http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc
>>>>>>
>>>>> ____________________________________________________________
>>>>> Extended Stay America
>>>>> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
>>>>>
>>> http://thirdpartyoffers.netzero.net/TGL3255/54ed04fb9538c4fb6188mp15duc
>>>>
>>> ____________________________________________________________
>>> Extended Stay America
>>> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
>>> http://thirdpartyoffers.netzero.net/TGL3255/54ee8dd7e1cd7dd777e8mp03duc
>>
> ____________________________________________________________
> Protect what matters
> Floods can happen anywhere. Learn your risk and find an agent today.
> http://thirdpartyoffers.netzero.net/TGL3255/54ef551637a8b5514471fmp01duc
Re: NetworkProcessorAvgIdlePercent
Posted by Jun Rao <ju...@confluent.io>.
That may be enough. What's the RequestQueueSize and RequestQueueTimeMs?
Thanks,
Jun
On Wed, Feb 25, 2015 at 10:24 PM, Zakee <kz...@netzero.net> wrote:
> Well currently I have configured 14 thread for both io and network. Do you
> think we should consider more?
>
> Thanks
> -Zakee
>
> On Wed, Feb 25, 2015 at 6:22 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Then you may want to consider increasing num.io.threads
> > and num.network.threads.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Feb 24, 2015 at 7:48 PM, Zakee <kz...@netzero.net> wrote:
> >
> > > Similar pattern for that too. Mostly hovering below.
> > >
> > > -Zakee
> > >
> > > On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > What about RequestHandlerAvgIdlePercent?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Mon, Feb 23, 2015 at 8:47 PM, Zakee <kz...@netzero.net> wrote:
> > > >
> > > > > Hi Jun,
> > > > >
> > > > > With ~100G of data being pushed per hour across 35 topics
> > > > > (replication-factor 3), the NetworkProcessorAvgIdlePercent is
> mostly
> > > > > showing below 0.5 sometimes when the producers send on a high rate.
> > > > >
> > > > > Thanks
> > > > > -Zakee
> > > > >
> > > > > On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <ju...@confluent.io>
> wrote:
> > > > >
> > > > > > What kind of load do you have on the brokers? On an idle cluster
> > > (just
> > > > > > fetch requests from the follower replicas), I
> > > > > > saw NetworkProcessorAvgIdlePercent at about 97%.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kz...@netzero.net>
> > wrote:
> > > > > >
> > > > > > > Jun,
> > > > > > >
> > > > > > > I am already using the latest release 0.8.2.1.
> > > > > > >
> > > > > > > -Zakee
> > > > > > >
> > > > > > > On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > > >
> > > > > > > > Could you try the 0.8.2.1 release being voted on now? It
> fixes
> > a
> > > > CPU
> > > > > > > issue
> > > > > > > > and should reduce the CPU load in network thread.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > > > On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kzakee1@netzero.net
> >
> > > > wrote:
> > > > > > > >
> > > > > > > > > Kafka documentation recommends <0.3 for above metric. I
> > assume
> > > > > > > processor
> > > > > > > > is
> > > > > > > > > busier if this goes below 0.3 and obviously it being < 0.3
> > for
> > > > long
> > > > > > > does
> > > > > > > > > not seem to be a good sign.
> > > > > > > > >
> > > > > > > > > What should be our criteria to raise an alert, I though it
> > > should
> > > > > be
> > > > > > > > when
> > > > > > > > > its value goes below 0.3. However, the value seems to be
> > below
> > > > 0.3
> > > > > a
> > > > > > > lot
> > > > > > > > of
> > > > > > > > > the times, almost always if we take samples every five
> mins.
> > > What
> > > > > > > should
> > > > > > > > be
> > > > > > > > > the threshold to raise an alarm ?
> > > > > > > > >
> > > > > > > > > What would be the impact of having this below 0.3 or even
> > zero
> > > > like
> > > > > > > most
> > > > > > > > of
> > > > > > > > > the times?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > -Zakee
> > > > > > > > >
> ____________________________________________________________
> > > > > > > > > How Old Men Tighten Skin
> > > > > > > > > 63 Year Old Man Shares DIY Skin Tightening Method You Can
> Do
> > > From
> > > > > > Home
> > > > > > > > >
> > > > > > >
> > > > >
> > >
> http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
> > > > > > > > ____________________________________________________________
> > > > > > > > 8% Annuity Return Secret
> > > > > > > > Earn Guaranteed Income for Life! Compare Rates Today.
> > > > > > > >
> > > > > >
> > > >
> > http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
> > > > > > >
> > > > > > ____________________________________________________________
> > > > > > High School Yearbooks
> > > > > > View Class Yearbooks Online Free. Reminisce & Buy a Reprint
> Today!
> > > > > >
> > > >
> > http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc
> > > > >
> > > > ____________________________________________________________
> > > > Extended Stay America
> > > > Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
> > > >
> > http://thirdpartyoffers.netzero.net/TGL3255/54ed04fb9538c4fb6188mp15duc
> > >
> > ____________________________________________________________
> > Extended Stay America
> > Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
> > http://thirdpartyoffers.netzero.net/TGL3255/54ee8dd7e1cd7dd777e8mp03duc
>
Re: NetworkProcessorAvgIdlePercent
Posted by Zakee <kz...@netzero.net>.
Well currently I have configured 14 thread for both io and network. Do you
think we should consider more?
Thanks
-Zakee
On Wed, Feb 25, 2015 at 6:22 PM, Jun Rao <ju...@confluent.io> wrote:
> Then you may want to consider increasing num.io.threads
> and num.network.threads.
>
> Thanks,
>
> Jun
>
> On Tue, Feb 24, 2015 at 7:48 PM, Zakee <kz...@netzero.net> wrote:
>
> > Similar pattern for that too. Mostly hovering below.
> >
> > -Zakee
> >
> > On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > What about RequestHandlerAvgIdlePercent?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Feb 23, 2015 at 8:47 PM, Zakee <kz...@netzero.net> wrote:
> > >
> > > > Hi Jun,
> > > >
> > > > With ~100G of data being pushed per hour across 35 topics
> > > > (replication-factor 3), the NetworkProcessorAvgIdlePercent is mostly
> > > > showing below 0.5 sometimes when the producers send on a high rate.
> > > >
> > > > Thanks
> > > > -Zakee
> > > >
> > > > On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > What kind of load do you have on the brokers? On an idle cluster
> > (just
> > > > > fetch requests from the follower replicas), I
> > > > > saw NetworkProcessorAvgIdlePercent at about 97%.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kz...@netzero.net>
> wrote:
> > > > >
> > > > > > Jun,
> > > > > >
> > > > > > I am already using the latest release 0.8.2.1.
> > > > > >
> > > > > > -Zakee
> > > > > >
> > > > > > On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io>
> wrote:
> > > > > >
> > > > > > > Could you try the 0.8.2.1 release being voted on now? It fixes
> a
> > > CPU
> > > > > > issue
> > > > > > > and should reduce the CPU load in network thread.
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > > > On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kz...@netzero.net>
> > > wrote:
> > > > > > >
> > > > > > > > Kafka documentation recommends <0.3 for above metric. I
> assume
> > > > > > processor
> > > > > > > is
> > > > > > > > busier if this goes below 0.3 and obviously it being < 0.3
> for
> > > long
> > > > > > does
> > > > > > > > not seem to be a good sign.
> > > > > > > >
> > > > > > > > What should be our criteria to raise an alert, I though it
> > should
> > > > be
> > > > > > > when
> > > > > > > > its value goes below 0.3. However, the value seems to be
> below
> > > 0.3
> > > > a
> > > > > > lot
> > > > > > > of
> > > > > > > > the times, almost always if we take samples every five mins.
> > What
> > > > > > should
> > > > > > > be
> > > > > > > > the threshold to raise an alarm ?
> > > > > > > >
> > > > > > > > What would be the impact of having this below 0.3 or even
> zero
> > > like
> > > > > > most
> > > > > > > of
> > > > > > > > the times?
> > > > > > > >
> > > > > > > >
> > > > > > > > -Zakee
> > > > > > > > ____________________________________________________________
> > > > > > > > How Old Men Tighten Skin
> > > > > > > > 63 Year Old Man Shares DIY Skin Tightening Method You Can Do
> > From
> > > > > Home
> > > > > > > >
> > > > > >
> > > >
> > http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
> > > > > > > ____________________________________________________________
> > > > > > > 8% Annuity Return Secret
> > > > > > > Earn Guaranteed Income for Life! Compare Rates Today.
> > > > > > >
> > > > >
> > >
> http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
> > > > > >
> > > > > ____________________________________________________________
> > > > > High School Yearbooks
> > > > > View Class Yearbooks Online Free. Reminisce & Buy a Reprint Today!
> > > > >
> > >
> http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc
> > > >
> > > ____________________________________________________________
> > > Extended Stay America
> > > Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
> > >
> http://thirdpartyoffers.netzero.net/TGL3255/54ed04fb9538c4fb6188mp15duc
> >
> ____________________________________________________________
> Extended Stay America
> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
> http://thirdpartyoffers.netzero.net/TGL3255/54ee8dd7e1cd7dd777e8mp03duc
Re: NetworkProcessorAvgIdlePercent
Posted by Jun Rao <ju...@confluent.io>.
Then you may want to consider increasing num.io.threads
and num.network.threads.
Thanks,
Jun
On Tue, Feb 24, 2015 at 7:48 PM, Zakee <kz...@netzero.net> wrote:
> Similar pattern for that too. Mostly hovering below.
>
> -Zakee
>
> On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > What about RequestHandlerAvgIdlePercent?
> >
> > Thanks,
> >
> > Jun
> >
> > On Mon, Feb 23, 2015 at 8:47 PM, Zakee <kz...@netzero.net> wrote:
> >
> > > Hi Jun,
> > >
> > > With ~100G of data being pushed per hour across 35 topics
> > > (replication-factor 3), the NetworkProcessorAvgIdlePercent is mostly
> > > showing below 0.5 sometimes when the producers send on a high rate.
> > >
> > > Thanks
> > > -Zakee
> > >
> > > On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > What kind of load do you have on the brokers? On an idle cluster
> (just
> > > > fetch requests from the follower replicas), I
> > > > saw NetworkProcessorAvgIdlePercent at about 97%.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kz...@netzero.net> wrote:
> > > >
> > > > > Jun,
> > > > >
> > > > > I am already using the latest release 0.8.2.1.
> > > > >
> > > > > -Zakee
> > > > >
> > > > > On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > > Could you try the 0.8.2.1 release being voted on now? It fixes a
> > CPU
> > > > > issue
> > > > > > and should reduce the CPU load in network thread.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kz...@netzero.net>
> > wrote:
> > > > > >
> > > > > > > Kafka documentation recommends <0.3 for above metric. I assume
> > > > > processor
> > > > > > is
> > > > > > > busier if this goes below 0.3 and obviously it being < 0.3 for
> > long
> > > > > does
> > > > > > > not seem to be a good sign.
> > > > > > >
> > > > > > > What should be our criteria to raise an alert, I though it
> should
> > > be
> > > > > > when
> > > > > > > its value goes below 0.3. However, the value seems to be below
> > 0.3
> > > a
> > > > > lot
> > > > > > of
> > > > > > > the times, almost always if we take samples every five mins.
> What
> > > > > should
> > > > > > be
> > > > > > > the threshold to raise an alarm ?
> > > > > > >
> > > > > > > What would be the impact of having this below 0.3 or even zero
> > like
> > > > > most
> > > > > > of
> > > > > > > the times?
> > > > > > >
> > > > > > >
> > > > > > > -Zakee
> > > > > > > ____________________________________________________________
> > > > > > > How Old Men Tighten Skin
> > > > > > > 63 Year Old Man Shares DIY Skin Tightening Method You Can Do
> From
> > > > Home
> > > > > > >
> > > > >
> > >
> http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
> > > > > > ____________________________________________________________
> > > > > > 8% Annuity Return Secret
> > > > > > Earn Guaranteed Income for Life! Compare Rates Today.
> > > > > >
> > > >
> > http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
> > > > >
> > > > ____________________________________________________________
> > > > High School Yearbooks
> > > > View Class Yearbooks Online Free. Reminisce & Buy a Reprint Today!
> > > >
> > http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc
> > >
> > ____________________________________________________________
> > Extended Stay America
> > Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
> > http://thirdpartyoffers.netzero.net/TGL3255/54ed04fb9538c4fb6188mp15duc
>
Re: NetworkProcessorAvgIdlePercent
Posted by Zakee <kz...@netzero.net>.
Similar pattern for that too. Mostly hovering below.
-Zakee
On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao <ju...@confluent.io> wrote:
> What about RequestHandlerAvgIdlePercent?
>
> Thanks,
>
> Jun
>
> On Mon, Feb 23, 2015 at 8:47 PM, Zakee <kz...@netzero.net> wrote:
>
> > Hi Jun,
> >
> > With ~100G of data being pushed per hour across 35 topics
> > (replication-factor 3), the NetworkProcessorAvgIdlePercent is mostly
> > showing below 0.5 sometimes when the producers send on a high rate.
> >
> > Thanks
> > -Zakee
> >
> > On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > What kind of load do you have on the brokers? On an idle cluster (just
> > > fetch requests from the follower replicas), I
> > > saw NetworkProcessorAvgIdlePercent at about 97%.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kz...@netzero.net> wrote:
> > >
> > > > Jun,
> > > >
> > > > I am already using the latest release 0.8.2.1.
> > > >
> > > > -Zakee
> > > >
> > > > On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Could you try the 0.8.2.1 release being voted on now? It fixes a
> CPU
> > > > issue
> > > > > and should reduce the CPU load in network thread.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kz...@netzero.net>
> wrote:
> > > > >
> > > > > > Kafka documentation recommends <0.3 for above metric. I assume
> > > > processor
> > > > > is
> > > > > > busier if this goes below 0.3 and obviously it being < 0.3 for
> long
> > > > does
> > > > > > not seem to be a good sign.
> > > > > >
> > > > > > What should be our criteria to raise an alert, I though it should
> > be
> > > > > when
> > > > > > its value goes below 0.3. However, the value seems to be below
> 0.3
> > a
> > > > lot
> > > > > of
> > > > > > the times, almost always if we take samples every five mins. What
> > > > should
> > > > > be
> > > > > > the threshold to raise an alarm ?
> > > > > >
> > > > > > What would be the impact of having this below 0.3 or even zero
> like
> > > > most
> > > > > of
> > > > > > the times?
> > > > > >
> > > > > >
> > > > > > -Zakee
> > > > > > ____________________________________________________________
> > > > > > How Old Men Tighten Skin
> > > > > > 63 Year Old Man Shares DIY Skin Tightening Method You Can Do From
> > > Home
> > > > > >
> > > >
> > http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
> > > > > ____________________________________________________________
> > > > > 8% Annuity Return Secret
> > > > > Earn Guaranteed Income for Life! Compare Rates Today.
> > > > >
> > >
> http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
> > > >
> > > ____________________________________________________________
> > > High School Yearbooks
> > > View Class Yearbooks Online Free. Reminisce & Buy a Reprint Today!
> > >
> http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc
> >
> ____________________________________________________________
> Extended Stay America
> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed.
> http://thirdpartyoffers.netzero.net/TGL3255/54ed04fb9538c4fb6188mp15duc
Re: NetworkProcessorAvgIdlePercent
Posted by Jun Rao <ju...@confluent.io>.
What about RequestHandlerAvgIdlePercent?
Thanks,
Jun
On Mon, Feb 23, 2015 at 8:47 PM, Zakee <kz...@netzero.net> wrote:
> Hi Jun,
>
> With ~100G of data being pushed per hour across 35 topics
> (replication-factor 3), the NetworkProcessorAvgIdlePercent is mostly
> showing below 0.5 sometimes when the producers send on a high rate.
>
> Thanks
> -Zakee
>
> On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > What kind of load do you have on the brokers? On an idle cluster (just
> > fetch requests from the follower replicas), I
> > saw NetworkProcessorAvgIdlePercent at about 97%.
> >
> > Thanks,
> >
> > Jun
> >
> > On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kz...@netzero.net> wrote:
> >
> > > Jun,
> > >
> > > I am already using the latest release 0.8.2.1.
> > >
> > > -Zakee
> > >
> > > On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Could you try the 0.8.2.1 release being voted on now? It fixes a CPU
> > > issue
> > > > and should reduce the CPU load in network thread.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kz...@netzero.net> wrote:
> > > >
> > > > > Kafka documentation recommends <0.3 for above metric. I assume
> > > processor
> > > > is
> > > > > busier if this goes below 0.3 and obviously it being < 0.3 for long
> > > does
> > > > > not seem to be a good sign.
> > > > >
> > > > > What should be our criteria to raise an alert, I though it should
> be
> > > > when
> > > > > its value goes below 0.3. However, the value seems to be below 0.3
> a
> > > lot
> > > > of
> > > > > the times, almost always if we take samples every five mins. What
> > > should
> > > > be
> > > > > the threshold to raise an alarm ?
> > > > >
> > > > > What would be the impact of having this below 0.3 or even zero like
> > > most
> > > > of
> > > > > the times?
> > > > >
> > > > >
> > > > > -Zakee
> > > > > ____________________________________________________________
> > > > > How Old Men Tighten Skin
> > > > > 63 Year Old Man Shares DIY Skin Tightening Method You Can Do From
> > Home
> > > > >
> > >
> http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
> > > > ____________________________________________________________
> > > > 8% Annuity Return Secret
> > > > Earn Guaranteed Income for Life! Compare Rates Today.
> > > >
> > http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
> > >
> > ____________________________________________________________
> > High School Yearbooks
> > View Class Yearbooks Online Free. Reminisce & Buy a Reprint Today!
> > http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc
>
Re: NetworkProcessorAvgIdlePercent
Posted by Zakee <kz...@netzero.net>.
Hi Jun,
With ~100G of data being pushed per hour across 35 topics
(replication-factor 3), the NetworkProcessorAvgIdlePercent is mostly
showing below 0.5 sometimes when the producers send on a high rate.
Thanks
-Zakee
On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <ju...@confluent.io> wrote:
> What kind of load do you have on the brokers? On an idle cluster (just
> fetch requests from the follower replicas), I
> saw NetworkProcessorAvgIdlePercent at about 97%.
>
> Thanks,
>
> Jun
>
> On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kz...@netzero.net> wrote:
>
> > Jun,
> >
> > I am already using the latest release 0.8.2.1.
> >
> > -Zakee
> >
> > On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Could you try the 0.8.2.1 release being voted on now? It fixes a CPU
> > issue
> > > and should reduce the CPU load in network thread.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kz...@netzero.net> wrote:
> > >
> > > > Kafka documentation recommends <0.3 for above metric. I assume
> > processor
> > > is
> > > > busier if this goes below 0.3 and obviously it being < 0.3 for long
> > does
> > > > not seem to be a good sign.
> > > >
> > > > What should be our criteria to raise an alert, I though it should be
> > > when
> > > > its value goes below 0.3. However, the value seems to be below 0.3 a
> > lot
> > > of
> > > > the times, almost always if we take samples every five mins. What
> > should
> > > be
> > > > the threshold to raise an alarm ?
> > > >
> > > > What would be the impact of having this below 0.3 or even zero like
> > most
> > > of
> > > > the times?
> > > >
> > > >
> > > > -Zakee
> > > > ____________________________________________________________
> > > > How Old Men Tighten Skin
> > > > 63 Year Old Man Shares DIY Skin Tightening Method You Can Do From
> Home
> > > >
> > http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
> > > ____________________________________________________________
> > > 8% Annuity Return Secret
> > > Earn Guaranteed Income for Life! Compare Rates Today.
> > >
> http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
> >
> ____________________________________________________________
> High School Yearbooks
> View Class Yearbooks Online Free. Reminisce & Buy a Reprint Today!
> http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc
Re: NetworkProcessorAvgIdlePercent
Posted by Jun Rao <ju...@confluent.io>.
What kind of load do you have on the brokers? On an idle cluster (just
fetch requests from the follower replicas), I
saw NetworkProcessorAvgIdlePercent at about 97%.
Thanks,
Jun
On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kz...@netzero.net> wrote:
> Jun,
>
> I am already using the latest release 0.8.2.1.
>
> -Zakee
>
> On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Could you try the 0.8.2.1 release being voted on now? It fixes a CPU
> issue
> > and should reduce the CPU load in network thread.
> >
> > Thanks,
> >
> > Jun
> >
> > On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kz...@netzero.net> wrote:
> >
> > > Kafka documentation recommends <0.3 for above metric. I assume
> processor
> > is
> > > busier if this goes below 0.3 and obviously it being < 0.3 for long
> does
> > > not seem to be a good sign.
> > >
> > > What should be our criteria to raise an alert, I though it should be
> > when
> > > its value goes below 0.3. However, the value seems to be below 0.3 a
> lot
> > of
> > > the times, almost always if we take samples every five mins. What
> should
> > be
> > > the threshold to raise an alarm ?
> > >
> > > What would be the impact of having this below 0.3 or even zero like
> most
> > of
> > > the times?
> > >
> > >
> > > -Zakee
> > > ____________________________________________________________
> > > How Old Men Tighten Skin
> > > 63 Year Old Man Shares DIY Skin Tightening Method You Can Do From Home
> > >
> http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
> > ____________________________________________________________
> > 8% Annuity Return Secret
> > Earn Guaranteed Income for Life! Compare Rates Today.
> > http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
>
Re: NetworkProcessorAvgIdlePercent
Posted by Zakee <kz...@netzero.net>.
Jun,
I am already using the latest release 0.8.2.1.
-Zakee
On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <ju...@confluent.io> wrote:
> Could you try the 0.8.2.1 release being voted on now? It fixes a CPU issue
> and should reduce the CPU load in network thread.
>
> Thanks,
>
> Jun
>
> On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kz...@netzero.net> wrote:
>
> > Kafka documentation recommends <0.3 for above metric. I assume processor
> is
> > busier if this goes below 0.3 and obviously it being < 0.3 for long does
> > not seem to be a good sign.
> >
> > What should be our criteria to raise an alert, I though it should be
> when
> > its value goes below 0.3. However, the value seems to be below 0.3 a lot
> of
> > the times, almost always if we take samples every five mins. What should
> be
> > the threshold to raise an alarm ?
> >
> > What would be the impact of having this below 0.3 or even zero like most
> of
> > the times?
> >
> >
> > -Zakee
> > ____________________________________________________________
> > How Old Men Tighten Skin
> > 63 Year Old Man Shares DIY Skin Tightening Method You Can Do From Home
> > http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc
> ____________________________________________________________
> 8% Annuity Return Secret
> Earn Guaranteed Income for Life! Compare Rates Today.
> http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc
Re: NetworkProcessorAvgIdlePercent
Posted by Jun Rao <ju...@confluent.io>.
Could you try the 0.8.2.1 release being voted on now? It fixes a CPU issue
and should reduce the CPU load in network thread.
Thanks,
Jun
On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kz...@netzero.net> wrote:
> Kafka documentation recommends <0.3 for above metric. I assume processor is
> busier if this goes below 0.3 and obviously it being < 0.3 for long does
> not seem to be a good sign.
>
> What should be our criteria to raise an alert, I though it should be when
> its value goes below 0.3. However, the value seems to be below 0.3 a lot of
> the times, almost always if we take samples every five mins. What should be
> the threshold to raise an alarm ?
>
> What would be the impact of having this below 0.3 or even zero like most of
> the times?
>
>
> -Zakee
> ____________________________________________________________
> How Old Men Tighten Skin
> 63 Year Old Man Shares DIY Skin Tightening Method You Can Do From Home
> http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc