You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Dhirendra Singh <dh...@gmail.com> on 2020/08/17 12:28:48 UTC
Kafka BrokerState Metric Value 3
I have a question regarding Kafka BrokerState Metric value 3. According to
the documentation value 3 means running state.
What does this running state mean for the broker? Does it mean data of all
partitions on this broker is in sync ?
Is it safe to assume that when broker transition to state 3 after restart
it recovered all partitions data from the leader and is in sync with the
leaders ?
Thanks,
dsingh
Re: Kafka BrokerState Metric Value 3
Posted by Karolis Pocius <ka...@sentiance.com.INVALID>.
Note that even when all partitions are in sync, leader election might have
not happened yet and the broker isn't serving anything. Which might be OK,
depending on your actual use case.
On Wed, Aug 19, 2020 at 11:40 AM Dhirendra Singh <dh...@gmail.com>
wrote:
> Thank you Peter !
> I intended to use broker state to determine the health but i was not sure.
> I will use under replicated partition metric instead.
>
> --dsingh
>
> On Wed, Aug 19, 2020 at 1:40 PM Peter Bukowinski <pm...@gmail.com> wrote:
>
> > The broker state metric just reports on the state of the broker itself,
> > not whether it is in sync. A replacement broker will quickly reach a
> broker
> > state of 3 on startup even though it has to catch up on many replicas.
> > Don’t rely on it for checking if a cluster/broker is healthy with no
> > under-replicated partitions.
> >
> > For that, you can look at the underreplicated partition count metric.
> >
> > -- Peter (from phone)
> >
> > > On Aug 19, 2020, at 12:52 AM, Dhirendra Singh <dh...@gmail.com>
> > wrote:
> > >
> > > So is this metric just gives information that broker process up and
> > running
> > > ? or does it indicate something more of broker state or partitions it
> > hold ?
> > >
> > >
> > >
> > >> On Mon, Aug 17, 2020 at 6:17 PM Karolis Pocius
> > >> <ka...@sentiance.com.invalid> wrote:
> > >>
> > >> I tried using this metric for determining when the broker is back in
> the
> > >> cluster and became the leader for partitions it owned before restart,
> > but
> > >> that's not the case.
> > >>
> > >> In the end I've settled for checking
> > >> kafka.server:name=LeaderCount,type=ReplicaManager which tells me when
> > the
> > >> broker is actually operational and serving data.
> > >>
> > >> On Mon, Aug 17, 2020 at 3:29 PM Dhirendra Singh <
> dhirendraks@gmail.com>
> > >> wrote:
> > >>
> > >>> I have a question regarding Kafka BrokerState Metric value 3.
> According
> > >> to
> > >>> the documentation value 3 means running state.
> > >>> What does this running state mean for the broker? Does it mean data
> of
> > >> all
> > >>> partitions on this broker is in sync ?
> > >>> Is it safe to assume that when broker transition to state 3 after
> > restart
> > >>> it recovered all partitions data from the leader and is in sync with
> > the
> > >>> leaders ?
> > >>>
> > >>> Thanks,
> > >>> dsingh
> > >>>
> > >>
> >
>
Re: Kafka BrokerState Metric Value 3
Posted by Dhirendra Singh <dh...@gmail.com>.
Thank you Peter !
I intended to use broker state to determine the health but i was not sure.
I will use under replicated partition metric instead.
--dsingh
On Wed, Aug 19, 2020 at 1:40 PM Peter Bukowinski <pm...@gmail.com> wrote:
> The broker state metric just reports on the state of the broker itself,
> not whether it is in sync. A replacement broker will quickly reach a broker
> state of 3 on startup even though it has to catch up on many replicas.
> Don’t rely on it for checking if a cluster/broker is healthy with no
> under-replicated partitions.
>
> For that, you can look at the underreplicated partition count metric.
>
> -- Peter (from phone)
>
> > On Aug 19, 2020, at 12:52 AM, Dhirendra Singh <dh...@gmail.com>
> wrote:
> >
> > So is this metric just gives information that broker process up and
> running
> > ? or does it indicate something more of broker state or partitions it
> hold ?
> >
> >
> >
> >> On Mon, Aug 17, 2020 at 6:17 PM Karolis Pocius
> >> <ka...@sentiance.com.invalid> wrote:
> >>
> >> I tried using this metric for determining when the broker is back in the
> >> cluster and became the leader for partitions it owned before restart,
> but
> >> that's not the case.
> >>
> >> In the end I've settled for checking
> >> kafka.server:name=LeaderCount,type=ReplicaManager which tells me when
> the
> >> broker is actually operational and serving data.
> >>
> >> On Mon, Aug 17, 2020 at 3:29 PM Dhirendra Singh <dh...@gmail.com>
> >> wrote:
> >>
> >>> I have a question regarding Kafka BrokerState Metric value 3. According
> >> to
> >>> the documentation value 3 means running state.
> >>> What does this running state mean for the broker? Does it mean data of
> >> all
> >>> partitions on this broker is in sync ?
> >>> Is it safe to assume that when broker transition to state 3 after
> restart
> >>> it recovered all partitions data from the leader and is in sync with
> the
> >>> leaders ?
> >>>
> >>> Thanks,
> >>> dsingh
> >>>
> >>
>
Re: Kafka BrokerState Metric Value 3
Posted by Peter Bukowinski <pm...@gmail.com>.
The broker state metric just reports on the state of the broker itself, not whether it is in sync. A replacement broker will quickly reach a broker state of 3 on startup even though it has to catch up on many replicas. Don’t rely on it for checking if a cluster/broker is healthy with no under-replicated partitions.
For that, you can look at the underreplicated partition count metric.
-- Peter (from phone)
> On Aug 19, 2020, at 12:52 AM, Dhirendra Singh <dh...@gmail.com> wrote:
>
> So is this metric just gives information that broker process up and running
> ? or does it indicate something more of broker state or partitions it hold ?
>
>
>
>> On Mon, Aug 17, 2020 at 6:17 PM Karolis Pocius
>> <ka...@sentiance.com.invalid> wrote:
>>
>> I tried using this metric for determining when the broker is back in the
>> cluster and became the leader for partitions it owned before restart, but
>> that's not the case.
>>
>> In the end I've settled for checking
>> kafka.server:name=LeaderCount,type=ReplicaManager which tells me when the
>> broker is actually operational and serving data.
>>
>> On Mon, Aug 17, 2020 at 3:29 PM Dhirendra Singh <dh...@gmail.com>
>> wrote:
>>
>>> I have a question regarding Kafka BrokerState Metric value 3. According
>> to
>>> the documentation value 3 means running state.
>>> What does this running state mean for the broker? Does it mean data of
>> all
>>> partitions on this broker is in sync ?
>>> Is it safe to assume that when broker transition to state 3 after restart
>>> it recovered all partitions data from the leader and is in sync with the
>>> leaders ?
>>>
>>> Thanks,
>>> dsingh
>>>
>>
Re: Kafka BrokerState Metric Value 3
Posted by Dhirendra Singh <dh...@gmail.com>.
So is this metric just gives information that broker process up and running
? or does it indicate something more of broker state or partitions it hold ?
On Mon, Aug 17, 2020 at 6:17 PM Karolis Pocius
<ka...@sentiance.com.invalid> wrote:
> I tried using this metric for determining when the broker is back in the
> cluster and became the leader for partitions it owned before restart, but
> that's not the case.
>
> In the end I've settled for checking
> kafka.server:name=LeaderCount,type=ReplicaManager which tells me when the
> broker is actually operational and serving data.
>
> On Mon, Aug 17, 2020 at 3:29 PM Dhirendra Singh <dh...@gmail.com>
> wrote:
>
> > I have a question regarding Kafka BrokerState Metric value 3. According
> to
> > the documentation value 3 means running state.
> > What does this running state mean for the broker? Does it mean data of
> all
> > partitions on this broker is in sync ?
> > Is it safe to assume that when broker transition to state 3 after restart
> > it recovered all partitions data from the leader and is in sync with the
> > leaders ?
> >
> > Thanks,
> > dsingh
> >
>
Re: Kafka BrokerState Metric Value 3
Posted by Karolis Pocius <ka...@sentiance.com.INVALID>.
I tried using this metric for determining when the broker is back in the
cluster and became the leader for partitions it owned before restart, but
that's not the case.
In the end I've settled for checking
kafka.server:name=LeaderCount,type=ReplicaManager which tells me when the
broker is actually operational and serving data.
On Mon, Aug 17, 2020 at 3:29 PM Dhirendra Singh <dh...@gmail.com>
wrote:
> I have a question regarding Kafka BrokerState Metric value 3. According to
> the documentation value 3 means running state.
> What does this running state mean for the broker? Does it mean data of all
> partitions on this broker is in sync ?
> Is it safe to assume that when broker transition to state 3 after restart
> it recovered all partitions data from the leader and is in sync with the
> leaders ?
>
> Thanks,
> dsingh
>