Re: [DISCUSS] KIP-164 Add unavailablePartitionCount and per-partition Unavailable metrics

[Dong Lin <li...@gmail.com>]

Hey Ismael,

Thanks for the feedback! Could you please vote for the KIP if it looks
good? Then I will find two more committers to vote as well.

Thanks,
Dong

On Tue, May 30, 2017 at 9:08 AM, Dong Lin <li...@gmail.com> wrote:

> Thanks Edoardo and everyone for the comment! That is a very good point. I
> have updated to wiki to use UnderMinIsrPartitionCount as the per-broker
> metric name and UnderMinIsr as the per-partition metric name. The
> motivation section has also been updated to clarify how the existence of
> UnderMinIsrPartition reduces the availability of the Kafka service.
>
> Please find the latest wiki at https://cwiki.apache.org/
> confluence/display/KAFKA/KIP-164-+Add+UnderMinIsrPartitionCount+and+
> per-partition+UnderMinIsr+metrics .
>
>
>
> On Tue, May 30, 2017 at 5:35 AM, Ismael Juma <is...@juma.me.uk> wrote:
>
>> Thanks for the KIP, Dong. I agree that that the metrics are useful. Like
>> Edoardo and Mickael said, it seems like it may be better to choose a
>> different name. A couple of additional suggestions:
>> `UnderMinIsrPartitionCount` and `UnderMinIsr`.
>>
>> Ismael
>>
>> On Tue, May 30, 2017 at 12:04 PM, Mickael Maison <
>> mickael.maison@gmail.com>
>> wrote:
>>
>> > What about simply calling them 'BelowIsrPartitionCount' and 'BelowIsr' ?
>> >
>> > On Tue, May 30, 2017 at 11:40 AM, Edoardo Comar <EC...@uk.ibm.com>
>> wrote:
>> > > Hi Dong,
>> > >
>> > > many thanks for the KIP. It's a very useful metric.
>> > >
>> > > by saying
>> > >> Unavailable partitions could be most easily defined as “The number of
>> > > partitions that this broker leads for which the ISR is insufficient to
>> > > meet the minimum ISR required.”
>> > >
>> > > I presume you meant to call 'Unavailable' the partitions whose
>> ISR.size <
>> > > min.insync  ?
>> > >
>> > > Now, a partition whose ISR is < min.insync can be still used to
>> consume
>> > > messages from. It also can be used to produce messages to, as long as
>> the
>> > > producer does not request acks=-1 (i.e. acks=all).
>> > >
>> > > So it is not exactly 'Unavailable' ... perhaps we could call it
>> 'Unsafe'
>> > ?
>> > > Or the community can come up with a better name.
>> > >
>> > > I recently had a few discussions about the issue, and I opened a PR to
>> > > update the docs (that's still hoping to be reviewed and merged ...
>> hint
>> > > hint :-)
>> > > https://github.com/apache/kafka/pull/3035
>> > > https://issues.apache.org/jira/browse/KAFKA-5290
>> > >
>> > > Thanks!
>> > > Edo
>> > > --------------------------------------------------
>> > > Edoardo Comar
>> > > IBM Message Hub
>> > > ecomar@uk.ibm.com
>> > > IBM UK Ltd, Hursley Park, SO21 2JN
>> > >
>> > >
>> > >
>> > >
>> > > From:   Mickael Maison <mi...@gmail.com>
>> > > To:     dev@kafka.apache.org
>> > > Date:   30/05/2017 10:51
>> > > Subject:        Re: [DISCUSS] KIP-164 Add unavailablePartitionCount
>> and
>> > > per-partition Unavailable metrics
>> > >
>> > >
>> > >
>> > > +1
>> > > It's a mystery how this didn't already exist as it's one of the key
>> > > cluster's health indicator
>> > >
>> > > On Mon, May 29, 2017 at 9:18 AM, Gwen Shapira <gw...@confluent.io>
>> wrote:
>> > >> Hi,
>> > >>
>> > >> Sounds good. I was sure this existed already for some reason :)
>> > >>
>> > >> On Sun, May 28, 2017 at 11:06 AM Dong Lin <li...@gmail.com>
>> wrote:
>> > >>
>> > >>> Hi,
>> > >>>
>> > >>> We created KIP-164 to propose adding per-partition metric
>> *Unavailable*
>> > > and
>> > >>> per-broker metric *UnavailablePartitionCount*
>> > >>>
>> > >>> The KIP wik can be found at
>> > >>>
>> > >>>
>> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-164-+Add+
>> > unavailablePartitionCount+and+per-partition+Unavailable+metrics
>> > >
>> > >>> .
>> > >>>
>> > >>> Comments are welcome.
>> > >>>
>> > >>> Thanks,
>> > >>> Dong
>> > >>>
>> > >
>> > >
>> > >
>> > >
>> > > Unless stated otherwise above:
>> > > IBM United Kingdom Limited - Registered in England and Wales with
>> number
>> > > 741598.
>> > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
>> > 3AU
>> > >
>> >
>>
>
>