You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Hu Xi <hu...@hotmail.com> on 2017/12/05 07:26:43 UTC
答复: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
Guozhang,
Thanks for the vote and comments. I am not sure if I fully understand the parent metrics here. This KIP will introduce a client-level metric named 'records-lead-min' and three per-partition metrics tagged with topic&partition. Is it the child-parent relationship you mean?
________________________________
发件人: Guozhang Wang <wa...@gmail.com>
发送时间: 2017年12月5日 15:16
收件人: dev@kafka.apache.org
主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
Thanks Hu Xi,
I made a pass over the KIP and it lgtm. +1.
Just a clarification question: for the cross-partition "records-lead-min"
metric, would that be implemented as a parent metric of the per-partition
metrics?
Guozhang
On Mon, Dec 4, 2017 at 3:07 PM, Dong Lin <li...@gmail.com> wrote:
> +1 (non-binding)
>
> On Wed, Nov 29, 2017 at 7:05 PM, Hu Xi <hu...@hotmail.com> wrote:
>
> > Hi all,
> >
> > As I didn't see any further discussion around this KIP, I'd like to start
> > voting.
> >
> > KIP documentation:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 223+-+Add+per-topic+min+lead+and+per-partition+lead+
> > metrics+to+KafkaConsumer
> >
> >
> >
> > Cheers,
> >
> > huxihx
> >
>
--
-- Guozhang
Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
metrics to KafkaConsumer
Posted by Becket Qin <be...@gmail.com>.
+1, Thanks for the KIP.
One thing is that we might want to make this in consistent with what
proposed in KIP-225.
On Wed, Dec 6, 2017 at 6:40 AM, Guozhang Wang <wa...@gmail.com> wrote:
> Hi Hu Xi,
>
> As I said before, it is only a clarification question for its internal
> implementation; it is not related to the public interfaces.
>
>
> Guozhang
>
> On Wed, Dec 6, 2017 at 12:34 AM, Hu Xi <hu...@hotmail.com> wrote:
>
> > Hi Guozhang,
> >
> >
> > Correct me if I am wrong. Of course I could construct the sensor where
> > per-partition metrics reside as a child of the client-level
> > "records-lead-min", but it seems that whether doing that way takes no
> > effects on what this KIP gonna do, so is it a must?
> >
> > ________________________________
> > 发件人: Guozhang Wang <wa...@gmail.com>
> > 发送时间: 2017年12月6日 15:05
> > 收件人: dev@kafka.apache.org
> > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> > metrics to KafkaConsumer
> >
> > Hello Xi,
> >
> > You can see that in o.a.k.common.metrics.Sensor, we allow constructors to
> > pass in one or more "parent" Sensors of the constructed Sensor, behind
> the
> > scene when a child sensor's metrics have been updated, the updates will
> be
> > propagated all the way up to its parents and ancestors (you can checkout
> > the source code for its impl). On different clients we have been using
> this
> > to build many hierarchical sensors, like per-dest-broker metrics v.s.
> > all-dest-broker metrics on selector, etc.
> >
> > My understanding is that the cross-all-partitions "records-lead-min" will
> > be constructed as the parent of all the per-partition "records-lead-min",
> > is that true?
> >
> >
> > Guozhang
> >
> > On Mon, Dec 4, 2017 at 11:26 PM, Hu Xi <hu...@hotmail.com> wrote:
> >
> > > Guozhang,
> > >
> > >
> > > Thanks for the vote and comments. I am not sure if I fully understand
> the
> > > parent metrics here. This KIP will introduce a client-level metric
> named
> > > 'records-lead-min' and three per-partition metrics tagged with
> > > topic&partition. Is it the child-parent relationship you mean?
> > >
> > >
> > > ________________________________
> > > 发件人: Guozhang Wang <wa...@gmail.com>
> > > 发送时间: 2017年12月5日 15:16
> > > 收件人: dev@kafka.apache.org
> > > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> > > metrics to KafkaConsumer
> > >
> > > Thanks Hu Xi,
> > >
> > > I made a pass over the KIP and it lgtm. +1.
> > >
> > > Just a clarification question: for the cross-partition
> "records-lead-min"
> > > metric, would that be implemented as a parent metric of the
> per-partition
> > > metrics?
> > >
> > >
> > > Guozhang
> > >
> > >
> > > On Mon, Dec 4, 2017 at 3:07 PM, Dong Lin <li...@gmail.com> wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > On Wed, Nov 29, 2017 at 7:05 PM, Hu Xi <hu...@hotmail.com> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > As I didn't see any further discussion around this KIP, I'd like to
> > > start
> > > > > voting.
> > > > >
> > > > > KIP documentation:
> > > > >
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 223+-+Add+per-topic+min+lead+and+per-partition+lead+
> > > > > metrics+to+KafkaConsumer
> > > > >
> > > > >
> > > > >
> > > > > Cheers,
> > > > >
> > > > > huxihx
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
> -- Guozhang
>
[VOTE] KIP-223 - Add per-topic min lead and per-partition lead
metrics to KafkaConsumer
Posted by Hu Xi <hu...@hotmail.com>.
Hi all,
KIP-223 passed with 3 binding votes and 4 non-binding votes:
Binding +1: Jun Rao, Guozhang, Becket
Non-binding +1: Manikumar, Ted Yu, Dong Lin, Charly Molter
Thanks to everyone who shared feedback and voted on this KIP.
huxihx
________________________________
发件人: Guozhang Wang <wa...@gmail.com>
发送时间: 2017年12月11日 12:19
收件人: dev@kafka.apache.org
主题: Re: 答复: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
The by-laws ask for 72 hours only, since the starting of the vote, and
since you have three binding votes you can close this voting now.
Please conclude by a summary of the voting status including non-binding and
binding votes, thanks.
Guozhang
On Sun, Dec 10, 2017 at 8:10 PM, Hu Xi <hu...@hotmail.com> wrote:
> Hi all, Would we safely accept this KIP since three binding votes have
> already been collected (from Jun, Guozhang and Becket)?
>
>
> ________________________________
> 发件人: Guozhang Wang <wa...@gmail.com>
> 发送时间: 2017年12月6日 22:40
> 收件人: dev@kafka.apache.org
> 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> metrics to KafkaConsumer
>
> Hi Hu Xi,
>
> As I said before, it is only a clarification question for its internal
> implementation; it is not related to the public interfaces.
>
>
> Guozhang
>
> On Wed, Dec 6, 2017 at 12:34 AM, Hu Xi <hu...@hotmail.com> wrote:
>
> > Hi Guozhang,
> >
> >
> > Correct me if I am wrong. Of course I could construct the sensor where
> > per-partition metrics reside as a child of the client-level
> > "records-lead-min", but it seems that whether doing that way takes no
> > effects on what this KIP gonna do, so is it a must?
> >
> > ________________________________
> > 发件人: Guozhang Wang <wa...@gmail.com>
> > 发送时间: 2017年12月6日 15:05
> > 收件人: dev@kafka.apache.org
> > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> > metrics to KafkaConsumer
> >
> > Hello Xi,
> >
> > You can see that in o.a.k.common.metrics.Sensor, we allow constructors to
> > pass in one or more "parent" Sensors of the constructed Sensor, behind
> the
> > scene when a child sensor's metrics have been updated, the updates will
> be
> > propagated all the way up to its parents and ancestors (you can checkout
> > the source code for its impl). On different clients we have been using
> this
> > to build many hierarchical sensors, like per-dest-broker metrics v.s.
> > all-dest-broker metrics on selector, etc.
> >
> > My understanding is that the cross-all-partitions "records-lead-min" will
> > be constructed as the parent of all the per-partition "records-lead-min",
> > is that true?
> >
> >
> > Guozhang
> >
> > On Mon, Dec 4, 2017 at 11:26 PM, Hu Xi <hu...@hotmail.com> wrote:
> >
> > > Guozhang,
> > >
> > >
> > > Thanks for the vote and comments. I am not sure if I fully understand
> the
> > > parent metrics here. This KIP will introduce a client-level metric
> named
> > > 'records-lead-min' and three per-partition metrics tagged with
> > > topic&partition. Is it the child-parent relationship you mean?
> > >
> > >
> > > ________________________________
> > > 发件人: Guozhang Wang <wa...@gmail.com>
> > > 发送时间: 2017年12月5日 15:16
> > > 收件人: dev@kafka.apache.org
> > > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> > > metrics to KafkaConsumer
> > >
> > > Thanks Hu Xi,
> > >
> > > I made a pass over the KIP and it lgtm. +1.
> > >
> > > Just a clarification question: for the cross-partition
> "records-lead-min"
> > > metric, would that be implemented as a parent metric of the
> per-partition
> > > metrics?
> > >
> > >
> > > Guozhang
> > >
> > >
> > > On Mon, Dec 4, 2017 at 3:07 PM, Dong Lin <li...@gmail.com> wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > On Wed, Nov 29, 2017 at 7:05 PM, Hu Xi <hu...@hotmail.com> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > As I didn't see any further discussion around this KIP, I'd like to
> > > start
> > > > > voting.
> > > > >
> > > > > KIP documentation:
> > > > >
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 223+-+Add+per-topic+min+lead+and+per-partition+lead+
> > > > > metrics+to+KafkaConsumer
> > > > >
> > > > >
> > > > >
> > > > > Cheers,
> > > > >
> > > > > huxihx
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
> -- Guozhang
>
--
-- Guozhang
答复: 答复: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
Posted by Hu Xi <hu...@hotmail.com>.
Hi Mickael Maison,
Thanks for the comments, but I think it deserves another KIP since this one mainly focuses on metrics things. The way to calculate the lag in ConsumerGroupCommand is seemingly different from what metrics do, so a new KIP might be required. What do you think?
huxihx
________________________________
发件人: Mickael Maison <mi...@gmail.com>
发送时间: 2017年12月11日 18:17
收件人: dev@kafka.apache.org
主题: Re: 答复: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
Sorry to only raise this now, but should we also update the
kafka-consumer-groups tool to display the start offset (and possibly
the lead) ?
Apart from that I'm +1 (non binding)
Thanks
On Mon, Dec 11, 2017 at 4:19 AM, Guozhang Wang <wa...@gmail.com> wrote:
> The by-laws ask for 72 hours only, since the starting of the vote, and
> since you have three binding votes you can close this voting now.
>
> Please conclude by a summary of the voting status including non-binding and
> binding votes, thanks.
>
>
> Guozhang
>
> On Sun, Dec 10, 2017 at 8:10 PM, Hu Xi <hu...@hotmail.com> wrote:
>
>> Hi all, Would we safely accept this KIP since three binding votes have
>> already been collected (from Jun, Guozhang and Becket)?
>>
>>
>> ________________________________
>> 发件人: Guozhang Wang <wa...@gmail.com>
>> 发送时间: 2017年12月6日 22:40
>> 收件人: dev@kafka.apache.org
>> 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
>> metrics to KafkaConsumer
>>
>> Hi Hu Xi,
>>
>> As I said before, it is only a clarification question for its internal
>> implementation; it is not related to the public interfaces.
>>
>>
>> Guozhang
>>
>> On Wed, Dec 6, 2017 at 12:34 AM, Hu Xi <hu...@hotmail.com> wrote:
>>
>> > Hi Guozhang,
>> >
>> >
>> > Correct me if I am wrong. Of course I could construct the sensor where
>> > per-partition metrics reside as a child of the client-level
>> > "records-lead-min", but it seems that whether doing that way takes no
>> > effects on what this KIP gonna do, so is it a must?
>> >
>> > ________________________________
>> > 发件人: Guozhang Wang <wa...@gmail.com>
>> > 发送时间: 2017年12月6日 15:05
>> > 收件人: dev@kafka.apache.org
>> > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
>> > metrics to KafkaConsumer
>> >
>> > Hello Xi,
>> >
>> > You can see that in o.a.k.common.metrics.Sensor, we allow constructors to
>> > pass in one or more "parent" Sensors of the constructed Sensor, behind
>> the
>> > scene when a child sensor's metrics have been updated, the updates will
>> be
>> > propagated all the way up to its parents and ancestors (you can checkout
>> > the source code for its impl). On different clients we have been using
>> this
>> > to build many hierarchical sensors, like per-dest-broker metrics v.s.
>> > all-dest-broker metrics on selector, etc.
>> >
>> > My understanding is that the cross-all-partitions "records-lead-min" will
>> > be constructed as the parent of all the per-partition "records-lead-min",
>> > is that true?
>> >
>> >
>> > Guozhang
>> >
>> > On Mon, Dec 4, 2017 at 11:26 PM, Hu Xi <hu...@hotmail.com> wrote:
>> >
>> > > Guozhang,
>> > >
>> > >
>> > > Thanks for the vote and comments. I am not sure if I fully understand
>> the
>> > > parent metrics here. This KIP will introduce a client-level metric
>> named
>> > > 'records-lead-min' and three per-partition metrics tagged with
>> > > topic&partition. Is it the child-parent relationship you mean?
>> > >
>> > >
>> > > ________________________________
>> > > 发件人: Guozhang Wang <wa...@gmail.com>
>> > > 发送时间: 2017年12月5日 15:16
>> > > 收件人: dev@kafka.apache.org
>> > > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
>> > > metrics to KafkaConsumer
>> > >
>> > > Thanks Hu Xi,
>> > >
>> > > I made a pass over the KIP and it lgtm. +1.
>> > >
>> > > Just a clarification question: for the cross-partition
>> "records-lead-min"
>> > > metric, would that be implemented as a parent metric of the
>> per-partition
>> > > metrics?
>> > >
>> > >
>> > > Guozhang
>> > >
>> > >
>> > > On Mon, Dec 4, 2017 at 3:07 PM, Dong Lin <li...@gmail.com> wrote:
>> > >
>> > > > +1 (non-binding)
>> > > >
>> > > > On Wed, Nov 29, 2017 at 7:05 PM, Hu Xi <hu...@hotmail.com> wrote:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > As I didn't see any further discussion around this KIP, I'd like to
>> > > start
>> > > > > voting.
>> > > > >
>> > > > > KIP documentation:
>> > > > >
>> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > > > > 223+-+Add+per-topic+min+lead+and+per-partition+lead+
>> > > > > metrics+to+KafkaConsumer
>> > > > >
>> > > > >
>> > > > >
>> > > > > Cheers,
>> > > > >
>> > > > > huxihx
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > -- Guozhang
>> > >
>> >
>> >
>> >
>> > --
>> > -- Guozhang
>> >
>>
>>
>>
>> --
>> -- Guozhang
>>
>
>
>
> --
> -- Guozhang
Re: 答复: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
Posted by Mickael Maison <mi...@gmail.com>.
Sorry to only raise this now, but should we also update the
kafka-consumer-groups tool to display the start offset (and possibly
the lead) ?
Apart from that I'm +1 (non binding)
Thanks
On Mon, Dec 11, 2017 at 4:19 AM, Guozhang Wang <wa...@gmail.com> wrote:
> The by-laws ask for 72 hours only, since the starting of the vote, and
> since you have three binding votes you can close this voting now.
>
> Please conclude by a summary of the voting status including non-binding and
> binding votes, thanks.
>
>
> Guozhang
>
> On Sun, Dec 10, 2017 at 8:10 PM, Hu Xi <hu...@hotmail.com> wrote:
>
>> Hi all, Would we safely accept this KIP since three binding votes have
>> already been collected (from Jun, Guozhang and Becket)?
>>
>>
>> ________________________________
>> 发件人: Guozhang Wang <wa...@gmail.com>
>> 发送时间: 2017年12月6日 22:40
>> 收件人: dev@kafka.apache.org
>> 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
>> metrics to KafkaConsumer
>>
>> Hi Hu Xi,
>>
>> As I said before, it is only a clarification question for its internal
>> implementation; it is not related to the public interfaces.
>>
>>
>> Guozhang
>>
>> On Wed, Dec 6, 2017 at 12:34 AM, Hu Xi <hu...@hotmail.com> wrote:
>>
>> > Hi Guozhang,
>> >
>> >
>> > Correct me if I am wrong. Of course I could construct the sensor where
>> > per-partition metrics reside as a child of the client-level
>> > "records-lead-min", but it seems that whether doing that way takes no
>> > effects on what this KIP gonna do, so is it a must?
>> >
>> > ________________________________
>> > 发件人: Guozhang Wang <wa...@gmail.com>
>> > 发送时间: 2017年12月6日 15:05
>> > 收件人: dev@kafka.apache.org
>> > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
>> > metrics to KafkaConsumer
>> >
>> > Hello Xi,
>> >
>> > You can see that in o.a.k.common.metrics.Sensor, we allow constructors to
>> > pass in one or more "parent" Sensors of the constructed Sensor, behind
>> the
>> > scene when a child sensor's metrics have been updated, the updates will
>> be
>> > propagated all the way up to its parents and ancestors (you can checkout
>> > the source code for its impl). On different clients we have been using
>> this
>> > to build many hierarchical sensors, like per-dest-broker metrics v.s.
>> > all-dest-broker metrics on selector, etc.
>> >
>> > My understanding is that the cross-all-partitions "records-lead-min" will
>> > be constructed as the parent of all the per-partition "records-lead-min",
>> > is that true?
>> >
>> >
>> > Guozhang
>> >
>> > On Mon, Dec 4, 2017 at 11:26 PM, Hu Xi <hu...@hotmail.com> wrote:
>> >
>> > > Guozhang,
>> > >
>> > >
>> > > Thanks for the vote and comments. I am not sure if I fully understand
>> the
>> > > parent metrics here. This KIP will introduce a client-level metric
>> named
>> > > 'records-lead-min' and three per-partition metrics tagged with
>> > > topic&partition. Is it the child-parent relationship you mean?
>> > >
>> > >
>> > > ________________________________
>> > > 发件人: Guozhang Wang <wa...@gmail.com>
>> > > 发送时间: 2017年12月5日 15:16
>> > > 收件人: dev@kafka.apache.org
>> > > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
>> > > metrics to KafkaConsumer
>> > >
>> > > Thanks Hu Xi,
>> > >
>> > > I made a pass over the KIP and it lgtm. +1.
>> > >
>> > > Just a clarification question: for the cross-partition
>> "records-lead-min"
>> > > metric, would that be implemented as a parent metric of the
>> per-partition
>> > > metrics?
>> > >
>> > >
>> > > Guozhang
>> > >
>> > >
>> > > On Mon, Dec 4, 2017 at 3:07 PM, Dong Lin <li...@gmail.com> wrote:
>> > >
>> > > > +1 (non-binding)
>> > > >
>> > > > On Wed, Nov 29, 2017 at 7:05 PM, Hu Xi <hu...@hotmail.com> wrote:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > As I didn't see any further discussion around this KIP, I'd like to
>> > > start
>> > > > > voting.
>> > > > >
>> > > > > KIP documentation:
>> > > > >
>> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > > > > 223+-+Add+per-topic+min+lead+and+per-partition+lead+
>> > > > > metrics+to+KafkaConsumer
>> > > > >
>> > > > >
>> > > > >
>> > > > > Cheers,
>> > > > >
>> > > > > huxihx
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > -- Guozhang
>> > >
>> >
>> >
>> >
>> > --
>> > -- Guozhang
>> >
>>
>>
>>
>> --
>> -- Guozhang
>>
>
>
>
> --
> -- Guozhang
Re: 答复: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
Posted by Guozhang Wang <wa...@gmail.com>.
The by-laws ask for 72 hours only, since the starting of the vote, and
since you have three binding votes you can close this voting now.
Please conclude by a summary of the voting status including non-binding and
binding votes, thanks.
Guozhang
On Sun, Dec 10, 2017 at 8:10 PM, Hu Xi <hu...@hotmail.com> wrote:
> Hi all, Would we safely accept this KIP since three binding votes have
> already been collected (from Jun, Guozhang and Becket)?
>
>
> ________________________________
> 发件人: Guozhang Wang <wa...@gmail.com>
> 发送时间: 2017年12月6日 22:40
> 收件人: dev@kafka.apache.org
> 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> metrics to KafkaConsumer
>
> Hi Hu Xi,
>
> As I said before, it is only a clarification question for its internal
> implementation; it is not related to the public interfaces.
>
>
> Guozhang
>
> On Wed, Dec 6, 2017 at 12:34 AM, Hu Xi <hu...@hotmail.com> wrote:
>
> > Hi Guozhang,
> >
> >
> > Correct me if I am wrong. Of course I could construct the sensor where
> > per-partition metrics reside as a child of the client-level
> > "records-lead-min", but it seems that whether doing that way takes no
> > effects on what this KIP gonna do, so is it a must?
> >
> > ________________________________
> > 发件人: Guozhang Wang <wa...@gmail.com>
> > 发送时间: 2017年12月6日 15:05
> > 收件人: dev@kafka.apache.org
> > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> > metrics to KafkaConsumer
> >
> > Hello Xi,
> >
> > You can see that in o.a.k.common.metrics.Sensor, we allow constructors to
> > pass in one or more "parent" Sensors of the constructed Sensor, behind
> the
> > scene when a child sensor's metrics have been updated, the updates will
> be
> > propagated all the way up to its parents and ancestors (you can checkout
> > the source code for its impl). On different clients we have been using
> this
> > to build many hierarchical sensors, like per-dest-broker metrics v.s.
> > all-dest-broker metrics on selector, etc.
> >
> > My understanding is that the cross-all-partitions "records-lead-min" will
> > be constructed as the parent of all the per-partition "records-lead-min",
> > is that true?
> >
> >
> > Guozhang
> >
> > On Mon, Dec 4, 2017 at 11:26 PM, Hu Xi <hu...@hotmail.com> wrote:
> >
> > > Guozhang,
> > >
> > >
> > > Thanks for the vote and comments. I am not sure if I fully understand
> the
> > > parent metrics here. This KIP will introduce a client-level metric
> named
> > > 'records-lead-min' and three per-partition metrics tagged with
> > > topic&partition. Is it the child-parent relationship you mean?
> > >
> > >
> > > ________________________________
> > > 发件人: Guozhang Wang <wa...@gmail.com>
> > > 发送时间: 2017年12月5日 15:16
> > > 收件人: dev@kafka.apache.org
> > > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> > > metrics to KafkaConsumer
> > >
> > > Thanks Hu Xi,
> > >
> > > I made a pass over the KIP and it lgtm. +1.
> > >
> > > Just a clarification question: for the cross-partition
> "records-lead-min"
> > > metric, would that be implemented as a parent metric of the
> per-partition
> > > metrics?
> > >
> > >
> > > Guozhang
> > >
> > >
> > > On Mon, Dec 4, 2017 at 3:07 PM, Dong Lin <li...@gmail.com> wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > On Wed, Nov 29, 2017 at 7:05 PM, Hu Xi <hu...@hotmail.com> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > As I didn't see any further discussion around this KIP, I'd like to
> > > start
> > > > > voting.
> > > > >
> > > > > KIP documentation:
> > > > >
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 223+-+Add+per-topic+min+lead+and+per-partition+lead+
> > > > > metrics+to+KafkaConsumer
> > > > >
> > > > >
> > > > >
> > > > > Cheers,
> > > > >
> > > > > huxihx
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
> -- Guozhang
>
--
-- Guozhang
答复: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
Posted by Hu Xi <hu...@hotmail.com>.
Hi all, Would we safely accept this KIP since three binding votes have already been collected (from Jun, Guozhang and Becket)?
________________________________
发件人: Guozhang Wang <wa...@gmail.com>
发送时间: 2017年12月6日 22:40
收件人: dev@kafka.apache.org
主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
Hi Hu Xi,
As I said before, it is only a clarification question for its internal
implementation; it is not related to the public interfaces.
Guozhang
On Wed, Dec 6, 2017 at 12:34 AM, Hu Xi <hu...@hotmail.com> wrote:
> Hi Guozhang,
>
>
> Correct me if I am wrong. Of course I could construct the sensor where
> per-partition metrics reside as a child of the client-level
> "records-lead-min", but it seems that whether doing that way takes no
> effects on what this KIP gonna do, so is it a must?
>
> ________________________________
> 发件人: Guozhang Wang <wa...@gmail.com>
> 发送时间: 2017年12月6日 15:05
> 收件人: dev@kafka.apache.org
> 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> metrics to KafkaConsumer
>
> Hello Xi,
>
> You can see that in o.a.k.common.metrics.Sensor, we allow constructors to
> pass in one or more "parent" Sensors of the constructed Sensor, behind the
> scene when a child sensor's metrics have been updated, the updates will be
> propagated all the way up to its parents and ancestors (you can checkout
> the source code for its impl). On different clients we have been using this
> to build many hierarchical sensors, like per-dest-broker metrics v.s.
> all-dest-broker metrics on selector, etc.
>
> My understanding is that the cross-all-partitions "records-lead-min" will
> be constructed as the parent of all the per-partition "records-lead-min",
> is that true?
>
>
> Guozhang
>
> On Mon, Dec 4, 2017 at 11:26 PM, Hu Xi <hu...@hotmail.com> wrote:
>
> > Guozhang,
> >
> >
> > Thanks for the vote and comments. I am not sure if I fully understand the
> > parent metrics here. This KIP will introduce a client-level metric named
> > 'records-lead-min' and three per-partition metrics tagged with
> > topic&partition. Is it the child-parent relationship you mean?
> >
> >
> > ________________________________
> > 发件人: Guozhang Wang <wa...@gmail.com>
> > 发送时间: 2017年12月5日 15:16
> > 收件人: dev@kafka.apache.org
> > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> > metrics to KafkaConsumer
> >
> > Thanks Hu Xi,
> >
> > I made a pass over the KIP and it lgtm. +1.
> >
> > Just a clarification question: for the cross-partition "records-lead-min"
> > metric, would that be implemented as a parent metric of the per-partition
> > metrics?
> >
> >
> > Guozhang
> >
> >
> > On Mon, Dec 4, 2017 at 3:07 PM, Dong Lin <li...@gmail.com> wrote:
> >
> > > +1 (non-binding)
> > >
> > > On Wed, Nov 29, 2017 at 7:05 PM, Hu Xi <hu...@hotmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > As I didn't see any further discussion around this KIP, I'd like to
> > start
> > > > voting.
> > > >
> > > > KIP documentation:
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 223+-+Add+per-topic+min+lead+and+per-partition+lead+
> > > > metrics+to+KafkaConsumer
> > > >
> > > >
> > > >
> > > > Cheers,
> > > >
> > > > huxihx
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
> -- Guozhang
>
--
-- Guozhang
Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
metrics to KafkaConsumer
Posted by Guozhang Wang <wa...@gmail.com>.
Hi Hu Xi,
As I said before, it is only a clarification question for its internal
implementation; it is not related to the public interfaces.
Guozhang
On Wed, Dec 6, 2017 at 12:34 AM, Hu Xi <hu...@hotmail.com> wrote:
> Hi Guozhang,
>
>
> Correct me if I am wrong. Of course I could construct the sensor where
> per-partition metrics reside as a child of the client-level
> "records-lead-min", but it seems that whether doing that way takes no
> effects on what this KIP gonna do, so is it a must?
>
> ________________________________
> 发件人: Guozhang Wang <wa...@gmail.com>
> 发送时间: 2017年12月6日 15:05
> 收件人: dev@kafka.apache.org
> 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> metrics to KafkaConsumer
>
> Hello Xi,
>
> You can see that in o.a.k.common.metrics.Sensor, we allow constructors to
> pass in one or more "parent" Sensors of the constructed Sensor, behind the
> scene when a child sensor's metrics have been updated, the updates will be
> propagated all the way up to its parents and ancestors (you can checkout
> the source code for its impl). On different clients we have been using this
> to build many hierarchical sensors, like per-dest-broker metrics v.s.
> all-dest-broker metrics on selector, etc.
>
> My understanding is that the cross-all-partitions "records-lead-min" will
> be constructed as the parent of all the per-partition "records-lead-min",
> is that true?
>
>
> Guozhang
>
> On Mon, Dec 4, 2017 at 11:26 PM, Hu Xi <hu...@hotmail.com> wrote:
>
> > Guozhang,
> >
> >
> > Thanks for the vote and comments. I am not sure if I fully understand the
> > parent metrics here. This KIP will introduce a client-level metric named
> > 'records-lead-min' and three per-partition metrics tagged with
> > topic&partition. Is it the child-parent relationship you mean?
> >
> >
> > ________________________________
> > 发件人: Guozhang Wang <wa...@gmail.com>
> > 发送时间: 2017年12月5日 15:16
> > 收件人: dev@kafka.apache.org
> > 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> > metrics to KafkaConsumer
> >
> > Thanks Hu Xi,
> >
> > I made a pass over the KIP and it lgtm. +1.
> >
> > Just a clarification question: for the cross-partition "records-lead-min"
> > metric, would that be implemented as a parent metric of the per-partition
> > metrics?
> >
> >
> > Guozhang
> >
> >
> > On Mon, Dec 4, 2017 at 3:07 PM, Dong Lin <li...@gmail.com> wrote:
> >
> > > +1 (non-binding)
> > >
> > > On Wed, Nov 29, 2017 at 7:05 PM, Hu Xi <hu...@hotmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > As I didn't see any further discussion around this KIP, I'd like to
> > start
> > > > voting.
> > > >
> > > > KIP documentation:
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 223+-+Add+per-topic+min+lead+and+per-partition+lead+
> > > > metrics+to+KafkaConsumer
> > > >
> > > >
> > > >
> > > > Cheers,
> > > >
> > > > huxihx
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
> -- Guozhang
>
--
-- Guozhang
[VOTE] KIP-223 - Add per-topic min lead and per-partition lead
metrics to KafkaConsumer
Posted by Hu Xi <hu...@hotmail.com>.
Hi Guozhang,
Correct me if I am wrong. Of course I could construct the sensor where per-partition metrics reside as a child of the client-level "records-lead-min", but it seems that whether doing that way takes no effects on what this KIP gonna do, so is it a must?
________________________________
发件人: Guozhang Wang <wa...@gmail.com>
发送时间: 2017年12月6日 15:05
收件人: dev@kafka.apache.org
主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
Hello Xi,
You can see that in o.a.k.common.metrics.Sensor, we allow constructors to
pass in one or more "parent" Sensors of the constructed Sensor, behind the
scene when a child sensor's metrics have been updated, the updates will be
propagated all the way up to its parents and ancestors (you can checkout
the source code for its impl). On different clients we have been using this
to build many hierarchical sensors, like per-dest-broker metrics v.s.
all-dest-broker metrics on selector, etc.
My understanding is that the cross-all-partitions "records-lead-min" will
be constructed as the parent of all the per-partition "records-lead-min",
is that true?
Guozhang
On Mon, Dec 4, 2017 at 11:26 PM, Hu Xi <hu...@hotmail.com> wrote:
> Guozhang,
>
>
> Thanks for the vote and comments. I am not sure if I fully understand the
> parent metrics here. This KIP will introduce a client-level metric named
> 'records-lead-min' and three per-partition metrics tagged with
> topic&partition. Is it the child-parent relationship you mean?
>
>
> ________________________________
> 发件人: Guozhang Wang <wa...@gmail.com>
> 发送时间: 2017年12月5日 15:16
> 收件人: dev@kafka.apache.org
> 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> metrics to KafkaConsumer
>
> Thanks Hu Xi,
>
> I made a pass over the KIP and it lgtm. +1.
>
> Just a clarification question: for the cross-partition "records-lead-min"
> metric, would that be implemented as a parent metric of the per-partition
> metrics?
>
>
> Guozhang
>
>
> On Mon, Dec 4, 2017 at 3:07 PM, Dong Lin <li...@gmail.com> wrote:
>
> > +1 (non-binding)
> >
> > On Wed, Nov 29, 2017 at 7:05 PM, Hu Xi <hu...@hotmail.com> wrote:
> >
> > > Hi all,
> > >
> > > As I didn't see any further discussion around this KIP, I'd like to
> start
> > > voting.
> > >
> > > KIP documentation:
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 223+-+Add+per-topic+min+lead+and+per-partition+lead+
> > > metrics+to+KafkaConsumer
> > >
> > >
> > >
> > > Cheers,
> > >
> > > huxihx
> > >
> >
>
>
>
> --
> -- Guozhang
>
--
-- Guozhang
Re: 答复: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead metrics to KafkaConsumer
Posted by Guozhang Wang <wa...@gmail.com>.
Hello Xi,
You can see that in o.a.k.common.metrics.Sensor, we allow constructors to
pass in one or more "parent" Sensors of the constructed Sensor, behind the
scene when a child sensor's metrics have been updated, the updates will be
propagated all the way up to its parents and ancestors (you can checkout
the source code for its impl). On different clients we have been using this
to build many hierarchical sensors, like per-dest-broker metrics v.s.
all-dest-broker metrics on selector, etc.
My understanding is that the cross-all-partitions "records-lead-min" will
be constructed as the parent of all the per-partition "records-lead-min",
is that true?
Guozhang
On Mon, Dec 4, 2017 at 11:26 PM, Hu Xi <hu...@hotmail.com> wrote:
> Guozhang,
>
>
> Thanks for the vote and comments. I am not sure if I fully understand the
> parent metrics here. This KIP will introduce a client-level metric named
> 'records-lead-min' and three per-partition metrics tagged with
> topic&partition. Is it the child-parent relationship you mean?
>
>
> ________________________________
> 发件人: Guozhang Wang <wa...@gmail.com>
> 发送时间: 2017年12月5日 15:16
> 收件人: dev@kafka.apache.org
> 主题: Re: [VOTE] KIP-223 - Add per-topic min lead and per-partition lead
> metrics to KafkaConsumer
>
> Thanks Hu Xi,
>
> I made a pass over the KIP and it lgtm. +1.
>
> Just a clarification question: for the cross-partition "records-lead-min"
> metric, would that be implemented as a parent metric of the per-partition
> metrics?
>
>
> Guozhang
>
>
> On Mon, Dec 4, 2017 at 3:07 PM, Dong Lin <li...@gmail.com> wrote:
>
> > +1 (non-binding)
> >
> > On Wed, Nov 29, 2017 at 7:05 PM, Hu Xi <hu...@hotmail.com> wrote:
> >
> > > Hi all,
> > >
> > > As I didn't see any further discussion around this KIP, I'd like to
> start
> > > voting.
> > >
> > > KIP documentation:
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 223+-+Add+per-topic+min+lead+and+per-partition+lead+
> > > metrics+to+KafkaConsumer
> > >
> > >
> > >
> > > Cheers,
> > >
> > > huxihx
> > >
> >
>
>
>
> --
> -- Guozhang
>
--
-- Guozhang