You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Thomas Becker <Th...@tivo.com> on 2020/01/23 21:17:27 UTC
[DISCUSS] KIP-566: Add rebalance callbacks to ConsumerInterceptor
Hi folks,
I'd like to open the discussion for KIP-566: Add rebalance callbacks to ConsumerInterceptor. We've been looking to implement some custom metrics via ConsumerInterceptor, and not knowing when partition ownership changes is a significant impediment. I'd appreciate your thoughts.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-566%3A+Add+rebalance+callbacks+to+ConsumerInterceptor
________________________________
This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo is authorized to conclude any binding agreement on behalf of TiVo by email. Binding agreements with TiVo may only be made by a signed written agreement.
Re: [DISCUSS] KIP-566: Add rebalance callbacks to ConsumerInterceptor
Posted by Thomas Becker <Th...@tivo.com>.
Bumping to get and get some attention on this KIP before initiating a vote. Using ConsumerInterceptor for its intended purpose quite difficult without this.
On Mon, 2020-02-10 at 15:50 +0000, Thomas Becker wrote:
[EXTERNAL EMAIL] Attention: This email was sent from outside TiVo. DO NOT CLICK any links or attachments unless you expected them.
Bumping this again for visibility. If no one has any comments, maybe I'll just start the VOTE thread?
On Wed, 2020-01-29 at 22:24 +0000, Thomas Becker wrote:
[EXTERNAL EMAIL] Attention: This email was sent from outside TiVo. DO NOT CLICK any links or attachments unless you expected them.
________________________________
Bumping this, hoping to get some additional feedback.
________________________________
From: M. Manna <ma...@gmail.com>>
Sent: Thursday, January 23, 2020 4:37 PM
To: dev@kafka.apache.org<ma...@kafka.apache.org> <de...@kafka.apache.org>>
Subject: Re: [DISCUSS] KIP-566: Add rebalance callbacks to ConsumerInterceptor
[EXTERNAL EMAIL] Attention: This email was sent from outside TiVo. DO NOT CLICK any links or attachments unless you expected them.
________________________________
Hey Thomas,
On Thu, 23 Jan 2020 at 21:17, Thomas Becker <Th...@tivo.com>> wrote:
Hi folks,
I'd like to open the discussion for KIP-566: Add rebalance callbacks to
ConsumerInterceptor. We've been looking to implement some custom metrics
via ConsumerInterceptor, and not knowing when partition ownership changes
is a significant impediment. I'd appreciate your thoughts.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-566%253A%2BAdd%2Brebalance%2Bcallbacks%2Bto%2BConsumerInterceptor&data=02%7C01%7CThomas.Becker%40tivo.com%7C660a1b4a8ba94d668c7708d7a50a10d3%7Cd05b7c6912014c0db45d7f1dcc227e4d%7C1%7C0%7C637159334967045569&sdata=RYUcrnvDWady9%2FAnLSvc2vvLlchIbj6w0og8Vxe1KN4%3D&reserved=0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-566%253A%2BAdd%2Brebalance%2Bcallbacks%2Bto%2BConsumerInterceptor&data=02%7C01%7CThomas.Becker%40tivo.com%7C4f1be50e5e1140dee63808d7ae40f935%7Cd05b7c6912014c0db45d7f1dcc227e4d%7C1%7C0%7C637169466398399057&sdata=M%2BzYH0%2FGii4ES8QqqEpZqN7n4YVinXvxjcIrAkuOi2o%3D&reserved=0>
I had a quick read through the KIP. I don't see any obvious issues.
Sounds like a simple improvement. Perhaps, you could add your thoughts
about RebalanceListener API in the future e.g. when to unify the
functionality. If implemented, we can simply use one implementation for
both things.
I would be interested to hear others' comments about this.
Thanks,
________________________________
This email and any attachments may contain confidential and privileged
material for the sole use of the intended recipient. Any review, copying,
or distribution of this email (or any attachments) by others is prohibited.
If you are not the intended recipient, please contact the sender
immediately and permanently delete this email and any attachments. No
employee or agent of TiVo is authorized to conclude any binding agreement
on behalf of TiVo by email. Binding agreements with TiVo may only be made
by a signed written agreement.
________________________________
This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo is authorized to conclude any binding agreement on behalf of TiVo by email. Binding agreements with TiVo may only be made by a signed written agreement.
--
[cid:9c5eb4be88341189d58cf171e402c865e96c80a8.camel@tivo.com] Tommy Becker
Principal Engineer
Personalized Content Discovery
O +1 919.460.4747
tivo.com<http://www.tivo.com/>
________________________________
This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo is authorized to conclude any binding agreement on behalf of TiVo by email. Binding agreements with TiVo may only be made by a signed written agreement.
--
[cid:ab14b394da4ece62fa660a6e6637a4a33ad36ba4.camel@tivo.com] Tommy Becker
Principal Engineer
Personalized Content Discovery
O +1 919.460.4747
tivo.com<http://www.tivo.com/>
________________________________
This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo is authorized to conclude any binding agreement on behalf of TiVo by email. Binding agreements with TiVo may only be made by a signed written agreement.
Re: [DISCUSS] KIP-566: Add rebalance callbacks to ConsumerInterceptor
Posted by Thomas Becker <Th...@tivo.com>.
Bumping this again for visibility. If no one has any comments, maybe I'll just start the VOTE thread?
On Wed, 2020-01-29 at 22:24 +0000, Thomas Becker wrote:
[EXTERNAL EMAIL] Attention: This email was sent from outside TiVo. DO NOT CLICK any links or attachments unless you expected them.
________________________________
Bumping this, hoping to get some additional feedback.
________________________________
From: M. Manna <ma...@gmail.com>>
Sent: Thursday, January 23, 2020 4:37 PM
To: dev@kafka.apache.org<ma...@kafka.apache.org> <de...@kafka.apache.org>>
Subject: Re: [DISCUSS] KIP-566: Add rebalance callbacks to ConsumerInterceptor
[EXTERNAL EMAIL] Attention: This email was sent from outside TiVo. DO NOT CLICK any links or attachments unless you expected them.
________________________________
Hey Thomas,
On Thu, 23 Jan 2020 at 21:17, Thomas Becker <Th...@tivo.com>> wrote:
Hi folks,
I'd like to open the discussion for KIP-566: Add rebalance callbacks to
ConsumerInterceptor. We've been looking to implement some custom metrics
via ConsumerInterceptor, and not knowing when partition ownership changes
is a significant impediment. I'd appreciate your thoughts.
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-566%253A%2BAdd%2Brebalance%2Bcallbacks%2Bto%2BConsumerInterceptor&data=02%7C01%7CThomas.Becker%40tivo.com%7C660a1b4a8ba94d668c7708d7a50a10d3%7Cd05b7c6912014c0db45d7f1dcc227e4d%7C1%7C0%7C637159334967045569&sdata=RYUcrnvDWady9%2FAnLSvc2vvLlchIbj6w0og8Vxe1KN4%3D&reserved=0
I had a quick read through the KIP. I don't see any obvious issues.
Sounds like a simple improvement. Perhaps, you could add your thoughts
about RebalanceListener API in the future e.g. when to unify the
functionality. If implemented, we can simply use one implementation for
both things.
I would be interested to hear others' comments about this.
Thanks,
________________________________
This email and any attachments may contain confidential and privileged
material for the sole use of the intended recipient. Any review, copying,
or distribution of this email (or any attachments) by others is prohibited.
If you are not the intended recipient, please contact the sender
immediately and permanently delete this email and any attachments. No
employee or agent of TiVo is authorized to conclude any binding agreement
on behalf of TiVo by email. Binding agreements with TiVo may only be made
by a signed written agreement.
________________________________
This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo is authorized to conclude any binding agreement on behalf of TiVo by email. Binding agreements with TiVo may only be made by a signed written agreement.
--
[cid:9c5eb4be88341189d58cf171e402c865e96c80a8.camel@tivo.com] Tommy Becker
Principal Engineer
Personalized Content Discovery
O +1 919.460.4747
tivo.com<http://www.tivo.com/>
________________________________
This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo is authorized to conclude any binding agreement on behalf of TiVo by email. Binding agreements with TiVo may only be made by a signed written agreement.
Re: [DISCUSS] KIP-566: Add rebalance callbacks to ConsumerInterceptor
Posted by Thomas Becker <Th...@tivo.com>.
Bumping this, hoping to get some additional feedback.
________________________________
From: M. Manna <ma...@gmail.com>
Sent: Thursday, January 23, 2020 4:37 PM
To: dev@kafka.apache.org <de...@kafka.apache.org>
Subject: Re: [DISCUSS] KIP-566: Add rebalance callbacks to ConsumerInterceptor
[EXTERNAL EMAIL] Attention: This email was sent from outside TiVo. DO NOT CLICK any links or attachments unless you expected them.
________________________________
Hey Thomas,
On Thu, 23 Jan 2020 at 21:17, Thomas Becker <Th...@tivo.com> wrote:
> Hi folks,
> I'd like to open the discussion for KIP-566: Add rebalance callbacks to
> ConsumerInterceptor. We've been looking to implement some custom metrics
> via ConsumerInterceptor, and not knowing when partition ownership changes
> is a significant impediment. I'd appreciate your thoughts.
>
>
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-566%253A%2BAdd%2Brebalance%2Bcallbacks%2Bto%2BConsumerInterceptor&data=02%7C01%7CThomas.Becker%40tivo.com%7C5aa11370b535485ec09a08d7a04c7e67%7Cd05b7c6912014c0db45d7f1dcc227e4d%7C1%7C0%7C637154122715479355&sdata=xcWznGTiCv8CvtEeakFYvODghf5m7Dr1HOyZcO4RUGE%3D&reserved=0
>
>
>
> I had a quick read through the KIP. I don't see any obvious issues.
Sounds like a simple improvement. Perhaps, you could add your thoughts
about RebalanceListener API in the future e.g. when to unify the
functionality. If implemented, we can simply use one implementation for
both things.
I would be interested to hear others' comments about this.
Thanks,
>
> ________________________________
>
> This email and any attachments may contain confidential and privileged
> material for the sole use of the intended recipient. Any review, copying,
> or distribution of this email (or any attachments) by others is prohibited.
> If you are not the intended recipient, please contact the sender
> immediately and permanently delete this email and any attachments. No
> employee or agent of TiVo is authorized to conclude any binding agreement
> on behalf of TiVo by email. Binding agreements with TiVo may only be made
> by a signed written agreement.
>
________________________________
This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo is authorized to conclude any binding agreement on behalf of TiVo by email. Binding agreements with TiVo may only be made by a signed written agreement.
Re: [DISCUSS] KIP-566: Add rebalance callbacks to ConsumerInterceptor
Posted by "M. Manna" <ma...@gmail.com>.
Hey Thomas,
On Thu, 23 Jan 2020 at 21:17, Thomas Becker <Th...@tivo.com> wrote:
> Hi folks,
> I'd like to open the discussion for KIP-566: Add rebalance callbacks to
> ConsumerInterceptor. We've been looking to implement some custom metrics
> via ConsumerInterceptor, and not knowing when partition ownership changes
> is a significant impediment. I'd appreciate your thoughts.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-566%3A+Add+rebalance+callbacks+to+ConsumerInterceptor
>
>
>
> I had a quick read through the KIP. I don't see any obvious issues.
Sounds like a simple improvement. Perhaps, you could add your thoughts
about RebalanceListener API in the future e.g. when to unify the
functionality. If implemented, we can simply use one implementation for
both things.
I would be interested to hear others' comments about this.
Thanks,
>
> ________________________________
>
> This email and any attachments may contain confidential and privileged
> material for the sole use of the intended recipient. Any review, copying,
> or distribution of this email (or any attachments) by others is prohibited.
> If you are not the intended recipient, please contact the sender
> immediately and permanently delete this email and any attachments. No
> employee or agent of TiVo is authorized to conclude any binding agreement
> on behalf of TiVo by email. Binding agreements with TiVo may only be made
> by a signed written agreement.
>