You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Hector Geraldino (BLOOMBERG/ 919 3RD A)" <hg...@bloomberg.net> on 2024/07/08 18:40:16 UTC
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
This will help eliminate some boilerplate code we have for our connectors.
+1 (non-binding)
From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To: dev@kafka.apache.org
Subject: Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Bumping this thread.
Let me know if you have any feedback.
Thanks,
Mickael
On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <fe...@gmail.com> wrote:
>
> +1 (binding), thanks Mickael!
>
> On Mon, Jun 10, 2024, 04:24 Mickael Maison <mi...@gmail.com> wrote:
>
> > Hi,
> >
> > Following the feedback in the DISCUSS thread, I made significant
> > changes to the proposal. So I'd like to restart a vote for KIP-877:
> >
> >
https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
ns+and+connectors+to+register+metrics
> >
> > Thanks,
> > Mickael
> >
> > On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <tb...@redhat.com> wrote:
> > >
> > > Hi Mickael,
> > >
> > > You'll have seen that I left some comments on the discussion thread, but
> > > they're minor enough that I'm happy to vote +1 here.
> > >
> > > Thanks,
> > >
> > > Tom
> > >
> > > On Thu, 11 Jan 2024 at 06:14, Mickael Maison <mi...@gmail.com>
> > > wrote:
> > >
> > > > Bumping this thread since I've not seen any feedback.
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
> > > > On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> > > > <mi...@gmail.com> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I'd like to start a vote on KIP-877:
> > > > >
> > > >
> >
https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
ns+and+connectors+to+register+metrics
> > > > >
> > > > > Let me know if you have any feedback.
> > > > >
> > > > > Thanks,
> > > > > Mickael
> > > >
> > > >
> >
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by 黃竣陽 <s7...@gmail.com>.
Hello,
I haven’t seen any feedback, so I’ll go ahead and implement the improvement.
Best Regards,
Jiunn-Yang
> 黃竣陽 <s7...@gmail.com> 於 2025年9月29日 晚上11:26 寫道：
>
> Hello all,
>
> There is an improvement in KIP-877 with the following changes:
>
> 1. Removed KafkaPrincipalBuilder plugin: Its lifecycle is short,
> so adding metrics for it would be meaningless.
>
> 2. Added new extra tags for Login, and AuthenticateCallbackHandler
> plugins to avoid collisions. These plugins now define metrics under three different scenarios:
>
> a. Server component, ConnectionMode = Server → registers: "mechanism", "networkProcessor", "listener"
> b. Server component, ConnectionMode = Client → registers: "component", "node-id"
> c. Client component, ConnectionMode = Client → no extra tags
>
> 3. Added new extra tags for SslEngineFactory
> plugins to avoid collisions. These plugins now define metrics under three different scenarios:
>
> a. Server component, ConnectionMode = Server → registers: "networkProcessor", "listener"
> b. Server component, ConnectionMode = Client → registers: "component", "node-id"
> c. Client component, ConnectionMode = Client → no extra tags
>
> <https://cwiki.apache.org/confluence/x/lY3GDQ>
>
> Best Regards,
> Jiunn-Yang
>
>> Jun Rao <ju...@confluent.io.INVALID> 於 2025年4月25日 凌晨2:47 寫道：
>>
>> Hi, Jiunn-Yang,
>>
>> Thanks for the explanation. The change sounds good to me then.
>>
>> Jun
>>
>> On Wed, Apr 23, 2025 at 4:57 AM 黃竣陽 <s7...@gmail.com> wrote:
>>
>>> Hello Jun,
>>>
>>> At this stage, KAFKA-18390 only involves modifications to internal data
>>> structures. Interface-level changes
>>> are currently out of scope. If any interface changes in the future, they
>>> will be tracked under a separate JIRA.
>>>
>>> Best Regards,
>>> Jiunn-Yang
>>>
>>>> Mickael Maison <mi...@gmail.com> 於 2025年4月23日 下午6:07 寫道：
>>>>
>>>> Hi Jun,
>>>>
>>>> KIP-877 is not in a release yet. So assuming nobody objects to the
>>>> change, we can do it before 4.1.0.
>>>> Then any further changes will require another KIP.
>>>>
>>>> Thanks,
>>>> Mickael
>>>>
>>>>
>>>> On Tue, Apr 22, 2025 at 8:05 PM Jun Rao <ju...@confluent.io.invalid>
>>> wrote:
>>>>>
>>>>> Hi, Jiunn-Yang,
>>>>>
>>>>> Thanks for the improvement.
>>>>>
>>>>> KAFKA-18390 changes a public interface. Do we have a KIP for that?
>>>>>
>>>>> Jun
>>>>>
>>>>> On Tue, Apr 22, 2025 at 3:39 AM 黃竣陽 <s7...@gmail.com> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I haven’t seen any feedback, so I’ll go ahead and implement the
>>>>>> improvement
>>>>>> regarding PluginMetrics#metricName.
>>>>>>
>>>>>> Best Regards,
>>>>>> Jiunn-Yang
>>>>>>
>>>>>>> 黃竣陽 <s7...@gmail.com> 於 2025年4月14日 晚上11:43 寫道：
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> There is an improvement in KIP-877.
>>>>>>>
>>>>>>> There will be an update to the PluginMetrics#metricName method: the
>>> type
>>>>>> of the tags parameter will be changed
>>>>>>> from Map to LinkedHashMap.
>>>>>>> This change is necessary because the order of metric tags is important
>>>>>>> 1. If the tag order is inconsistent, identical metrics may be treated
>>> as
>>>>>> distinct ones by the metrics backend
>>>>>>> 2. KAFKA-18390 is updating metric naming to use LinkedHashMap. For
>>>>>> consistency, we should follow the same approach here.
>>>>>>>
>>>>>>> <https://cwiki.apache.org/confluence/x/lY3GDQ>
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Jiunn-Yang
>>>>>>>
>>>>>>>> Luke Chen <sh...@gmail.com> 於 2025年3月12日 上午10:21 寫道：
>>>>>>>>
>>>>>>>> HI Mickael,
>>>>>>>>
>>>>>>>> Thanks for the update.
>>>>>>>> The extra tags addition makes sense to me.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>> Luke
>>>>>>>>
>>>>>>>> On Tue, Mar 11, 2025 at 8:31 PM Mickael Maison <
>>>>>> mickael.maison@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I've not seen any concerns or feedback.
>>>>>>>>> I'll wait a few more days, then we'll carry on the implementation of
>>>>>>>>> KIP-877 with the updated proposal (extra tags for some server side
>>>>>>>>> plugin).
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Mickael
>>>>>>>>>
>>>>>>>>> On Tue, Mar 4, 2025 at 6:57 PM Mickael Maison <
>>>>>> mickael.maison@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I found an issue while implementing KIP-877 for server plugins.
>>>>>>>>>>
>>>>>>>>>> Several server plugins can be instantiated multiple times. This can
>>>>>>>>> happen:
>>>>>>>>>> - when a server is running in combined mode (for example
>>> Authorizer is
>>>>>>>>>> instantiated by both brokers and controllers)
>>>>>>>>>> - when using multiple listeners (for example KafkaPrincipalBuilder
>>> is
>>>>>>>>>> instantiated per processor per listener)
>>>>>>>>>> In the voted proposal, I did not include enough tags for some
>>> plugins
>>>>>>>>>> to uniquely identify each instance and avoid metric name
>>> collisions.
>>>>>>>>>>
>>>>>>>>>> For that reason, I updated the KIP to specify extra tags (in
>>> addition
>>>>>>>>>> of "config" and "class") for the following plugin types:
>>>>>>>>>> - KafkaPrincipalBuilder
>>>>>>>>>> - Authorizer
>>>>>>>>>> - ClientQuotaCallback
>>>>>>>>>> - ConfigProvider
>>>>>>>>>> - AuthenticateCallbackHandler
>>>>>>>>>> - Login
>>>>>>>>>> - SslEngineFactory
>>>>>>>>>>
>>>>>>>>>> You can find the exact tags in the Proposed Changes section of the
>>>>>> KIP:
>>>>>>>>>>
>>>>>>>>>
>>>>>>
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
>>>>>>>>>>
>>>>>>>>>> Let me know if you have questions or concerns.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Mickael
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <
>>>>>> mickael.maison@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> While implementing the KIP I noticed the Kafka Connect specifics
>>> had
>>>>>>>>>>> not been updated correctly based on the latest discussions. The
>>>>>>>>>>> consensus was that connector and tasks should use their context
>>> to
>>>>>>>>>>> retrieve the PluginMetrics instance if they want to register
>>> metrics.
>>>>>>>>>>> While it's a slightly different mechanism than for the other
>>> plugins
>>>>>>>>>>> (they need to implement Monitorable), this allows connector to
>>>>>> support
>>>>>>>>>>> this new feature and still work when deployed on an older Connect
>>>>>>>>>>> runtime.
>>>>>>>>>>>
>>>>>>>>>>> I have updated the KIP and I have 2 PRs ready for review:
>>>>>>>>>>> - Adding the APIs and support for producer and consumer plugins:
>>>>>>>>>>> https://github.com/apache/kafka/pull/17511
>>>>>>>>>>> - Adding support for Connect plugins:
>>>>>>>>> https://github.com/apache/kafka/pull/17804
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Mickael
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <
>>>>>>>>> mickael.maison@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I'm closing the vote. The vote has passed with
>>>>>>>>>>>> + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding
>>> vote
>>>>>>>>>>>> from Hector.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Mickael
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com>
>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Mickael,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the KIP.
>>>>>>>>>>>>> +1 (binding) from me.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>> Luke
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tbentley@redhat.com
>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 (binding).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks Mickael!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, 1 Aug 2024 at 05:12, Mickael Maison <
>>>>>>>>> mickael.maison@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Bumping this thread to get some more votes and/or feedback.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As I restarted the vote on June 10 after major changes, I'm
>>>>>>>>> only
>>>>>>>>>>>>>>> counting votes since then.
>>>>>>>>>>>>>>> So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/
>>>>>>>>> 919 3RD A)
>>>>>>>>>>>>>>> <hg...@bloomberg.net> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This will help eliminate some boilerplate code we have for
>>>>>>>>> our
>>>>>>>>>>>>>>> connectors.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 (non-binding)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
>>>>>>>>>>>>>>> dev@kafka.apache.org
>>>>>>>>>>>>>>>> Subject: Re: [VOTE] KIP-877: Mechanism for plugins and
>>>>>>>>> connectors to
>>>>>>>>>>>>>>> register metrics
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Bumping this thread.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Let me know if you have any feedback.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <
>>>>>>>>> fearthecellos@gmail.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> +1 (binding), thanks Mickael!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Jun 10, 2024, 04:24 Mickael Maison <
>>>>>>>>> mickael.maison@gmail.com
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Following the feedback in the DISCUSS thread, I made
>>>>>>>>> significant
>>>>>>>>>>>>>>>>>> changes to the proposal. So I'd like to restart a vote
>>>>>>>>> for KIP-877:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
>>>>>>>>>>>>>>>> ns+and+connectors+to+register+metrics
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <
>>>>>>>>> tbentley@redhat.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Mickael,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> You'll have seen that I left some comments on the
>>>>>>>>> discussion
>>>>>>>>>>>>>>> thread, but
>>>>>>>>>>>>>>>>>>> they're minor enough that I'm happy to vote +1 here.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
>>>>>>>>>>>>>>> mickael.maison@gmail.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Bumping this thread since I've not seen any feedback.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
>>>>>>>>>>>>>>>>>>>> <mi...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I'd like to start a vote on KIP-877:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
>>>>>>>>>>>>>>>> ns+and+connectors+to+register+metrics
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Let me know if you have any feedback.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>
>>>
>
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by 黃竣陽 <s7...@gmail.com>.
Hello all,
There is an improvement in KIP-877 with the following changes:
1. Removed KafkaPrincipalBuilder plugin: Its lifecycle is short,
so adding metrics for it would be meaningless.
2. Added new extra tags for Login, and AuthenticateCallbackHandler
plugins to avoid collisions. These plugins now define metrics under three different scenarios:
a. Server component, ConnectionMode = Server → registers: "mechanism", "networkProcessor", "listener"
b. Server component, ConnectionMode = Client → registers: "component", "node-id"
c. Client component, ConnectionMode = Client → no extra tags
3. Added new extra tags for SslEngineFactory
plugins to avoid collisions. These plugins now define metrics under three different scenarios:
a. Server component, ConnectionMode = Server → registers: "networkProcessor", "listener"
b. Server component, ConnectionMode = Client → registers: "component", "node-id"
c. Client component, ConnectionMode = Client → no extra tags
<https://cwiki.apache.org/confluence/x/lY3GDQ>
Best Regards,
Jiunn-Yang
> Jun Rao <ju...@confluent.io.INVALID> 於 2025年4月25日 凌晨2:47 寫道：
>
> Hi, Jiunn-Yang,
>
> Thanks for the explanation. The change sounds good to me then.
>
> Jun
>
> On Wed, Apr 23, 2025 at 4:57 AM 黃竣陽 <s7...@gmail.com> wrote:
>
>> Hello Jun,
>>
>> At this stage, KAFKA-18390 only involves modifications to internal data
>> structures. Interface-level changes
>> are currently out of scope. If any interface changes in the future, they
>> will be tracked under a separate JIRA.
>>
>> Best Regards,
>> Jiunn-Yang
>>
>>> Mickael Maison <mi...@gmail.com> 於 2025年4月23日 下午6:07 寫道：
>>>
>>> Hi Jun,
>>>
>>> KIP-877 is not in a release yet. So assuming nobody objects to the
>>> change, we can do it before 4.1.0.
>>> Then any further changes will require another KIP.
>>>
>>> Thanks,
>>> Mickael
>>>
>>>
>>> On Tue, Apr 22, 2025 at 8:05 PM Jun Rao <ju...@confluent.io.invalid>
>> wrote:
>>>>
>>>> Hi, Jiunn-Yang,
>>>>
>>>> Thanks for the improvement.
>>>>
>>>> KAFKA-18390 changes a public interface. Do we have a KIP for that?
>>>>
>>>> Jun
>>>>
>>>> On Tue, Apr 22, 2025 at 3:39 AM 黃竣陽 <s7...@gmail.com> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I haven’t seen any feedback, so I’ll go ahead and implement the
>>>>> improvement
>>>>> regarding PluginMetrics#metricName.
>>>>>
>>>>> Best Regards,
>>>>> Jiunn-Yang
>>>>>
>>>>>> 黃竣陽 <s7...@gmail.com> 於 2025年4月14日 晚上11:43 寫道：
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> There is an improvement in KIP-877.
>>>>>>
>>>>>> There will be an update to the PluginMetrics#metricName method: the
>> type
>>>>> of the tags parameter will be changed
>>>>>> from Map to LinkedHashMap.
>>>>>> This change is necessary because the order of metric tags is important
>>>>>> 1. If the tag order is inconsistent, identical metrics may be treated
>> as
>>>>> distinct ones by the metrics backend
>>>>>> 2. KAFKA-18390 is updating metric naming to use LinkedHashMap. For
>>>>> consistency, we should follow the same approach here.
>>>>>>
>>>>>> <https://cwiki.apache.org/confluence/x/lY3GDQ>
>>>>>>
>>>>>> Best Regards,
>>>>>> Jiunn-Yang
>>>>>>
>>>>>>> Luke Chen <sh...@gmail.com> 於 2025年3月12日 上午10:21 寫道：
>>>>>>>
>>>>>>> HI Mickael,
>>>>>>>
>>>>>>> Thanks for the update.
>>>>>>> The extra tags addition makes sense to me.
>>>>>>>
>>>>>>> Thanks.
>>>>>>> Luke
>>>>>>>
>>>>>>> On Tue, Mar 11, 2025 at 8:31 PM Mickael Maison <
>>>>> mickael.maison@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I've not seen any concerns or feedback.
>>>>>>>> I'll wait a few more days, then we'll carry on the implementation of
>>>>>>>> KIP-877 with the updated proposal (extra tags for some server side
>>>>>>>> plugin).
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Mickael
>>>>>>>>
>>>>>>>> On Tue, Mar 4, 2025 at 6:57 PM Mickael Maison <
>>>>> mickael.maison@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I found an issue while implementing KIP-877 for server plugins.
>>>>>>>>>
>>>>>>>>> Several server plugins can be instantiated multiple times. This can
>>>>>>>> happen:
>>>>>>>>> - when a server is running in combined mode (for example
>> Authorizer is
>>>>>>>>> instantiated by both brokers and controllers)
>>>>>>>>> - when using multiple listeners (for example KafkaPrincipalBuilder
>> is
>>>>>>>>> instantiated per processor per listener)
>>>>>>>>> In the voted proposal, I did not include enough tags for some
>> plugins
>>>>>>>>> to uniquely identify each instance and avoid metric name
>> collisions.
>>>>>>>>>
>>>>>>>>> For that reason, I updated the KIP to specify extra tags (in
>> addition
>>>>>>>>> of "config" and "class") for the following plugin types:
>>>>>>>>> - KafkaPrincipalBuilder
>>>>>>>>> - Authorizer
>>>>>>>>> - ClientQuotaCallback
>>>>>>>>> - ConfigProvider
>>>>>>>>> - AuthenticateCallbackHandler
>>>>>>>>> - Login
>>>>>>>>> - SslEngineFactory
>>>>>>>>>
>>>>>>>>> You can find the exact tags in the Proposed Changes section of the
>>>>> KIP:
>>>>>>>>>
>>>>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
>>>>>>>>>
>>>>>>>>> Let me know if you have questions or concerns.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Mickael
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <
>>>>> mickael.maison@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> While implementing the KIP I noticed the Kafka Connect specifics
>> had
>>>>>>>>>> not been updated correctly based on the latest discussions. The
>>>>>>>>>> consensus was that connector and tasks should use their context
>> to
>>>>>>>>>> retrieve the PluginMetrics instance if they want to register
>> metrics.
>>>>>>>>>> While it's a slightly different mechanism than for the other
>> plugins
>>>>>>>>>> (they need to implement Monitorable), this allows connector to
>>>>> support
>>>>>>>>>> this new feature and still work when deployed on an older Connect
>>>>>>>>>> runtime.
>>>>>>>>>>
>>>>>>>>>> I have updated the KIP and I have 2 PRs ready for review:
>>>>>>>>>> - Adding the APIs and support for producer and consumer plugins:
>>>>>>>>>> https://github.com/apache/kafka/pull/17511
>>>>>>>>>> - Adding support for Connect plugins:
>>>>>>>> https://github.com/apache/kafka/pull/17804
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Mickael
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <
>>>>>>>> mickael.maison@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I'm closing the vote. The vote has passed with
>>>>>>>>>>> + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding
>> vote
>>>>>>>>>>> from Hector.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Mickael
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com>
>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Mickael,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the KIP.
>>>>>>>>>>>> +1 (binding) from me.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks.
>>>>>>>>>>>> Luke
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tbentley@redhat.com
>>>
>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1 (binding).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks Mickael!
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, 1 Aug 2024 at 05:12, Mickael Maison <
>>>>>>>> mickael.maison@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Bumping this thread to get some more votes and/or feedback.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As I restarted the vote on June 10 after major changes, I'm
>>>>>>>> only
>>>>>>>>>>>>>> counting votes since then.
>>>>>>>>>>>>>> So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/
>>>>>>>> 919 3RD A)
>>>>>>>>>>>>>> <hg...@bloomberg.net> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This will help eliminate some boilerplate code we have for
>>>>>>>> our
>>>>>>>>>>>>>> connectors.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> +1 (non-binding)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
>>>>>>>>>>>>>> dev@kafka.apache.org
>>>>>>>>>>>>>>> Subject: Re: [VOTE] KIP-877: Mechanism for plugins and
>>>>>>>> connectors to
>>>>>>>>>>>>>> register metrics
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Bumping this thread.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Let me know if you have any feedback.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <
>>>>>>>> fearthecellos@gmail.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 (binding), thanks Mickael!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Jun 10, 2024, 04:24 Mickael Maison <
>>>>>>>> mickael.maison@gmail.com
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Following the feedback in the DISCUSS thread, I made
>>>>>>>> significant
>>>>>>>>>>>>>>>>> changes to the proposal. So I'd like to restart a vote
>>>>>>>> for KIP-877:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
>>>>>>>>>>>>>>> ns+and+connectors+to+register+metrics
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <
>>>>>>>> tbentley@redhat.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Mickael,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> You'll have seen that I left some comments on the
>>>>>>>> discussion
>>>>>>>>>>>>>> thread, but
>>>>>>>>>>>>>>>>>> they're minor enough that I'm happy to vote +1 here.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
>>>>>>>>>>>>>> mickael.maison@gmail.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Bumping this thread since I've not seen any feedback.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
>>>>>>>>>>>>>>>>>>> <mi...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I'd like to start a vote on KIP-877:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>
>>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
>>>>>>>>>>>>>>> ns+and+connectors+to+register+metrics
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Let me know if you have any feedback.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>>
>>
>>
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Jun Rao <ju...@confluent.io.INVALID>.
Hi, Jiunn-Yang,
Thanks for the explanation. The change sounds good to me then.
Jun
On Wed, Apr 23, 2025 at 4:57 AM 黃竣陽 <s7...@gmail.com> wrote:
> Hello Jun,
>
> At this stage, KAFKA-18390 only involves modifications to internal data
> structures. Interface-level changes
> are currently out of scope. If any interface changes in the future, they
> will be tracked under a separate JIRA.
>
> Best Regards,
> Jiunn-Yang
>
> > Mickael Maison <mi...@gmail.com> 於 2025年4月23日 下午6:07 寫道：
> >
> > Hi Jun,
> >
> > KIP-877 is not in a release yet. So assuming nobody objects to the
> > change, we can do it before 4.1.0.
> > Then any further changes will require another KIP.
> >
> > Thanks,
> > Mickael
> >
> >
> > On Tue, Apr 22, 2025 at 8:05 PM Jun Rao <ju...@confluent.io.invalid>
> wrote:
> >>
> >> Hi, Jiunn-Yang,
> >>
> >> Thanks for the improvement.
> >>
> >> KAFKA-18390 changes a public interface. Do we have a KIP for that?
> >>
> >> Jun
> >>
> >> On Tue, Apr 22, 2025 at 3:39 AM 黃竣陽 <s7...@gmail.com> wrote:
> >>
> >>> Hello,
> >>>
> >>> I haven’t seen any feedback, so I’ll go ahead and implement the
> >>> improvement
> >>> regarding PluginMetrics#metricName.
> >>>
> >>> Best Regards,
> >>> Jiunn-Yang
> >>>
> >>>> 黃竣陽 <s7...@gmail.com> 於 2025年4月14日 晚上11:43 寫道：
> >>>>
> >>>> Hi,
> >>>>
> >>>> There is an improvement in KIP-877.
> >>>>
> >>>> There will be an update to the PluginMetrics#metricName method: the
> type
> >>> of the tags parameter will be changed
> >>>> from Map to LinkedHashMap.
> >>>> This change is necessary because the order of metric tags is important
> >>>> 1. If the tag order is inconsistent, identical metrics may be treated
> as
> >>> distinct ones by the metrics backend
> >>>> 2. KAFKA-18390 is updating metric naming to use LinkedHashMap. For
> >>> consistency, we should follow the same approach here.
> >>>>
> >>>> <https://cwiki.apache.org/confluence/x/lY3GDQ>
> >>>>
> >>>> Best Regards,
> >>>> Jiunn-Yang
> >>>>
> >>>>> Luke Chen <sh...@gmail.com> 於 2025年3月12日 上午10:21 寫道：
> >>>>>
> >>>>> HI Mickael,
> >>>>>
> >>>>> Thanks for the update.
> >>>>> The extra tags addition makes sense to me.
> >>>>>
> >>>>> Thanks.
> >>>>> Luke
> >>>>>
> >>>>> On Tue, Mar 11, 2025 at 8:31 PM Mickael Maison <
> >>> mickael.maison@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> I've not seen any concerns or feedback.
> >>>>>> I'll wait a few more days, then we'll carry on the implementation of
> >>>>>> KIP-877 with the updated proposal (extra tags for some server side
> >>>>>> plugin).
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Mickael
> >>>>>>
> >>>>>> On Tue, Mar 4, 2025 at 6:57 PM Mickael Maison <
> >>> mickael.maison@gmail.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I found an issue while implementing KIP-877 for server plugins.
> >>>>>>>
> >>>>>>> Several server plugins can be instantiated multiple times. This can
> >>>>>> happen:
> >>>>>>> - when a server is running in combined mode (for example
> Authorizer is
> >>>>>>> instantiated by both brokers and controllers)
> >>>>>>> - when using multiple listeners (for example KafkaPrincipalBuilder
> is
> >>>>>>> instantiated per processor per listener)
> >>>>>>> In the voted proposal, I did not include enough tags for some
> plugins
> >>>>>>> to uniquely identify each instance and avoid metric name
> collisions.
> >>>>>>>
> >>>>>>> For that reason, I updated the KIP to specify extra tags (in
> addition
> >>>>>>> of "config" and "class") for the following plugin types:
> >>>>>>> - KafkaPrincipalBuilder
> >>>>>>> - Authorizer
> >>>>>>> - ClientQuotaCallback
> >>>>>>> - ConfigProvider
> >>>>>>> - AuthenticateCallbackHandler
> >>>>>>> - Login
> >>>>>>> - SslEngineFactory
> >>>>>>>
> >>>>>>> You can find the exact tags in the Proposed Changes section of the
> >>> KIP:
> >>>>>>>
> >>>>>>
> >>>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
> >>>>>>>
> >>>>>>> Let me know if you have questions or concerns.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Mickael
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <
> >>> mickael.maison@gmail.com>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> While implementing the KIP I noticed the Kafka Connect specifics
> had
> >>>>>>>> not been updated correctly based on the latest discussions. The
> >>>>>>>> consensus was that connector and tasks should use their context
> to
> >>>>>>>> retrieve the PluginMetrics instance if they want to register
> metrics.
> >>>>>>>> While it's a slightly different mechanism than for the other
> plugins
> >>>>>>>> (they need to implement Monitorable), this allows connector to
> >>> support
> >>>>>>>> this new feature and still work when deployed on an older Connect
> >>>>>>>> runtime.
> >>>>>>>>
> >>>>>>>> I have updated the KIP and I have 2 PRs ready for review:
> >>>>>>>> - Adding the APIs and support for producer and consumer plugins:
> >>>>>>>> https://github.com/apache/kafka/pull/17511
> >>>>>>>> - Adding support for Connect plugins:
> >>>>>> https://github.com/apache/kafka/pull/17804
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Mickael
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <
> >>>>>> mickael.maison@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> I'm closing the vote. The vote has passed with
> >>>>>>>>> + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding
> vote
> >>>>>>>>> from Hector.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Mickael
> >>>>>>>>>
> >>>>>>>>> On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com>
> >>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi Mickael,
> >>>>>>>>>>
> >>>>>>>>>> Thanks for the KIP.
> >>>>>>>>>> +1 (binding) from me.
> >>>>>>>>>>
> >>>>>>>>>> Thanks.
> >>>>>>>>>> Luke
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tbentley@redhat.com
> >
> >>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> +1 (binding).
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks Mickael!
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, 1 Aug 2024 at 05:12, Mickael Maison <
> >>>>>> mickael.maison@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Bumping this thread to get some more votes and/or feedback.
> >>>>>>>>>>>>
> >>>>>>>>>>>> As I restarted the vote on June 10 after major changes, I'm
> >>>>>> only
> >>>>>>>>>>>> counting votes since then.
> >>>>>>>>>>>> So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Mickael
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/
> >>>>>> 919 3RD A)
> >>>>>>>>>>>> <hg...@bloomberg.net> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> This will help eliminate some boilerplate code we have for
> >>>>>> our
> >>>>>>>>>>>> connectors.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> +1 (non-binding)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
> >>>>>>>>>>>> dev@kafka.apache.org
> >>>>>>>>>>>>> Subject: Re: [VOTE] KIP-877: Mechanism for plugins and
> >>>>>> connectors to
> >>>>>>>>>>>> register metrics
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Bumping this thread.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Let me know if you have any feedback.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Mickael
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <
> >>>>>> fearthecellos@gmail.com
> >>>>>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> +1 (binding), thanks Mickael!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Jun 10, 2024, 04:24 Mickael Maison <
> >>>>>> mickael.maison@gmail.com
> >>>>>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Following the feedback in the DISCUSS thread, I made
> >>>>>> significant
> >>>>>>>>>>>>>>> changes to the proposal. So I'd like to restart a vote
> >>>>>> for KIP-877:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> >>>>>>>>>>>>> ns+and+connectors+to+register+metrics
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Mickael
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <
> >>>>>> tbentley@redhat.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi Mickael,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> You'll have seen that I left some comments on the
> >>>>>> discussion
> >>>>>>>>>>>> thread, but
> >>>>>>>>>>>>>>>> they're minor enough that I'm happy to vote +1 here.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Tom
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
> >>>>>>>>>>>> mickael.maison@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Bumping this thread since I've not seen any feedback.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>> Mickael
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> >>>>>>>>>>>>>>>>> <mi...@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I'd like to start a vote on KIP-877:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> >>>>>>>>>>>>> ns+and+connectors+to+register+metrics
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Let me know if you have any feedback.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> Mickael
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
> >>>>
> >>>
> >>>
>
>
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by 黃竣陽 <s7...@gmail.com>.
Hello Jun,
At this stage, KAFKA-18390 only involves modifications to internal data structures. Interface-level changes
are currently out of scope. If any interface changes in the future, they will be tracked under a separate JIRA.
Best Regards,
Jiunn-Yang
> Mickael Maison <mi...@gmail.com> 於 2025年4月23日 下午6:07 寫道：
>
> Hi Jun,
>
> KIP-877 is not in a release yet. So assuming nobody objects to the
> change, we can do it before 4.1.0.
> Then any further changes will require another KIP.
>
> Thanks,
> Mickael
>
>
> On Tue, Apr 22, 2025 at 8:05 PM Jun Rao <ju...@confluent.io.invalid> wrote:
>>
>> Hi, Jiunn-Yang,
>>
>> Thanks for the improvement.
>>
>> KAFKA-18390 changes a public interface. Do we have a KIP for that?
>>
>> Jun
>>
>> On Tue, Apr 22, 2025 at 3:39 AM 黃竣陽 <s7...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I haven’t seen any feedback, so I’ll go ahead and implement the
>>> improvement
>>> regarding PluginMetrics#metricName.
>>>
>>> Best Regards,
>>> Jiunn-Yang
>>>
>>>> 黃竣陽 <s7...@gmail.com> 於 2025年4月14日 晚上11:43 寫道：
>>>>
>>>> Hi,
>>>>
>>>> There is an improvement in KIP-877.
>>>>
>>>> There will be an update to the PluginMetrics#metricName method: the type
>>> of the tags parameter will be changed
>>>> from Map to LinkedHashMap.
>>>> This change is necessary because the order of metric tags is important
>>>> 1. If the tag order is inconsistent, identical metrics may be treated as
>>> distinct ones by the metrics backend
>>>> 2. KAFKA-18390 is updating metric naming to use LinkedHashMap. For
>>> consistency, we should follow the same approach here.
>>>>
>>>> <https://cwiki.apache.org/confluence/x/lY3GDQ>
>>>>
>>>> Best Regards,
>>>> Jiunn-Yang
>>>>
>>>>> Luke Chen <sh...@gmail.com> 於 2025年3月12日 上午10:21 寫道：
>>>>>
>>>>> HI Mickael,
>>>>>
>>>>> Thanks for the update.
>>>>> The extra tags addition makes sense to me.
>>>>>
>>>>> Thanks.
>>>>> Luke
>>>>>
>>>>> On Tue, Mar 11, 2025 at 8:31 PM Mickael Maison <
>>> mickael.maison@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I've not seen any concerns or feedback.
>>>>>> I'll wait a few more days, then we'll carry on the implementation of
>>>>>> KIP-877 with the updated proposal (extra tags for some server side
>>>>>> plugin).
>>>>>>
>>>>>> Thanks,
>>>>>> Mickael
>>>>>>
>>>>>> On Tue, Mar 4, 2025 at 6:57 PM Mickael Maison <
>>> mickael.maison@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I found an issue while implementing KIP-877 for server plugins.
>>>>>>>
>>>>>>> Several server plugins can be instantiated multiple times. This can
>>>>>> happen:
>>>>>>> - when a server is running in combined mode (for example Authorizer is
>>>>>>> instantiated by both brokers and controllers)
>>>>>>> - when using multiple listeners (for example KafkaPrincipalBuilder is
>>>>>>> instantiated per processor per listener)
>>>>>>> In the voted proposal, I did not include enough tags for some plugins
>>>>>>> to uniquely identify each instance and avoid metric name collisions.
>>>>>>>
>>>>>>> For that reason, I updated the KIP to specify extra tags (in addition
>>>>>>> of "config" and "class") for the following plugin types:
>>>>>>> - KafkaPrincipalBuilder
>>>>>>> - Authorizer
>>>>>>> - ClientQuotaCallback
>>>>>>> - ConfigProvider
>>>>>>> - AuthenticateCallbackHandler
>>>>>>> - Login
>>>>>>> - SslEngineFactory
>>>>>>>
>>>>>>> You can find the exact tags in the Proposed Changes section of the
>>> KIP:
>>>>>>>
>>>>>>
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
>>>>>>>
>>>>>>> Let me know if you have questions or concerns.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mickael
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <
>>> mickael.maison@gmail.com>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> While implementing the KIP I noticed the Kafka Connect specifics had
>>>>>>>> not been updated correctly based on the latest discussions. The
>>>>>>>> consensus was that connector and tasks should use their context to
>>>>>>>> retrieve the PluginMetrics instance if they want to register metrics.
>>>>>>>> While it's a slightly different mechanism than for the other plugins
>>>>>>>> (they need to implement Monitorable), this allows connector to
>>> support
>>>>>>>> this new feature and still work when deployed on an older Connect
>>>>>>>> runtime.
>>>>>>>>
>>>>>>>> I have updated the KIP and I have 2 PRs ready for review:
>>>>>>>> - Adding the APIs and support for producer and consumer plugins:
>>>>>>>> https://github.com/apache/kafka/pull/17511
>>>>>>>> - Adding support for Connect plugins:
>>>>>> https://github.com/apache/kafka/pull/17804
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Mickael
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <
>>>>>> mickael.maison@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I'm closing the vote. The vote has passed with
>>>>>>>>> + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote
>>>>>>>>> from Hector.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Mickael
>>>>>>>>>
>>>>>>>>> On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com>
>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Mickael,
>>>>>>>>>>
>>>>>>>>>> Thanks for the KIP.
>>>>>>>>>> +1 (binding) from me.
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>> Luke
>>>>>>>>>>
>>>>>>>>>> On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com>
>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1 (binding).
>>>>>>>>>>>
>>>>>>>>>>> Thanks Mickael!
>>>>>>>>>>>
>>>>>>>>>>> On Thu, 1 Aug 2024 at 05:12, Mickael Maison <
>>>>>> mickael.maison@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Bumping this thread to get some more votes and/or feedback.
>>>>>>>>>>>>
>>>>>>>>>>>> As I restarted the vote on June 10 after major changes, I'm
>>>>>> only
>>>>>>>>>>>> counting votes since then.
>>>>>>>>>>>> So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Mickael
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/
>>>>>> 919 3RD A)
>>>>>>>>>>>> <hg...@bloomberg.net> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> This will help eliminate some boilerplate code we have for
>>>>>> our
>>>>>>>>>>>> connectors.
>>>>>>>>>>>>>
>>>>>>>>>>>>> +1 (non-binding)
>>>>>>>>>>>>>
>>>>>>>>>>>>> From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
>>>>>>>>>>>> dev@kafka.apache.org
>>>>>>>>>>>>> Subject: Re: [VOTE] KIP-877: Mechanism for plugins and
>>>>>> connectors to
>>>>>>>>>>>> register metrics
>>>>>>>>>>>>>
>>>>>>>>>>>>> Bumping this thread.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let me know if you have any feedback.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <
>>>>>> fearthecellos@gmail.com
>>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 (binding), thanks Mickael!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Jun 10, 2024, 04:24 Mickael Maison <
>>>>>> mickael.maison@gmail.com
>>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Following the feedback in the DISCUSS thread, I made
>>>>>> significant
>>>>>>>>>>>>>>> changes to the proposal. So I'd like to restart a vote
>>>>>> for KIP-877:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
>>>>>>>>>>>>> ns+and+connectors+to+register+metrics
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <
>>>>>> tbentley@redhat.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Mickael,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You'll have seen that I left some comments on the
>>>>>> discussion
>>>>>>>>>>>> thread, but
>>>>>>>>>>>>>>>> they're minor enough that I'm happy to vote +1 here.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
>>>>>>>>>>>> mickael.maison@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Bumping this thread since I've not seen any feedback.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
>>>>>>>>>>>>>>>>> <mi...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'd like to start a vote on KIP-877:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
>>>>>>>>>>>>> ns+and+connectors+to+register+metrics
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Let me know if you have any feedback.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>>>
>>>
>>>
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Mickael Maison <mi...@gmail.com>.
Hi Jun,
KIP-877 is not in a release yet. So assuming nobody objects to the
change, we can do it before 4.1.0.
Then any further changes will require another KIP.
Thanks,
Mickael
On Tue, Apr 22, 2025 at 8:05 PM Jun Rao <ju...@confluent.io.invalid> wrote:
>
> Hi, Jiunn-Yang,
>
> Thanks for the improvement.
>
> KAFKA-18390 changes a public interface. Do we have a KIP for that?
>
> Jun
>
> On Tue, Apr 22, 2025 at 3:39 AM 黃竣陽 <s7...@gmail.com> wrote:
>
> > Hello,
> >
> > I haven’t seen any feedback, so I’ll go ahead and implement the
> > improvement
> > regarding PluginMetrics#metricName.
> >
> > Best Regards,
> > Jiunn-Yang
> >
> > > 黃竣陽 <s7...@gmail.com> 於 2025年4月14日 晚上11:43 寫道：
> > >
> > > Hi,
> > >
> > > There is an improvement in KIP-877.
> > >
> > > There will be an update to the PluginMetrics#metricName method: the type
> > of the tags parameter will be changed
> > > from Map to LinkedHashMap.
> > > This change is necessary because the order of metric tags is important
> > > 1. If the tag order is inconsistent, identical metrics may be treated as
> > distinct ones by the metrics backend
> > > 2. KAFKA-18390 is updating metric naming to use LinkedHashMap. For
> > consistency, we should follow the same approach here.
> > >
> > > <https://cwiki.apache.org/confluence/x/lY3GDQ>
> > >
> > > Best Regards,
> > > Jiunn-Yang
> > >
> > >> Luke Chen <sh...@gmail.com> 於 2025年3月12日 上午10:21 寫道：
> > >>
> > >> HI Mickael,
> > >>
> > >> Thanks for the update.
> > >> The extra tags addition makes sense to me.
> > >>
> > >> Thanks.
> > >> Luke
> > >>
> > >> On Tue, Mar 11, 2025 at 8:31 PM Mickael Maison <
> > mickael.maison@gmail.com>
> > >> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> I've not seen any concerns or feedback.
> > >>> I'll wait a few more days, then we'll carry on the implementation of
> > >>> KIP-877 with the updated proposal (extra tags for some server side
> > >>> plugin).
> > >>>
> > >>> Thanks,
> > >>> Mickael
> > >>>
> > >>> On Tue, Mar 4, 2025 at 6:57 PM Mickael Maison <
> > mickael.maison@gmail.com>
> > >>> wrote:
> > >>>>
> > >>>> Hi,
> > >>>>
> > >>>> I found an issue while implementing KIP-877 for server plugins.
> > >>>>
> > >>>> Several server plugins can be instantiated multiple times. This can
> > >>> happen:
> > >>>> - when a server is running in combined mode (for example Authorizer is
> > >>>> instantiated by both brokers and controllers)
> > >>>> - when using multiple listeners (for example KafkaPrincipalBuilder is
> > >>>> instantiated per processor per listener)
> > >>>> In the voted proposal, I did not include enough tags for some plugins
> > >>>> to uniquely identify each instance and avoid metric name collisions.
> > >>>>
> > >>>> For that reason, I updated the KIP to specify extra tags (in addition
> > >>>> of "config" and "class") for the following plugin types:
> > >>>> - KafkaPrincipalBuilder
> > >>>> - Authorizer
> > >>>> - ClientQuotaCallback
> > >>>> - ConfigProvider
> > >>>> - AuthenticateCallbackHandler
> > >>>> - Login
> > >>>> - SslEngineFactory
> > >>>>
> > >>>> You can find the exact tags in the Proposed Changes section of the
> > KIP:
> > >>>>
> > >>>
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
> > >>>>
> > >>>> Let me know if you have questions or concerns.
> > >>>>
> > >>>> Thanks,
> > >>>> Mickael
> > >>>>
> > >>>>
> > >>>> On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <
> > mickael.maison@gmail.com>
> > >>> wrote:
> > >>>>>
> > >>>>> Hi,
> > >>>>>
> > >>>>> While implementing the KIP I noticed the Kafka Connect specifics had
> > >>>>> not been updated correctly based on the latest discussions. The
> > >>>>> consensus was that connector and tasks should use their context to
> > >>>>> retrieve the PluginMetrics instance if they want to register metrics.
> > >>>>> While it's a slightly different mechanism than for the other plugins
> > >>>>> (they need to implement Monitorable), this allows connector to
> > support
> > >>>>> this new feature and still work when deployed on an older Connect
> > >>>>> runtime.
> > >>>>>
> > >>>>> I have updated the KIP and I have 2 PRs ready for review:
> > >>>>> - Adding the APIs and support for producer and consumer plugins:
> > >>>>> https://github.com/apache/kafka/pull/17511
> > >>>>> - Adding support for Connect plugins:
> > >>> https://github.com/apache/kafka/pull/17804
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Mickael
> > >>>>>
> > >>>>>
> > >>>>> On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <
> > >>> mickael.maison@gmail.com> wrote:
> > >>>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> I'm closing the vote. The vote has passed with
> > >>>>>> + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote
> > >>>>>> from Hector.
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> Mickael
> > >>>>>>
> > >>>>>> On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com>
> > wrote:
> > >>>>>>>
> > >>>>>>> Hi Mickael,
> > >>>>>>>
> > >>>>>>> Thanks for the KIP.
> > >>>>>>> +1 (binding) from me.
> > >>>>>>>
> > >>>>>>> Thanks.
> > >>>>>>> Luke
> > >>>>>>>
> > >>>>>>> On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com>
> > >>> wrote:
> > >>>>>>>
> > >>>>>>>> +1 (binding).
> > >>>>>>>>
> > >>>>>>>> Thanks Mickael!
> > >>>>>>>>
> > >>>>>>>> On Thu, 1 Aug 2024 at 05:12, Mickael Maison <
> > >>> mickael.maison@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi,
> > >>>>>>>>>
> > >>>>>>>>> Bumping this thread to get some more votes and/or feedback.
> > >>>>>>>>>
> > >>>>>>>>> As I restarted the vote on June 10 after major changes, I'm
> > >>> only
> > >>>>>>>>> counting votes since then.
> > >>>>>>>>> So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>> Mickael
> > >>>>>>>>>
> > >>>>>>>>> On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/
> > >>> 919 3RD A)
> > >>>>>>>>> <hg...@bloomberg.net> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> This will help eliminate some boilerplate code we have for
> > >>> our
> > >>>>>>>>> connectors.
> > >>>>>>>>>>
> > >>>>>>>>>> +1 (non-binding)
> > >>>>>>>>>>
> > >>>>>>>>>> From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
> > >>>>>>>>> dev@kafka.apache.org
> > >>>>>>>>>> Subject: Re: [VOTE] KIP-877: Mechanism for plugins and
> > >>> connectors to
> > >>>>>>>>> register metrics
> > >>>>>>>>>>
> > >>>>>>>>>> Bumping this thread.
> > >>>>>>>>>>
> > >>>>>>>>>> Let me know if you have any feedback.
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks,
> > >>>>>>>>>> Mickael
> > >>>>>>>>>>
> > >>>>>>>>>> On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <
> > >>> fearthecellos@gmail.com
> > >>>>>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> +1 (binding), thanks Mickael!
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Mon, Jun 10, 2024, 04:24 Mickael Maison <
> > >>> mickael.maison@gmail.com
> > >>>>>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Hi,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Following the feedback in the DISCUSS thread, I made
> > >>> significant
> > >>>>>>>>>>>> changes to the proposal. So I'd like to restart a vote
> > >>> for KIP-877:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > >>>>>>>>>> ns+and+connectors+to+register+metrics
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>> Mickael
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <
> > >>> tbentley@redhat.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Hi Mickael,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> You'll have seen that I left some comments on the
> > >>> discussion
> > >>>>>>>>> thread, but
> > >>>>>>>>>>>>> they're minor enough that I'm happy to vote +1 here.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Tom
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
> > >>>>>>>>> mickael.maison@gmail.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Bumping this thread since I've not seen any feedback.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>> Mickael
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> > >>>>>>>>>>>>>> <mi...@gmail.com> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I'd like to start a vote on KIP-877:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > >>>>>>>>>> ns+and+connectors+to+register+metrics
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Let me know if you have any feedback.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>> Mickael
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>
> > >
> >
> >
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Jun Rao <ju...@confluent.io.INVALID>.
Hi, Jiunn-Yang,
Thanks for the improvement.
KAFKA-18390 changes a public interface. Do we have a KIP for that?
Jun
On Tue, Apr 22, 2025 at 3:39 AM 黃竣陽 <s7...@gmail.com> wrote:
> Hello,
>
> I haven’t seen any feedback, so I’ll go ahead and implement the
> improvement
> regarding PluginMetrics#metricName.
>
> Best Regards,
> Jiunn-Yang
>
> > 黃竣陽 <s7...@gmail.com> 於 2025年4月14日 晚上11:43 寫道：
> >
> > Hi,
> >
> > There is an improvement in KIP-877.
> >
> > There will be an update to the PluginMetrics#metricName method: the type
> of the tags parameter will be changed
> > from Map to LinkedHashMap.
> > This change is necessary because the order of metric tags is important
> > 1. If the tag order is inconsistent, identical metrics may be treated as
> distinct ones by the metrics backend
> > 2. KAFKA-18390 is updating metric naming to use LinkedHashMap. For
> consistency, we should follow the same approach here.
> >
> > <https://cwiki.apache.org/confluence/x/lY3GDQ>
> >
> > Best Regards,
> > Jiunn-Yang
> >
> >> Luke Chen <sh...@gmail.com> 於 2025年3月12日 上午10:21 寫道：
> >>
> >> HI Mickael,
> >>
> >> Thanks for the update.
> >> The extra tags addition makes sense to me.
> >>
> >> Thanks.
> >> Luke
> >>
> >> On Tue, Mar 11, 2025 at 8:31 PM Mickael Maison <
> mickael.maison@gmail.com>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> I've not seen any concerns or feedback.
> >>> I'll wait a few more days, then we'll carry on the implementation of
> >>> KIP-877 with the updated proposal (extra tags for some server side
> >>> plugin).
> >>>
> >>> Thanks,
> >>> Mickael
> >>>
> >>> On Tue, Mar 4, 2025 at 6:57 PM Mickael Maison <
> mickael.maison@gmail.com>
> >>> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> I found an issue while implementing KIP-877 for server plugins.
> >>>>
> >>>> Several server plugins can be instantiated multiple times. This can
> >>> happen:
> >>>> - when a server is running in combined mode (for example Authorizer is
> >>>> instantiated by both brokers and controllers)
> >>>> - when using multiple listeners (for example KafkaPrincipalBuilder is
> >>>> instantiated per processor per listener)
> >>>> In the voted proposal, I did not include enough tags for some plugins
> >>>> to uniquely identify each instance and avoid metric name collisions.
> >>>>
> >>>> For that reason, I updated the KIP to specify extra tags (in addition
> >>>> of "config" and "class") for the following plugin types:
> >>>> - KafkaPrincipalBuilder
> >>>> - Authorizer
> >>>> - ClientQuotaCallback
> >>>> - ConfigProvider
> >>>> - AuthenticateCallbackHandler
> >>>> - Login
> >>>> - SslEngineFactory
> >>>>
> >>>> You can find the exact tags in the Proposed Changes section of the
> KIP:
> >>>>
> >>>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
> >>>>
> >>>> Let me know if you have questions or concerns.
> >>>>
> >>>> Thanks,
> >>>> Mickael
> >>>>
> >>>>
> >>>> On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <
> mickael.maison@gmail.com>
> >>> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> While implementing the KIP I noticed the Kafka Connect specifics had
> >>>>> not been updated correctly based on the latest discussions. The
> >>>>> consensus was that connector and tasks should use their context to
> >>>>> retrieve the PluginMetrics instance if they want to register metrics.
> >>>>> While it's a slightly different mechanism than for the other plugins
> >>>>> (they need to implement Monitorable), this allows connector to
> support
> >>>>> this new feature and still work when deployed on an older Connect
> >>>>> runtime.
> >>>>>
> >>>>> I have updated the KIP and I have 2 PRs ready for review:
> >>>>> - Adding the APIs and support for producer and consumer plugins:
> >>>>> https://github.com/apache/kafka/pull/17511
> >>>>> - Adding support for Connect plugins:
> >>> https://github.com/apache/kafka/pull/17804
> >>>>>
> >>>>> Thanks,
> >>>>> Mickael
> >>>>>
> >>>>>
> >>>>> On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <
> >>> mickael.maison@gmail.com> wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> I'm closing the vote. The vote has passed with
> >>>>>> + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote
> >>>>>> from Hector.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Mickael
> >>>>>>
> >>>>>> On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com>
> wrote:
> >>>>>>>
> >>>>>>> Hi Mickael,
> >>>>>>>
> >>>>>>> Thanks for the KIP.
> >>>>>>> +1 (binding) from me.
> >>>>>>>
> >>>>>>> Thanks.
> >>>>>>> Luke
> >>>>>>>
> >>>>>>> On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com>
> >>> wrote:
> >>>>>>>
> >>>>>>>> +1 (binding).
> >>>>>>>>
> >>>>>>>> Thanks Mickael!
> >>>>>>>>
> >>>>>>>> On Thu, 1 Aug 2024 at 05:12, Mickael Maison <
> >>> mickael.maison@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Bumping this thread to get some more votes and/or feedback.
> >>>>>>>>>
> >>>>>>>>> As I restarted the vote on June 10 after major changes, I'm
> >>> only
> >>>>>>>>> counting votes since then.
> >>>>>>>>> So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Mickael
> >>>>>>>>>
> >>>>>>>>> On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/
> >>> 919 3RD A)
> >>>>>>>>> <hg...@bloomberg.net> wrote:
> >>>>>>>>>>
> >>>>>>>>>> This will help eliminate some boilerplate code we have for
> >>> our
> >>>>>>>>> connectors.
> >>>>>>>>>>
> >>>>>>>>>> +1 (non-binding)
> >>>>>>>>>>
> >>>>>>>>>> From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
> >>>>>>>>> dev@kafka.apache.org
> >>>>>>>>>> Subject: Re: [VOTE] KIP-877: Mechanism for plugins and
> >>> connectors to
> >>>>>>>>> register metrics
> >>>>>>>>>>
> >>>>>>>>>> Bumping this thread.
> >>>>>>>>>>
> >>>>>>>>>> Let me know if you have any feedback.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Mickael
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <
> >>> fearthecellos@gmail.com
> >>>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> +1 (binding), thanks Mickael!
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Jun 10, 2024, 04:24 Mickael Maison <
> >>> mickael.maison@gmail.com
> >>>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Following the feedback in the DISCUSS thread, I made
> >>> significant
> >>>>>>>>>>>> changes to the proposal. So I'd like to restart a vote
> >>> for KIP-877:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> >>>>>>>>>> ns+and+connectors+to+register+metrics
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Mickael
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <
> >>> tbentley@redhat.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Mickael,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> You'll have seen that I left some comments on the
> >>> discussion
> >>>>>>>>> thread, but
> >>>>>>>>>>>>> they're minor enough that I'm happy to vote +1 here.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Tom
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
> >>>>>>>>> mickael.maison@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Bumping this thread since I've not seen any feedback.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> Mickael
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> >>>>>>>>>>>>>> <mi...@gmail.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'd like to start a vote on KIP-877:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> >>>>>>>>>> ns+and+connectors+to+register+metrics
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Let me know if you have any feedback.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Mickael
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>
> >
>
>
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by 黃竣陽 <s7...@gmail.com>.
Hello,
I haven’t seen any feedback, so I’ll go ahead and implement the improvement
regarding PluginMetrics#metricName.
Best Regards,
Jiunn-Yang
> 黃竣陽 <s7...@gmail.com> 於 2025年4月14日 晚上11:43 寫道：
>
> Hi,
>
> There is an improvement in KIP-877.
>
> There will be an update to the PluginMetrics#metricName method: the type of the tags parameter will be changed
> from Map to LinkedHashMap.
> This change is necessary because the order of metric tags is important
> 1. If the tag order is inconsistent, identical metrics may be treated as distinct ones by the metrics backend
> 2. KAFKA-18390 is updating metric naming to use LinkedHashMap. For consistency, we should follow the same approach here.
>
> <https://cwiki.apache.org/confluence/x/lY3GDQ>
>
> Best Regards,
> Jiunn-Yang
>
>> Luke Chen <sh...@gmail.com> 於 2025年3月12日 上午10:21 寫道：
>>
>> HI Mickael,
>>
>> Thanks for the update.
>> The extra tags addition makes sense to me.
>>
>> Thanks.
>> Luke
>>
>> On Tue, Mar 11, 2025 at 8:31 PM Mickael Maison <mi...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I've not seen any concerns or feedback.
>>> I'll wait a few more days, then we'll carry on the implementation of
>>> KIP-877 with the updated proposal (extra tags for some server side
>>> plugin).
>>>
>>> Thanks,
>>> Mickael
>>>
>>> On Tue, Mar 4, 2025 at 6:57 PM Mickael Maison <mi...@gmail.com>
>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I found an issue while implementing KIP-877 for server plugins.
>>>>
>>>> Several server plugins can be instantiated multiple times. This can
>>> happen:
>>>> - when a server is running in combined mode (for example Authorizer is
>>>> instantiated by both brokers and controllers)
>>>> - when using multiple listeners (for example KafkaPrincipalBuilder is
>>>> instantiated per processor per listener)
>>>> In the voted proposal, I did not include enough tags for some plugins
>>>> to uniquely identify each instance and avoid metric name collisions.
>>>>
>>>> For that reason, I updated the KIP to specify extra tags (in addition
>>>> of "config" and "class") for the following plugin types:
>>>> - KafkaPrincipalBuilder
>>>> - Authorizer
>>>> - ClientQuotaCallback
>>>> - ConfigProvider
>>>> - AuthenticateCallbackHandler
>>>> - Login
>>>> - SslEngineFactory
>>>>
>>>> You can find the exact tags in the Proposed Changes section of the KIP:
>>>>
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
>>>>
>>>> Let me know if you have questions or concerns.
>>>>
>>>> Thanks,
>>>> Mickael
>>>>
>>>>
>>>> On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <mi...@gmail.com>
>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> While implementing the KIP I noticed the Kafka Connect specifics had
>>>>> not been updated correctly based on the latest discussions. The
>>>>> consensus was that connector and tasks should use their context to
>>>>> retrieve the PluginMetrics instance if they want to register metrics.
>>>>> While it's a slightly different mechanism than for the other plugins
>>>>> (they need to implement Monitorable), this allows connector to support
>>>>> this new feature and still work when deployed on an older Connect
>>>>> runtime.
>>>>>
>>>>> I have updated the KIP and I have 2 PRs ready for review:
>>>>> - Adding the APIs and support for producer and consumer plugins:
>>>>> https://github.com/apache/kafka/pull/17511
>>>>> - Adding support for Connect plugins:
>>> https://github.com/apache/kafka/pull/17804
>>>>>
>>>>> Thanks,
>>>>> Mickael
>>>>>
>>>>>
>>>>> On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <
>>> mickael.maison@gmail.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm closing the vote. The vote has passed with
>>>>>> + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote
>>>>>> from Hector.
>>>>>>
>>>>>> Thanks,
>>>>>> Mickael
>>>>>>
>>>>>> On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi Mickael,
>>>>>>>
>>>>>>> Thanks for the KIP.
>>>>>>> +1 (binding) from me.
>>>>>>>
>>>>>>> Thanks.
>>>>>>> Luke
>>>>>>>
>>>>>>> On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com>
>>> wrote:
>>>>>>>
>>>>>>>> +1 (binding).
>>>>>>>>
>>>>>>>> Thanks Mickael!
>>>>>>>>
>>>>>>>> On Thu, 1 Aug 2024 at 05:12, Mickael Maison <
>>> mickael.maison@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Bumping this thread to get some more votes and/or feedback.
>>>>>>>>>
>>>>>>>>> As I restarted the vote on June 10 after major changes, I'm
>>> only
>>>>>>>>> counting votes since then.
>>>>>>>>> So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Mickael
>>>>>>>>>
>>>>>>>>> On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/
>>> 919 3RD A)
>>>>>>>>> <hg...@bloomberg.net> wrote:
>>>>>>>>>>
>>>>>>>>>> This will help eliminate some boilerplate code we have for
>>> our
>>>>>>>>> connectors.
>>>>>>>>>>
>>>>>>>>>> +1 (non-binding)
>>>>>>>>>>
>>>>>>>>>> From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
>>>>>>>>> dev@kafka.apache.org
>>>>>>>>>> Subject: Re: [VOTE] KIP-877: Mechanism for plugins and
>>> connectors to
>>>>>>>>> register metrics
>>>>>>>>>>
>>>>>>>>>> Bumping this thread.
>>>>>>>>>>
>>>>>>>>>> Let me know if you have any feedback.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Mickael
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <
>>> fearthecellos@gmail.com
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> +1 (binding), thanks Mickael!
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jun 10, 2024, 04:24 Mickael Maison <
>>> mickael.maison@gmail.com
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Following the feedback in the DISCUSS thread, I made
>>> significant
>>>>>>>>>>>> changes to the proposal. So I'd like to restart a vote
>>> for KIP-877:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
>>>>>>>>>> ns+and+connectors+to+register+metrics
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Mickael
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <
>>> tbentley@redhat.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Mickael,
>>>>>>>>>>>>>
>>>>>>>>>>>>> You'll have seen that I left some comments on the
>>> discussion
>>>>>>>>> thread, but
>>>>>>>>>>>>> they're minor enough that I'm happy to vote +1 here.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Tom
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
>>>>>>>>> mickael.maison@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Bumping this thread since I've not seen any feedback.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
>>>>>>>>>>>>>> <mi...@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'd like to start a vote on KIP-877:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
>>>>>>>>>> ns+and+connectors+to+register+metrics
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Let me know if you have any feedback.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>
>
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by 黃竣陽 <s7...@gmail.com>.
Hi,
There is an improvement in KIP-877.
There will be an update to the PluginMetrics#metricName method: the type of the tags parameter will be changed
from Map to LinkedHashMap.
This change is necessary because the order of metric tags is important
1. If the tag order is inconsistent, identical metrics may be treated as distinct ones by the metrics backend
2. KAFKA-18390 is updating metric naming to use LinkedHashMap. For consistency, we should follow the same approach here.
<https://cwiki.apache.org/confluence/x/lY3GDQ>
Best Regards,
Jiunn-Yang
> Luke Chen <sh...@gmail.com> 於 2025年3月12日 上午10:21 寫道：
>
> HI Mickael,
>
> Thanks for the update.
> The extra tags addition makes sense to me.
>
> Thanks.
> Luke
>
> On Tue, Mar 11, 2025 at 8:31 PM Mickael Maison <mi...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I've not seen any concerns or feedback.
>> I'll wait a few more days, then we'll carry on the implementation of
>> KIP-877 with the updated proposal (extra tags for some server side
>> plugin).
>>
>> Thanks,
>> Mickael
>>
>> On Tue, Mar 4, 2025 at 6:57 PM Mickael Maison <mi...@gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> I found an issue while implementing KIP-877 for server plugins.
>>>
>>> Several server plugins can be instantiated multiple times. This can
>> happen:
>>> - when a server is running in combined mode (for example Authorizer is
>>> instantiated by both brokers and controllers)
>>> - when using multiple listeners (for example KafkaPrincipalBuilder is
>>> instantiated per processor per listener)
>>> In the voted proposal, I did not include enough tags for some plugins
>>> to uniquely identify each instance and avoid metric name collisions.
>>>
>>> For that reason, I updated the KIP to specify extra tags (in addition
>>> of "config" and "class") for the following plugin types:
>>> - KafkaPrincipalBuilder
>>> - Authorizer
>>> - ClientQuotaCallback
>>> - ConfigProvider
>>> - AuthenticateCallbackHandler
>>> - Login
>>> - SslEngineFactory
>>>
>>> You can find the exact tags in the Proposed Changes section of the KIP:
>>>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
>>>
>>> Let me know if you have questions or concerns.
>>>
>>> Thanks,
>>> Mickael
>>>
>>>
>>> On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <mi...@gmail.com>
>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> While implementing the KIP I noticed the Kafka Connect specifics had
>>>> not been updated correctly based on the latest discussions. The
>>>> consensus was that connector and tasks should use their context to
>>>> retrieve the PluginMetrics instance if they want to register metrics.
>>>> While it's a slightly different mechanism than for the other plugins
>>>> (they need to implement Monitorable), this allows connector to support
>>>> this new feature and still work when deployed on an older Connect
>>>> runtime.
>>>>
>>>> I have updated the KIP and I have 2 PRs ready for review:
>>>> - Adding the APIs and support for producer and consumer plugins:
>>>> https://github.com/apache/kafka/pull/17511
>>>> - Adding support for Connect plugins:
>> https://github.com/apache/kafka/pull/17804
>>>>
>>>> Thanks,
>>>> Mickael
>>>>
>>>>
>>>> On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <
>> mickael.maison@gmail.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm closing the vote. The vote has passed with
>>>>> + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote
>>>>> from Hector.
>>>>>
>>>>> Thanks,
>>>>> Mickael
>>>>>
>>>>> On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com> wrote:
>>>>>>
>>>>>> Hi Mickael,
>>>>>>
>>>>>> Thanks for the KIP.
>>>>>> +1 (binding) from me.
>>>>>>
>>>>>> Thanks.
>>>>>> Luke
>>>>>>
>>>>>> On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com>
>> wrote:
>>>>>>
>>>>>>> +1 (binding).
>>>>>>>
>>>>>>> Thanks Mickael!
>>>>>>>
>>>>>>> On Thu, 1 Aug 2024 at 05:12, Mickael Maison <
>> mickael.maison@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Bumping this thread to get some more votes and/or feedback.
>>>>>>>>
>>>>>>>> As I restarted the vote on June 10 after major changes, I'm
>> only
>>>>>>>> counting votes since then.
>>>>>>>> So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Mickael
>>>>>>>>
>>>>>>>> On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/
>> 919 3RD A)
>>>>>>>> <hg...@bloomberg.net> wrote:
>>>>>>>>>
>>>>>>>>> This will help eliminate some boilerplate code we have for
>> our
>>>>>>>> connectors.
>>>>>>>>>
>>>>>>>>> +1 (non-binding)
>>>>>>>>>
>>>>>>>>> From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
>>>>>>>> dev@kafka.apache.org
>>>>>>>>> Subject: Re: [VOTE] KIP-877: Mechanism for plugins and
>> connectors to
>>>>>>>> register metrics
>>>>>>>>>
>>>>>>>>> Bumping this thread.
>>>>>>>>>
>>>>>>>>> Let me know if you have any feedback.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Mickael
>>>>>>>>>
>>>>>>>>> On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <
>> fearthecellos@gmail.com
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> +1 (binding), thanks Mickael!
>>>>>>>>>>
>>>>>>>>>> On Mon, Jun 10, 2024, 04:24 Mickael Maison <
>> mickael.maison@gmail.com
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Following the feedback in the DISCUSS thread, I made
>> significant
>>>>>>>>>>> changes to the proposal. So I'd like to restart a vote
>> for KIP-877:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
>>>>>>>>> ns+and+connectors+to+register+metrics
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Mickael
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <
>> tbentley@redhat.com>
>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Mickael,
>>>>>>>>>>>>
>>>>>>>>>>>> You'll have seen that I left some comments on the
>> discussion
>>>>>>>> thread, but
>>>>>>>>>>>> they're minor enough that I'm happy to vote +1 here.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> Tom
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
>>>>>>>> mickael.maison@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Bumping this thread since I've not seen any feedback.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
>>>>>>>>>>>>> <mi...@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'd like to start a vote on KIP-877:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
>>>>>>>>> ns+and+connectors+to+register+metrics
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Let me know if you have any feedback.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Mickael
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Luke Chen <sh...@gmail.com>.
HI Mickael,
Thanks for the update.
The extra tags addition makes sense to me.
Thanks.
Luke
On Tue, Mar 11, 2025 at 8:31 PM Mickael Maison <mi...@gmail.com>
wrote:
> Hi,
>
> I've not seen any concerns or feedback.
> I'll wait a few more days, then we'll carry on the implementation of
> KIP-877 with the updated proposal (extra tags for some server side
> plugin).
>
> Thanks,
> Mickael
>
> On Tue, Mar 4, 2025 at 6:57 PM Mickael Maison <mi...@gmail.com>
> wrote:
> >
> > Hi,
> >
> > I found an issue while implementing KIP-877 for server plugins.
> >
> > Several server plugins can be instantiated multiple times. This can
> happen:
> > - when a server is running in combined mode (for example Authorizer is
> > instantiated by both brokers and controllers)
> > - when using multiple listeners (for example KafkaPrincipalBuilder is
> > instantiated per processor per listener)
> > In the voted proposal, I did not include enough tags for some plugins
> > to uniquely identify each instance and avoid metric name collisions.
> >
> > For that reason, I updated the KIP to specify extra tags (in addition
> > of "config" and "class") for the following plugin types:
> > - KafkaPrincipalBuilder
> > - Authorizer
> > - ClientQuotaCallback
> > - ConfigProvider
> > - AuthenticateCallbackHandler
> > - Login
> > - SslEngineFactory
> >
> > You can find the exact tags in the Proposed Changes section of the KIP:
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
> >
> > Let me know if you have questions or concerns.
> >
> > Thanks,
> > Mickael
> >
> >
> > On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <mi...@gmail.com>
> wrote:
> > >
> > > Hi,
> > >
> > > While implementing the KIP I noticed the Kafka Connect specifics had
> > > not been updated correctly based on the latest discussions. The
> > > consensus was that connector and tasks should use their context to
> > > retrieve the PluginMetrics instance if they want to register metrics.
> > > While it's a slightly different mechanism than for the other plugins
> > > (they need to implement Monitorable), this allows connector to support
> > > this new feature and still work when deployed on an older Connect
> > > runtime.
> > >
> > > I have updated the KIP and I have 2 PRs ready for review:
> > > - Adding the APIs and support for producer and consumer plugins:
> > > https://github.com/apache/kafka/pull/17511
> > > - Adding support for Connect plugins:
> https://github.com/apache/kafka/pull/17804
> > >
> > > Thanks,
> > > Mickael
> > >
> > >
> > > On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <
> mickael.maison@gmail.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I'm closing the vote. The vote has passed with
> > > > + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote
> > > > from Hector.
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
> > > > On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com> wrote:
> > > > >
> > > > > Hi Mickael,
> > > > >
> > > > > Thanks for the KIP.
> > > > > +1 (binding) from me.
> > > > >
> > > > > Thanks.
> > > > > Luke
> > > > >
> > > > > On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com>
> wrote:
> > > > >
> > > > > > +1 (binding).
> > > > > >
> > > > > > Thanks Mickael!
> > > > > >
> > > > > > On Thu, 1 Aug 2024 at 05:12, Mickael Maison <
> mickael.maison@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Bumping this thread to get some more votes and/or feedback.
> > > > > > >
> > > > > > > As I restarted the vote on June 10 after major changes, I'm
> only
> > > > > > > counting votes since then.
> > > > > > > So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Mickael
> > > > > > >
> > > > > > > On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/
> 919 3RD A)
> > > > > > > <hg...@bloomberg.net> wrote:
> > > > > > > >
> > > > > > > > This will help eliminate some boilerplate code we have for
> our
> > > > > > > connectors.
> > > > > > > >
> > > > > > > > +1 (non-binding)
> > > > > > > >
> > > > > > > > From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
> > > > > > > dev@kafka.apache.org
> > > > > > > > Subject: Re: [VOTE] KIP-877: Mechanism for plugins and
> connectors to
> > > > > > > register metrics
> > > > > > > >
> > > > > > > > Bumping this thread.
> > > > > > > >
> > > > > > > > Let me know if you have any feedback.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Mickael
> > > > > > > >
> > > > > > > > On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <
> fearthecellos@gmail.com
> > > > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > +1 (binding), thanks Mickael!
> > > > > > > > >
> > > > > > > > > On Mon, Jun 10, 2024, 04:24 Mickael Maison <
> mickael.maison@gmail.com
> > > > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > Following the feedback in the DISCUSS thread, I made
> significant
> > > > > > > > > > changes to the proposal. So I'd like to restart a vote
> for KIP-877:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > > > > > > ns+and+connectors+to+register+metrics
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Mickael
> > > > > > > > > >
> > > > > > > > > > On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <
> tbentley@redhat.com>
> > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi Mickael,
> > > > > > > > > > >
> > > > > > > > > > > You'll have seen that I left some comments on the
> discussion
> > > > > > > thread, but
> > > > > > > > > > > they're minor enough that I'm happy to vote +1 here.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > >
> > > > > > > > > > > Tom
> > > > > > > > > > >
> > > > > > > > > > > On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
> > > > > > > mickael.maison@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Bumping this thread since I've not seen any feedback.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > Mickael
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> > > > > > > > > > > > <mi...@gmail.com> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hi,
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'd like to start a vote on KIP-877:
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > > > > > > ns+and+connectors+to+register+metrics
> > > > > > > > > > > > >
> > > > > > > > > > > > > Let me know if you have any feedback.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > Mickael
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
>
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Mickael Maison <mi...@gmail.com>.
Hi,
I've not seen any concerns or feedback.
I'll wait a few more days, then we'll carry on the implementation of
KIP-877 with the updated proposal (extra tags for some server side
plugin).
Thanks,
Mickael
On Tue, Mar 4, 2025 at 6:57 PM Mickael Maison <mi...@gmail.com> wrote:
>
> Hi,
>
> I found an issue while implementing KIP-877 for server plugins.
>
> Several server plugins can be instantiated multiple times. This can happen:
> - when a server is running in combined mode (for example Authorizer is
> instantiated by both brokers and controllers)
> - when using multiple listeners (for example KafkaPrincipalBuilder is
> instantiated per processor per listener)
> In the voted proposal, I did not include enough tags for some plugins
> to uniquely identify each instance and avoid metric name collisions.
>
> For that reason, I updated the KIP to specify extra tags (in addition
> of "config" and "class") for the following plugin types:
> - KafkaPrincipalBuilder
> - Authorizer
> - ClientQuotaCallback
> - ConfigProvider
> - AuthenticateCallbackHandler
> - Login
> - SslEngineFactory
>
> You can find the exact tags in the Proposed Changes section of the KIP:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
>
> Let me know if you have questions or concerns.
>
> Thanks,
> Mickael
>
>
> On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <mi...@gmail.com> wrote:
> >
> > Hi,
> >
> > While implementing the KIP I noticed the Kafka Connect specifics had
> > not been updated correctly based on the latest discussions. The
> > consensus was that connector and tasks should use their context to
> > retrieve the PluginMetrics instance if they want to register metrics.
> > While it's a slightly different mechanism than for the other plugins
> > (they need to implement Monitorable), this allows connector to support
> > this new feature and still work when deployed on an older Connect
> > runtime.
> >
> > I have updated the KIP and I have 2 PRs ready for review:
> > - Adding the APIs and support for producer and consumer plugins:
> > https://github.com/apache/kafka/pull/17511
> > - Adding support for Connect plugins: https://github.com/apache/kafka/pull/17804
> >
> > Thanks,
> > Mickael
> >
> >
> > On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <mi...@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > I'm closing the vote. The vote has passed with
> > > + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote
> > > from Hector.
> > >
> > > Thanks,
> > > Mickael
> > >
> > > On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com> wrote:
> > > >
> > > > Hi Mickael,
> > > >
> > > > Thanks for the KIP.
> > > > +1 (binding) from me.
> > > >
> > > > Thanks.
> > > > Luke
> > > >
> > > > On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com> wrote:
> > > >
> > > > > +1 (binding).
> > > > >
> > > > > Thanks Mickael!
> > > > >
> > > > > On Thu, 1 Aug 2024 at 05:12, Mickael Maison <mi...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Bumping this thread to get some more votes and/or feedback.
> > > > > >
> > > > > > As I restarted the vote on June 10 after major changes, I'm only
> > > > > > counting votes since then.
> > > > > > So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
> > > > > >
> > > > > > Thanks,
> > > > > > Mickael
> > > > > >
> > > > > > On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/ 919 3RD A)
> > > > > > <hg...@bloomberg.net> wrote:
> > > > > > >
> > > > > > > This will help eliminate some boilerplate code we have for our
> > > > > > connectors.
> > > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
> > > > > > dev@kafka.apache.org
> > > > > > > Subject: Re: [VOTE] KIP-877: Mechanism for plugins and connectors to
> > > > > > register metrics
> > > > > > >
> > > > > > > Bumping this thread.
> > > > > > >
> > > > > > > Let me know if you have any feedback.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Mickael
> > > > > > >
> > > > > > > On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <fearthecellos@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > +1 (binding), thanks Mickael!
> > > > > > > >
> > > > > > > > On Mon, Jun 10, 2024, 04:24 Mickael Maison <mickael.maison@gmail.com
> > > > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > Following the feedback in the DISCUSS thread, I made significant
> > > > > > > > > changes to the proposal. So I'd like to restart a vote for KIP-877:
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > > > > > ns+and+connectors+to+register+metrics
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Mickael
> > > > > > > > >
> > > > > > > > > On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <tb...@redhat.com>
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hi Mickael,
> > > > > > > > > >
> > > > > > > > > > You'll have seen that I left some comments on the discussion
> > > > > > thread, but
> > > > > > > > > > they're minor enough that I'm happy to vote +1 here.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > Tom
> > > > > > > > > >
> > > > > > > > > > On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
> > > > > > mickael.maison@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Bumping this thread since I've not seen any feedback.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Mickael
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> > > > > > > > > > > <mi...@gmail.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi,
> > > > > > > > > > > >
> > > > > > > > > > > > I'd like to start a vote on KIP-877:
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > > > > > ns+and+connectors+to+register+metrics
> > > > > > > > > > > >
> > > > > > > > > > > > Let me know if you have any feedback.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > Mickael
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Mickael Maison <mi...@gmail.com>.
Hi,
I found an issue while implementing KIP-877 for server plugins.
Several server plugins can be instantiated multiple times. This can happen:
- when a server is running in combined mode (for example Authorizer is
instantiated by both brokers and controllers)
- when using multiple listeners (for example KafkaPrincipalBuilder is
instantiated per processor per listener)
In the voted proposal, I did not include enough tags for some plugins
to uniquely identify each instance and avoid metric name collisions.
For that reason, I updated the KIP to specify extra tags (in addition
of "config" and "class") for the following plugin types:
- KafkaPrincipalBuilder
- Authorizer
- ClientQuotaCallback
- ConfigProvider
- AuthenticateCallbackHandler
- Login
- SslEngineFactory
You can find the exact tags in the Proposed Changes section of the KIP:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=231116181#KIP877:Mechanismforpluginsandconnectorstoregistermetrics-ProposedChanges
Let me know if you have questions or concerns.
Thanks,
Mickael
On Thu, Nov 14, 2024 at 3:07 PM Mickael Maison <mi...@gmail.com> wrote:
>
> Hi,
>
> While implementing the KIP I noticed the Kafka Connect specifics had
> not been updated correctly based on the latest discussions. The
> consensus was that connector and tasks should use their context to
> retrieve the PluginMetrics instance if they want to register metrics.
> While it's a slightly different mechanism than for the other plugins
> (they need to implement Monitorable), this allows connector to support
> this new feature and still work when deployed on an older Connect
> runtime.
>
> I have updated the KIP and I have 2 PRs ready for review:
> - Adding the APIs and support for producer and consumer plugins:
> https://github.com/apache/kafka/pull/17511
> - Adding support for Connect plugins: https://github.com/apache/kafka/pull/17804
>
> Thanks,
> Mickael
>
>
> On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <mi...@gmail.com> wrote:
> >
> > Hi,
> >
> > I'm closing the vote. The vote has passed with
> > + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote
> > from Hector.
> >
> > Thanks,
> > Mickael
> >
> > On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com> wrote:
> > >
> > > Hi Mickael,
> > >
> > > Thanks for the KIP.
> > > +1 (binding) from me.
> > >
> > > Thanks.
> > > Luke
> > >
> > > On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com> wrote:
> > >
> > > > +1 (binding).
> > > >
> > > > Thanks Mickael!
> > > >
> > > > On Thu, 1 Aug 2024 at 05:12, Mickael Maison <mi...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Bumping this thread to get some more votes and/or feedback.
> > > > >
> > > > > As I restarted the vote on June 10 after major changes, I'm only
> > > > > counting votes since then.
> > > > > So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
> > > > >
> > > > > Thanks,
> > > > > Mickael
> > > > >
> > > > > On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/ 919 3RD A)
> > > > > <hg...@bloomberg.net> wrote:
> > > > > >
> > > > > > This will help eliminate some boilerplate code we have for our
> > > > > connectors.
> > > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
> > > > > dev@kafka.apache.org
> > > > > > Subject: Re: [VOTE] KIP-877: Mechanism for plugins and connectors to
> > > > > register metrics
> > > > > >
> > > > > > Bumping this thread.
> > > > > >
> > > > > > Let me know if you have any feedback.
> > > > > >
> > > > > > Thanks,
> > > > > > Mickael
> > > > > >
> > > > > > On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <fearthecellos@gmail.com
> > > > >
> > > > > wrote:
> > > > > > >
> > > > > > > +1 (binding), thanks Mickael!
> > > > > > >
> > > > > > > On Mon, Jun 10, 2024, 04:24 Mickael Maison <mickael.maison@gmail.com
> > > > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > Following the feedback in the DISCUSS thread, I made significant
> > > > > > > > changes to the proposal. So I'd like to restart a vote for KIP-877:
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > > > > ns+and+connectors+to+register+metrics
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Mickael
> > > > > > > >
> > > > > > > > On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <tb...@redhat.com>
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > Hi Mickael,
> > > > > > > > >
> > > > > > > > > You'll have seen that I left some comments on the discussion
> > > > > thread, but
> > > > > > > > > they're minor enough that I'm happy to vote +1 here.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Tom
> > > > > > > > >
> > > > > > > > > On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
> > > > > mickael.maison@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Bumping this thread since I've not seen any feedback.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Mickael
> > > > > > > > > >
> > > > > > > > > > On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> > > > > > > > > > <mi...@gmail.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > I'd like to start a vote on KIP-877:
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > > > > ns+and+connectors+to+register+metrics
> > > > > > > > > > >
> > > > > > > > > > > Let me know if you have any feedback.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > Mickael
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Mickael Maison <mi...@gmail.com>.
Hi,
While implementing the KIP I noticed the Kafka Connect specifics had
not been updated correctly based on the latest discussions. The
consensus was that connector and tasks should use their context to
retrieve the PluginMetrics instance if they want to register metrics.
While it's a slightly different mechanism than for the other plugins
(they need to implement Monitorable), this allows connector to support
this new feature and still work when deployed on an older Connect
runtime.
I have updated the KIP and I have 2 PRs ready for review:
- Adding the APIs and support for producer and consumer plugins:
https://github.com/apache/kafka/pull/17511
- Adding support for Connect plugins: https://github.com/apache/kafka/pull/17804
Thanks,
Mickael
On Thu, Aug 8, 2024 at 2:21 PM Mickael Maison <mi...@gmail.com> wrote:
>
> Hi,
>
> I'm closing the vote. The vote has passed with
> + 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote
> from Hector.
>
> Thanks,
> Mickael
>
> On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com> wrote:
> >
> > Hi Mickael,
> >
> > Thanks for the KIP.
> > +1 (binding) from me.
> >
> > Thanks.
> > Luke
> >
> > On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com> wrote:
> >
> > > +1 (binding).
> > >
> > > Thanks Mickael!
> > >
> > > On Thu, 1 Aug 2024 at 05:12, Mickael Maison <mi...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Bumping this thread to get some more votes and/or feedback.
> > > >
> > > > As I restarted the vote on June 10 after major changes, I'm only
> > > > counting votes since then.
> > > > So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
> > > > On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/ 919 3RD A)
> > > > <hg...@bloomberg.net> wrote:
> > > > >
> > > > > This will help eliminate some boilerplate code we have for our
> > > > connectors.
> > > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
> > > > dev@kafka.apache.org
> > > > > Subject: Re: [VOTE] KIP-877: Mechanism for plugins and connectors to
> > > > register metrics
> > > > >
> > > > > Bumping this thread.
> > > > >
> > > > > Let me know if you have any feedback.
> > > > >
> > > > > Thanks,
> > > > > Mickael
> > > > >
> > > > > On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <fearthecellos@gmail.com
> > > >
> > > > wrote:
> > > > > >
> > > > > > +1 (binding), thanks Mickael!
> > > > > >
> > > > > > On Mon, Jun 10, 2024, 04:24 Mickael Maison <mickael.maison@gmail.com
> > > >
> > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Following the feedback in the DISCUSS thread, I made significant
> > > > > > > changes to the proposal. So I'd like to restart a vote for KIP-877:
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > > > ns+and+connectors+to+register+metrics
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Mickael
> > > > > > >
> > > > > > > On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <tb...@redhat.com>
> > > > wrote:
> > > > > > > >
> > > > > > > > Hi Mickael,
> > > > > > > >
> > > > > > > > You'll have seen that I left some comments on the discussion
> > > > thread, but
> > > > > > > > they're minor enough that I'm happy to vote +1 here.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Tom
> > > > > > > >
> > > > > > > > On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
> > > > mickael.maison@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Bumping this thread since I've not seen any feedback.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Mickael
> > > > > > > > >
> > > > > > > > > On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> > > > > > > > > <mi...@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > I'd like to start a vote on KIP-877:
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > > > ns+and+connectors+to+register+metrics
> > > > > > > > > >
> > > > > > > > > > Let me know if you have any feedback.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Mickael
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Mickael Maison <mi...@gmail.com>.
Hi,
I'm closing the vote. The vote has passed with
+ 3 bindings votes from Chris, Tom and Luke, and +1 non-binding vote
from Hector.
Thanks,
Mickael
On Thu, Aug 8, 2024 at 11:28 AM Luke Chen <sh...@gmail.com> wrote:
>
> Hi Mickael,
>
> Thanks for the KIP.
> +1 (binding) from me.
>
> Thanks.
> Luke
>
> On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com> wrote:
>
> > +1 (binding).
> >
> > Thanks Mickael!
> >
> > On Thu, 1 Aug 2024 at 05:12, Mickael Maison <mi...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Bumping this thread to get some more votes and/or feedback.
> > >
> > > As I restarted the vote on June 10 after major changes, I'm only
> > > counting votes since then.
> > > So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
> > >
> > > Thanks,
> > > Mickael
> > >
> > > On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/ 919 3RD A)
> > > <hg...@bloomberg.net> wrote:
> > > >
> > > > This will help eliminate some boilerplate code we have for our
> > > connectors.
> > > >
> > > > +1 (non-binding)
> > > >
> > > > From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
> > > dev@kafka.apache.org
> > > > Subject: Re: [VOTE] KIP-877: Mechanism for plugins and connectors to
> > > register metrics
> > > >
> > > > Bumping this thread.
> > > >
> > > > Let me know if you have any feedback.
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
> > > > On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <fearthecellos@gmail.com
> > >
> > > wrote:
> > > > >
> > > > > +1 (binding), thanks Mickael!
> > > > >
> > > > > On Mon, Jun 10, 2024, 04:24 Mickael Maison <mickael.maison@gmail.com
> > >
> > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Following the feedback in the DISCUSS thread, I made significant
> > > > > > changes to the proposal. So I'd like to restart a vote for KIP-877:
> > > > > >
> > > > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > > ns+and+connectors+to+register+metrics
> > > > > >
> > > > > > Thanks,
> > > > > > Mickael
> > > > > >
> > > > > > On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <tb...@redhat.com>
> > > wrote:
> > > > > > >
> > > > > > > Hi Mickael,
> > > > > > >
> > > > > > > You'll have seen that I left some comments on the discussion
> > > thread, but
> > > > > > > they're minor enough that I'm happy to vote +1 here.
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Tom
> > > > > > >
> > > > > > > On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
> > > mickael.maison@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Bumping this thread since I've not seen any feedback.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Mickael
> > > > > > > >
> > > > > > > > On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> > > > > > > > <mi...@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > I'd like to start a vote on KIP-877:
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > > ns+and+connectors+to+register+metrics
> > > > > > > > >
> > > > > > > > > Let me know if you have any feedback.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Mickael
> > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > > >
> > >
> > >
> >
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Luke Chen <sh...@gmail.com>.
Hi Mickael,
Thanks for the KIP.
+1 (binding) from me.
Thanks.
Luke
On Fri, Aug 2, 2024 at 4:57 AM Tom Bentley <tb...@redhat.com> wrote:
> +1 (binding).
>
> Thanks Mickael!
>
> On Thu, 1 Aug 2024 at 05:12, Mickael Maison <mi...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Bumping this thread to get some more votes and/or feedback.
> >
> > As I restarted the vote on June 10 after major changes, I'm only
> > counting votes since then.
> > So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
> >
> > Thanks,
> > Mickael
> >
> > On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/ 919 3RD A)
> > <hg...@bloomberg.net> wrote:
> > >
> > > This will help eliminate some boilerplate code we have for our
> > connectors.
> > >
> > > +1 (non-binding)
> > >
> > > From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
> > dev@kafka.apache.org
> > > Subject: Re: [VOTE] KIP-877: Mechanism for plugins and connectors to
> > register metrics
> > >
> > > Bumping this thread.
> > >
> > > Let me know if you have any feedback.
> > >
> > > Thanks,
> > > Mickael
> > >
> > > On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <fearthecellos@gmail.com
> >
> > wrote:
> > > >
> > > > +1 (binding), thanks Mickael!
> > > >
> > > > On Mon, Jun 10, 2024, 04:24 Mickael Maison <mickael.maison@gmail.com
> >
> > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Following the feedback in the DISCUSS thread, I made significant
> > > > > changes to the proposal. So I'd like to restart a vote for KIP-877:
> > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > ns+and+connectors+to+register+metrics
> > > > >
> > > > > Thanks,
> > > > > Mickael
> > > > >
> > > > > On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <tb...@redhat.com>
> > wrote:
> > > > > >
> > > > > > Hi Mickael,
> > > > > >
> > > > > > You'll have seen that I left some comments on the discussion
> > thread, but
> > > > > > they're minor enough that I'm happy to vote +1 here.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Tom
> > > > > >
> > > > > > On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
> > mickael.maison@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Bumping this thread since I've not seen any feedback.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Mickael
> > > > > > >
> > > > > > > On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> > > > > > > <mi...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I'd like to start a vote on KIP-877:
> > > > > > > >
> > > > > > >
> > > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > > ns+and+connectors+to+register+metrics
> > > > > > > >
> > > > > > > > Let me know if you have any feedback.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Mickael
> > > > > > >
> > > > > > >
> > > > >
> > >
> > >
> >
> >
>
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Tom Bentley <tb...@redhat.com>.
+1 (binding).
Thanks Mickael!
On Thu, 1 Aug 2024 at 05:12, Mickael Maison <mi...@gmail.com>
wrote:
> Hi,
>
> Bumping this thread to get some more votes and/or feedback.
>
> As I restarted the vote on June 10 after major changes, I'm only
> counting votes since then.
> So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
>
> Thanks,
> Mickael
>
> On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/ 919 3RD A)
> <hg...@bloomberg.net> wrote:
> >
> > This will help eliminate some boilerplate code we have for our
> connectors.
> >
> > +1 (non-binding)
> >
> > From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To:
> dev@kafka.apache.org
> > Subject: Re: [VOTE] KIP-877: Mechanism for plugins and connectors to
> register metrics
> >
> > Bumping this thread.
> >
> > Let me know if you have any feedback.
> >
> > Thanks,
> > Mickael
> >
> > On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <fe...@gmail.com>
> wrote:
> > >
> > > +1 (binding), thanks Mickael!
> > >
> > > On Mon, Jun 10, 2024, 04:24 Mickael Maison <mi...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > Following the feedback in the DISCUSS thread, I made significant
> > > > changes to the proposal. So I'd like to restart a vote for KIP-877:
> > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > ns+and+connectors+to+register+metrics
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
> > > > On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <tb...@redhat.com>
> wrote:
> > > > >
> > > > > Hi Mickael,
> > > > >
> > > > > You'll have seen that I left some comments on the discussion
> thread, but
> > > > > they're minor enough that I'm happy to vote +1 here.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Tom
> > > > >
> > > > > On Thu, 11 Jan 2024 at 06:14, Mickael Maison <
> mickael.maison@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Bumping this thread since I've not seen any feedback.
> > > > > >
> > > > > > Thanks,
> > > > > > Mickael
> > > > > >
> > > > > > On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> > > > > > <mi...@gmail.com> wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I'd like to start a vote on KIP-877:
> > > > > > >
> > > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> > ns+and+connectors+to+register+metrics
> > > > > > >
> > > > > > > Let me know if you have any feedback.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Mickael
> > > > > >
> > > > > >
> > > >
> >
> >
>
>
Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
Posted by Mickael Maison <mi...@gmail.com>.
Hi,
Bumping this thread to get some more votes and/or feedback.
As I restarted the vote on June 10 after major changes, I'm only
counting votes since then.
So we have 1 binding (Chris) and 1 non-binding (Hector) votes.
Thanks,
Mickael
On Mon, Jul 8, 2024 at 8:40 PM Hector Geraldino (BLOOMBERG/ 919 3RD A)
<hg...@bloomberg.net> wrote:
>
> This will help eliminate some boilerplate code we have for our connectors.
>
> +1 (non-binding)
>
> From: dev@kafka.apache.org At: 06/25/24 04:30:27 UTC-4:00To: dev@kafka.apache.org
> Subject: Re: [VOTE] KIP-877: Mechanism for plugins and connectors to register metrics
>
> Bumping this thread.
>
> Let me know if you have any feedback.
>
> Thanks,
> Mickael
>
> On Mon, Jun 10, 2024 at 1:44 PM Chris Egerton <fe...@gmail.com> wrote:
> >
> > +1 (binding), thanks Mickael!
> >
> > On Mon, Jun 10, 2024, 04:24 Mickael Maison <mi...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > Following the feedback in the DISCUSS thread, I made significant
> > > changes to the proposal. So I'd like to restart a vote for KIP-877:
> > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> ns+and+connectors+to+register+metrics
> > >
> > > Thanks,
> > > Mickael
> > >
> > > On Thu, Jan 25, 2024 at 2:59 AM Tom Bentley <tb...@redhat.com> wrote:
> > > >
> > > > Hi Mickael,
> > > >
> > > > You'll have seen that I left some comments on the discussion thread, but
> > > > they're minor enough that I'm happy to vote +1 here.
> > > >
> > > > Thanks,
> > > >
> > > > Tom
> > > >
> > > > On Thu, 11 Jan 2024 at 06:14, Mickael Maison <mi...@gmail.com>
> > > > wrote:
> > > >
> > > > > Bumping this thread since I've not seen any feedback.
> > > > >
> > > > > Thanks,
> > > > > Mickael
> > > > >
> > > > > On Tue, Dec 19, 2023 at 10:03 AM Mickael Maison
> > > > > <mi...@gmail.com> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'd like to start a vote on KIP-877:
> > > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-877%3A+Mechanism+for+plugi
> ns+and+connectors+to+register+metrics
> > > > > >
> > > > > > Let me know if you have any feedback.
> > > > > >
> > > > > > Thanks,
> > > > > > Mickael
> > > > >
> > > > >
> > >
>
>