You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Andrew Schofield <an...@outlook.com> on 2023/11/07 17:41:55 UTC
[DISCUSS] KIP-1000: List Client Metrics Configuration Resources
Hi,
I would like to start discussion of a small KIP which fills a gap in the administration of client metrics configuration.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources
Thanks,
Andrew
Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources
Posted by Adam Bellemare <ad...@gmail.com>.
Let's take this moment to celebrate the creation of KIP-1000!
On Tue, Nov 7, 2023 at 3:49 PM Andrew Schofield <
andrew_schofield_jira@outlook.com> wrote:
> Hi,
> I would like to start discussion of a small KIP which fills a gap in the
> administration of client metrics configuration.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources
>
> Thanks,
> Andrew
Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources
Posted by Jun Rao <ju...@confluent.io.INVALID>.
Hi, Andrew,
Thanks for the reply. LGTM.
Jun
On Sat, Nov 18, 2023 at 12:28 AM Andrew Schofield <
andrew_schofield_jira@outlook.com> wrote:
> Hi Jun,
> This is an example of inconsistency between the tools. I do agree that
> it would be nice to have `--list` on the `kafka-client-metrics.sh` tool.
> However, `kafka-config.sh` has the convention that `--describe` with no
> entity name supplied first gets a list of all the entities, and then
> describes
> them all (ConfigCommand.scala/describeResourceConfig). `kafka-topics.sh`
> also describes all topics if you use `—describe` with no topic name.
>
> So, I have added a new `--list` option to `kafka-client-metrics.sh` in
> this KIP.
> The `--describe` option remains as before - if you supply a name, it
> describes
> just that resource, and if you do not, it describes all resources.
>
> Thanks,
> Andrew
>
> > On 17 Nov 2023, at 22:13, Jun Rao <ju...@confluent.io.INVALID> wrote:
> >
> > Hi, Andrew,
> >
> > Thanks for the reply. KIP-714 proposes to use the following command for
> > listing metric subscriptions.
> >
> > kafka-client-metrics.sh --bootstrap-server $BROKERS --describe
> > kafka-configs.sh --bootstrap-server $BROKERS --describe --entity-type
> > client-metrics
> >
> > Should we use --list instead to be more consistent with other tools like
> > (kafka-topics)?
> >
> > Thanks,
> >
> > Jun
> >
> > On Thu, Nov 16, 2023 at 10:23 AM Andrew Schofield <
> > andrew_schofield_jira@outlook.com> wrote:
> >
> >> Hi Jun,
> >> KIP-714 includes `kafka-client-metrics.sh` which provides an easier way
> to
> >> work with client metrics config
> >> than the general-purpose `kafka-configs.sh`. So, this new RPC will
> >> actually be used in the
> >> `kafka-client-metrics.sh` tool.
> >>
> >> Thanks,
> >> Andrew
> >>
> >>> On 16 Nov 2023, at 18:00, Jun Rao <ju...@confluent.io.INVALID> wrote:
> >>>
> >>> Hi, Andrew,
> >>>
> >>> Thanks for the KIP. Just one comment.
> >>>
> >>> Should we extend ConfigCommand or add a new tool to list client
> metrics?
> >>>
> >>> Thanks,
> >>>
> >>> Jun
> >>>
> >>> On Tue, Nov 7, 2023 at 9:42 AM Andrew Schofield <
> >>> andrew_schofield_jira@outlook.com> wrote:
> >>>
> >>>> Hi,
> >>>> I would like to start discussion of a small KIP which fills a gap in
> the
> >>>> administration of client metrics configuration.
> >>>>
> >>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources
> >>>>
> >>>> Thanks,
> >>>> Andrew
> >>
> >>
>
>
Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources
Posted by Andrew Schofield <an...@outlook.com>.
Hi Jun,
This is an example of inconsistency between the tools. I do agree that
it would be nice to have `--list` on the `kafka-client-metrics.sh` tool.
However, `kafka-config.sh` has the convention that `--describe` with no
entity name supplied first gets a list of all the entities, and then describes
them all (ConfigCommand.scala/describeResourceConfig). `kafka-topics.sh`
also describes all topics if you use `—describe` with no topic name.
So, I have added a new `--list` option to `kafka-client-metrics.sh` in this KIP.
The `--describe` option remains as before - if you supply a name, it describes
just that resource, and if you do not, it describes all resources.
Thanks,
Andrew
> On 17 Nov 2023, at 22:13, Jun Rao <ju...@confluent.io.INVALID> wrote:
>
> Hi, Andrew,
>
> Thanks for the reply. KIP-714 proposes to use the following command for
> listing metric subscriptions.
>
> kafka-client-metrics.sh --bootstrap-server $BROKERS --describe
> kafka-configs.sh --bootstrap-server $BROKERS --describe --entity-type
> client-metrics
>
> Should we use --list instead to be more consistent with other tools like
> (kafka-topics)?
>
> Thanks,
>
> Jun
>
> On Thu, Nov 16, 2023 at 10:23 AM Andrew Schofield <
> andrew_schofield_jira@outlook.com> wrote:
>
>> Hi Jun,
>> KIP-714 includes `kafka-client-metrics.sh` which provides an easier way to
>> work with client metrics config
>> than the general-purpose `kafka-configs.sh`. So, this new RPC will
>> actually be used in the
>> `kafka-client-metrics.sh` tool.
>>
>> Thanks,
>> Andrew
>>
>>> On 16 Nov 2023, at 18:00, Jun Rao <ju...@confluent.io.INVALID> wrote:
>>>
>>> Hi, Andrew,
>>>
>>> Thanks for the KIP. Just one comment.
>>>
>>> Should we extend ConfigCommand or add a new tool to list client metrics?
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>> On Tue, Nov 7, 2023 at 9:42 AM Andrew Schofield <
>>> andrew_schofield_jira@outlook.com> wrote:
>>>
>>>> Hi,
>>>> I would like to start discussion of a small KIP which fills a gap in the
>>>> administration of client metrics configuration.
>>>>
>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources
>>>>
>>>> Thanks,
>>>> Andrew
>>
>>
Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources
Posted by Jun Rao <ju...@confluent.io.INVALID>.
Hi, Andrew,
Thanks for the reply. KIP-714 proposes to use the following command for
listing metric subscriptions.
kafka-client-metrics.sh --bootstrap-server $BROKERS --describe
kafka-configs.sh --bootstrap-server $BROKERS --describe --entity-type
client-metrics
Should we use --list instead to be more consistent with other tools like
(kafka-topics)?
Thanks,
Jun
On Thu, Nov 16, 2023 at 10:23 AM Andrew Schofield <
andrew_schofield_jira@outlook.com> wrote:
> Hi Jun,
> KIP-714 includes `kafka-client-metrics.sh` which provides an easier way to
> work with client metrics config
> than the general-purpose `kafka-configs.sh`. So, this new RPC will
> actually be used in the
> `kafka-client-metrics.sh` tool.
>
> Thanks,
> Andrew
>
> > On 16 Nov 2023, at 18:00, Jun Rao <ju...@confluent.io.INVALID> wrote:
> >
> > Hi, Andrew,
> >
> > Thanks for the KIP. Just one comment.
> >
> > Should we extend ConfigCommand or add a new tool to list client metrics?
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Nov 7, 2023 at 9:42 AM Andrew Schofield <
> > andrew_schofield_jira@outlook.com> wrote:
> >
> >> Hi,
> >> I would like to start discussion of a small KIP which fills a gap in the
> >> administration of client metrics configuration.
> >>
> >>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources
> >>
> >> Thanks,
> >> Andrew
>
>
Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources
Posted by Andrew Schofield <an...@outlook.com>.
Hi Jun,
KIP-714 includes `kafka-client-metrics.sh` which provides an easier way to work with client metrics config
than the general-purpose `kafka-configs.sh`. So, this new RPC will actually be used in the
`kafka-client-metrics.sh` tool.
Thanks,
Andrew
> On 16 Nov 2023, at 18:00, Jun Rao <ju...@confluent.io.INVALID> wrote:
>
> Hi, Andrew,
>
> Thanks for the KIP. Just one comment.
>
> Should we extend ConfigCommand or add a new tool to list client metrics?
>
> Thanks,
>
> Jun
>
> On Tue, Nov 7, 2023 at 9:42 AM Andrew Schofield <
> andrew_schofield_jira@outlook.com> wrote:
>
>> Hi,
>> I would like to start discussion of a small KIP which fills a gap in the
>> administration of client metrics configuration.
>>
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources
>>
>> Thanks,
>> Andrew
Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources
Posted by Andrew Schofield <an...@outlook.com>.
Hi Jason,
Thanks for your comments.
1) Any broker can handle this API, so admin clients will choose a node randomly.
2) I was following the RPCs for configs which support controller and broker. However,
looking at all of the List… and Describe… RPCs, I see that the majority are broker-only.
I have change the KIP to have only “broker” in the “listeners”.
Thanks,
Andrew
> On 16 Nov 2023, at 18:16, Jason Gustafson <ja...@confluent.io.INVALID> wrote:
>
> Hey Andrew,
>
> Thanks for the KIP. Just clarifying a couple small details.
>
> 1. I assume any broker can handle this API, so admin clients will choose a
> node randomly?
> 2. Does the controller need to support this API? If not, we can drop
> "controller" from "listeners."
>
> Thanks,
> Jason
>
> On Thu, Nov 16, 2023 at 10:00 AM Jun Rao <ju...@confluent.io.invalid> wrote:
>
>> Hi, Andrew,
>>
>> Thanks for the KIP. Just one comment.
>>
>> Should we extend ConfigCommand or add a new tool to list client metrics?
>>
>> Thanks,
>>
>> Jun
>>
>> On Tue, Nov 7, 2023 at 9:42 AM Andrew Schofield <
>> andrew_schofield_jira@outlook.com> wrote:
>>
>>> Hi,
>>> I would like to start discussion of a small KIP which fills a gap in the
>>> administration of client metrics configuration.
>>>
>>>
>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources
>>>
>>> Thanks,
>>> Andrew
>>
Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources
Posted by Jason Gustafson <ja...@confluent.io.INVALID>.
Hey Andrew,
Thanks for the KIP. Just clarifying a couple small details.
1. I assume any broker can handle this API, so admin clients will choose a
node randomly?
2. Does the controller need to support this API? If not, we can drop
"controller" from "listeners."
Thanks,
Jason
On Thu, Nov 16, 2023 at 10:00 AM Jun Rao <ju...@confluent.io.invalid> wrote:
> Hi, Andrew,
>
> Thanks for the KIP. Just one comment.
>
> Should we extend ConfigCommand or add a new tool to list client metrics?
>
> Thanks,
>
> Jun
>
> On Tue, Nov 7, 2023 at 9:42 AM Andrew Schofield <
> andrew_schofield_jira@outlook.com> wrote:
>
> > Hi,
> > I would like to start discussion of a small KIP which fills a gap in the
> > administration of client metrics configuration.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources
> >
> > Thanks,
> > Andrew
>
Re: [DISCUSS] KIP-1000: List Client Metrics Configuration Resources
Posted by Jun Rao <ju...@confluent.io.INVALID>.
Hi, Andrew,
Thanks for the KIP. Just one comment.
Should we extend ConfigCommand or add a new tool to list client metrics?
Thanks,
Jun
On Tue, Nov 7, 2023 at 9:42 AM Andrew Schofield <
andrew_schofield_jira@outlook.com> wrote:
> Hi,
> I would like to start discussion of a small KIP which fills a gap in the
> administration of client metrics configuration.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1000%3A+List+Client+Metrics+Configuration+Resources
>
> Thanks,
> Andrew