You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Mario Fiore Vitale via dev <de...@kafka.apache.org> on 2026/01/21 15:32:54 UTC
[DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Hi Everyone,
I would like to start a discussion on KIP-1273: Improve Connect
configurable components discoverability [1].
Summary:
The idea is to introduce a common ConfigSpecifier interface for Kafka
Connect components that expose configuration metadata. By unifying the
existing config() method across connectors, converters, transformations,
and predicates under a single contract, it simplifies component discovery,
reduces code duplication and enables uniform tooling for configuration
introspection, validation, documentation, and UI generation. The change is
fully backward compatible and requires no modifications to existing
component implementations.
[1]
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
Regards,
Mario
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Chris Egerton <fe...@gmail.com>.
Hi Mario,
I think it'd be fine to start a vote once the changes are made. If further
discussion is needed we can always handle that as it comes.
Cheers,
Cheers
On Mon, Feb 16, 2026, 06:07 Mario Fiore Vitale via dev <de...@kafka.apache.org>
wrote:
> Hi Chris and Mickael,
>
> So, in this case, I like the second option, which is also more aligned with
> my proposal to use "ConnectPlugin" as the name.
> In the end, we will have the "ConnectPlugin" used for both connector and
> worker plugins.
>
> I'll do the changes on the KIP. Do you think we are ready for a vote then?
>
> Thanks,
> Mario.
>
> Il giorno ven 13 feb 2026 alle ore 16:18 Mickael Maison <
> mickael.maison@gmail.com> ha scritto:
>
> > Hi,
> >
> > I agree on the "pick one of these two mutually-exclusive options" point.
> > I'm fine with either option.
> >
> > Thanks,
> > Mickael
> >
> > On Thu, Feb 12, 2026 at 8:29 PM Chris Egerton <fe...@gmail.com>
> > wrote:
> > >
> > > Hi Mario,
> > >
> > > I think the idea with "ConnectorPlugin" is that it only applies to
> > plugins
> > > that are used directly by connectors, which excludes worker-level
> plugins
> > > like the connector client override policy and REST extension
> interfaces.
> > So
> > > IMO we should pick one of these two mutually-exclusive options:
> > >
> > > 1. Name the parent interface "ConnectorPlugin" and only use it for
> > > connector plugins but not worker plugins*
> > > 2. Name the parent interface something else (maybe just
> "ConnectPlugin"?)
> > > and use it for both connector and worker plugins
> > >
> > > * - Note that this doesn't preclude us from adding a config() method to
> > the
> > > worker plugin interfaces, just from using the same parent interface
> name
> > as
> > > the one for connector plugins
> > >
> > > I'd lean towards the second option but I'd accept either. I suspect
> > Mickael
> > > would prefer the first but will let him weigh in :)
> > >
> > > Cheers,
> > >
> > > Chris
> > >
> > > On Fri, Feb 6, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> > > dev@kafka.apache.org> wrote:
> > >
> > > > Hi Everyone,
> > > >
> > > > I have updated the KIP document with the ConnectorPlugin. I'll
> > appreciate
> > > > any feedback.
> > > >
> > > > Thanks,
> > > > Mario.
> > > >
> > > > Il giorno ven 6 feb 2026 alle ore 09:37 Mario Fiore Vitale <
> > > > mfvitale@commonhaus.dev> ha scritto:
> > > >
> > > > > Hi Mickael,
> > > > >
> > > > > I got it. So I'll update the KIP with the ConnectorPlugin interface
> > > > having
> > > > > version() and config() methods.
> > > > >
> > > > > Thanks,
> > > > > Mario
> > > > >
> > > > > Il giorno gio 5 feb 2026 alle ore 18:24 Mickael Maison <
> > > > > mickael.maison@gmail.com> ha scritto:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> Connectors, transformations, predicates are typically referred as
> > > > >> "connector plugins".
> > > > >> On the other hand ConnectRestExtension,
> > > > >> ConnectorClientConfigOverridePolicy and ConfigProvider are "worker
> > > > >> plugins".
> > > > >> Yes it's confusing! The terms connectors and plugins are really
> > > > overused.
> > > > >>
> > > > >> Mickael
> > > > >>
> > > > >> On Thu, Feb 5, 2026 at 5:58 PM Mario Fiore Vitale via dev
> > > > >> <de...@kafka.apache.org> wrote:
> > > > >> >
> > > > >> > > Otherwise we could have a ConnectorPlugin interface (putting
> > aside
> > > > the
> > > > >> > worker plugins for now as they don't have a config() method)
> > bringing
> > > > >> > config() and version() that all plugins interface would
> implement?
> > > > >> >
> > > > >> > So If I get it correctly, you are proposing to have a single
> > interface
> > > > >> > (ConnectorPlugin) that replaces the Versioned and the
> > > > >> > ConfigSpecifier, merging them.
> > > > >> > Sounds reasonable. In that case, I'll use a more generic name
> like
> > > > >> > *ConnectPlugin
> > > > >> > *since it will be used not only for connectors but also for
> > > > >> transformation,
> > > > >> > predicates, and converters.
> > > > >> >
> > > > >> > Mickael, Chris, WDYT?
> > > > >> >
> > > > >> > > I like the ConnectorPlugin interface idea. Mario, would that
> > suit
> > > > >> your use
> > > > >> > case?
> > > > >> >
> > > > >> > The result is the same, so it is good.
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Mario.
> > > > >> >
> > > > >> > Il giorno gio 5 feb 2026 alle ore 16:53 Chris Egerton <
> > > > >> > fearthecellos@gmail.com> ha scritto:
> > > > >> >
> > > > >> > > I agree that ConfigSpecifier is a little clunky, but IMO
> > Configured
> > > > >> is a
> > > > >> > > little too be similar to the existing Configurable interface.
> > > > >> > >
> > > > >> > > I like the ConnectorPlugin interface idea. Mario, would that
> > suit
> > > > >> your use
> > > > >> > > case?
> > > > >> > >
> > > > >> > > On Thu, Feb 5, 2026, 10:24 Mickael Maison <
> > mickael.maison@gmail.com
> > > > >
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hi Mario,
> > > > >> > > >
> > > > >> > > > Thanks for the KIP.
> > > > >> > > >
> > > > >> > > > Today the mechanism to discover plugins and their
> > configurations
> > > > is
> > > > >> > > > the REST API.
> > > > >> > > > You can retrieve the list of all installed connectors
> plugins
> > > > >> > > > (connectors, transformations, predicates) via GET
> > > > >> > > > /connector-plugins?connectorsOnly=false
> > > > >> > > > You can get their configurations via GET
> > > > >> > > /connector-plugins/{plugin}/config
> > > > >> > > >
> > > > >> > > > That said, I'm not against bringing consistency to the
> plugin
> > > > APIs,
> > > > >> > > > especially as the drawbacks are really minor.
> > > > >> > > > In terms of naming I'm not a huge fan of ConfigSpecifier. I
> > don't
> > > > >> have
> > > > >> > > > a great alternative to suggest.
> > > > >> > > > To follow the naming of Versioned, maybe we can consider
> > > > Configured?
> > > > >> > > > Otherwise we could have a ConnectorPlugin interface (putting
> > aside
> > > > >> the
> > > > >> > > > worker plugins for now as they don't have a config() method)
> > > > >> bringing
> > > > >> > > > config() and version() that all plugins interface would
> > implement?
> > > > >> > > >
> > > > >> > > > Thanks,
> > > > >> > > > Mickael
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Mon, Feb 2, 2026 at 9:48 AM Mario Fiore Vitale via dev
> > > > >> > > > <de...@kafka.apache.org> wrote:
> > > > >> > > > >
> > > > >> > > > > Hi Everyone,
> > > > >> > > > >
> > > > >> > > > > Just a ping to bring the discussion up.
> > > > >> > > > >
> > > > >> > > > > Thanks,
> > > > >> > > > > Mario.
> > > > >> > > > >
> > > > >> > > > > Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore
> Vitale
> > <
> > > > >> > > > > mfvitale@commonhaus.dev> ha scritto:
> > > > >> > > > >
> > > > >> > > > > > Hi Chris,
> > > > >> > > > > >
> > > > >> > > > > > Thanks for the feedback. I updated the wrong Javadoc.
> > > > >> > > > > >
> > > > >> > > > > > Any other comments from anybody?
> > > > >> > > > > >
> > > > >> > > > > > Thanks,
> > > > >> > > > > > Mario.
> > > > >> > > > > >
> > > > >> > > > > > Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
> > > > >> > > > > > fearthecellos@gmail.com> ha scritto:
> > > > >> > > > > >
> > > > >> > > > > >> Hi Mario,
> > > > >> > > > > >>
> > > > >> > > > > >> Thanks for the updates, especially the comprehensive
> > > > >> compatibility
> > > > >> > > > > >> section!
> > > > >> > > > > >> The KIP looks good to me, though you may want to
> > double-check
> > > > >> the
> > > > >> > > > example
> > > > >> > > > > >> Javadocs on ConnectRestExtension::config
> > > > >> > > > > >> and ConnectorClientConfigOverridePolicy::config :)
> > > > >> > > > > >>
> > > > >> > > > > >> Cheers,
> > > > >> > > > > >>
> > > > >> > > > > >> Chris
> > > > >> > > > > >>
> > > > >> > > > > >> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via
> > dev <
> > > > >> > > > > >> dev@kafka.apache.org> wrote:
> > > > >> > > > > >>
> > > > >> > > > > >> > Hi Chris, thanks for your valuable feedback!
> > > > >> > > > > >> >
> > > > >> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config
> > is a
> > > > >> little
> > > > >> > > > > >> > strange--is
> > > > >> > > > > >> > it meant to refer to "this set of header converters"
> or
> > > > >> should it
> > > > >> > > > just
> > > > >> > > > > >> be
> > > > >> > > > > >> > "this header converter"?
> > > > >> > > > > >> >
> > > > >> > > > > >> > This was just taken from the current source. But I
> > agree
> > > > >> that it
> > > > >> > > > should
> > > > >> > > > > >> be
> > > > >> > > > > >> > "this header converter".
> > > > >> > > > > >> >
> > > > >> > > > > >> > > For example, there's the obvious case
> > > > >> > > > > >> > where a plugin instantiates an object with a
> > left-hand-type
> > > > >> of
> > > > >> > > > > >> > ConfigSpecifier in its constructor. That plugin would
> > be
> > > > >> > > > incompatible
> > > > >> > > > > >> with
> > > > >> > > > > >> > older versions of Connect.
> > > > >> > > > > >> >
> > > > >> > > > > >> > Do you have an example of this? As you said, it's
> > unlikely
> > > > >> that
> > > > >> > > this
> > > > >> > > > > >> would
> > > > >> > > > > >> > occur in practice, also to me.
> > > > >> > > > > >> >
> > > > >> > > > > >> > > If a plugin explicitly
> > > > >> > > > > >> > implements ConfigSpecifier, would that break
> > compatibility?
> > > > >> > > > > >> >
> > > > >> > > > > >> > In this case, it should.
> > > > >> > > > > >> >
> > > > >> > > > > >> > > If a plugin
> > > > >> > > > > >> > doesn't explicitly implement ConfigSpecifier, would
> > that
> > > > >> break
> > > > >> > > > > >> > compatibility?
> > > > >> > > > > >> >
> > > > >> > > > > >> > This should not break it.
> > > > >> > > > > >> >
> > > > >> > > > > >> > > These are the two cases that come to mind
> > immediately but
> > > > >> if
> > > > >> > > > > >> > any others occur to you feel free to document them as
> > well.
> > > > >> > > > > >> >
> > > > >> > > > > >> > No other cases come to mind. I'll update the
> > compatibility
> > > > >> section
> > > > >> > > > with
> > > > >> > > > > >> the
> > > > >> > > > > >> > forward compatibility consideration.
> > > > >> > > > > >> >
> > > > >> > > > > >> > > There's also the pluggable ConnectRestExtension
> > > > >> > > > > >> > and ConnectorClientConfigOverridePolicy interfaces.
> > Neither
> > > > >> > > > currently
> > > > >> > > > > >> > exposes a config method but it wouldn't be out of the
> > > > >> question to
> > > > >> > > > add
> > > > >> > > > > >> one
> > > > >> > > > > >> > now to them with a default implementation that
> returns
> > an
> > > > >> empty
> > > > >> > > > > >> ConfigDef.
> > > > >> > > > > >> >
> > > > >> > > > > >> > Well, I would say that since they are
> `Configurable`, I
> > > > could
> > > > >> > > expect
> > > > >> > > > > >> that
> > > > >> > > > > >> > maybe in some future there could also be the
> > possibility to
> > > > >> > > declare
> > > > >> > > > a
> > > > >> > > > > >> > specific configuration.
> > > > >> > > > > >> > So your proposal to add it and implement it as a
> > default
> > > > >> seems
> > > > >> > > good
> > > > >> > > > to
> > > > >> > > > > >> me.
> > > > >> > > > > >> >
> > > > >> > > > > >> > Regards,
> > > > >> > > > > >> > Mario.
> > > > >> > > > > >> >
> > > > >> > > > > >> >
> > > > >> > > > > >> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris
> Egerton
> > <
> > > > >> > > > > >> > fearthecellos@gmail.com> ha scritto:
> > > > >> > > > > >> >
> > > > >> > > > > >> > > Hi Mario,
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > Thanks for the KIP! My thoughts:
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config
> > is a
> > > > >> little
> > > > >> > > > > >> > strange--is
> > > > >> > > > > >> > > it meant to refer to "this set of header
> converters"
> > or
> > > > >> should
> > > > >> > > it
> > > > >> > > > > >> just be
> > > > >> > > > > >> > > "this header converter"?
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > 2. The compatibility problem with Connect is always
> > > > >> gnarly. The
> > > > >> > > > > >> section
> > > > >> > > > > >> > on
> > > > >> > > > > >> > > backward compatibility is very well done in that it
> > > > >> demonstrates
> > > > >> > > > with
> > > > >> > > > > >> > > certainty that older plugins will be able to run
> > smoothly
> > > > >> on
> > > > >> > > newer
> > > > >> > > > > >> > versions
> > > > >> > > > > >> > > of the Connect runtime. However, there's also the
> > > > question
> > > > >> of
> > > > >> > > > whether
> > > > >> > > > > >> > newer
> > > > >> > > > > >> > > plugins (compiled against a future version of the
> > Connect
> > > > >> API
> > > > >> > > > with the
> > > > >> > > > > >> > > change proposed in this KIP) will continue to be
> > > > >> compatible with
> > > > >> > > > older
> > > > >> > > > > >> > > versions of the Connect runtime. For example,
> > there's the
> > > > >> > > obvious
> > > > >> > > > case
> > > > >> > > > > >> > > where a plugin instantiates an object with a
> > > > >> left-hand-type of
> > > > >> > > > > >> > > ConfigSpecifier in its constructor. That plugin
> > would be
> > > > >> > > > incompatible
> > > > >> > > > > >> > with
> > > > >> > > > > >> > > older versions of Connect. It's unlikely that this
> > would
> > > > >> occur
> > > > >> > > in
> > > > >> > > > > >> > practice
> > > > >> > > > > >> > > and even if it does, I think it's fine to accept
> > that as
> > > > a
> > > > >> > > > tradeoff.
> > > > >> > > > > >> > > However, it'd be nice to see the compatibility
> > section
> > > > >> explore
> > > > >> > > > exactly
> > > > >> > > > > >> > what
> > > > >> > > > > >> > > would render a plugin compiled against the newer
> > Connect
> > > > >> API
> > > > >> > > > > >> incompatible
> > > > >> > > > > >> > > with older versions of the Connect runtime. If a
> > plugin
> > > > >> > > explicitly
> > > > >> > > > > >> > > implements ConfigSpecifier, would that break
> > > > >> compatibility? If a
> > > > >> > > > > >> plugin
> > > > >> > > > > >> > > doesn't explicitly implement ConfigSpecifier, would
> > that
> > > > >> break
> > > > >> > > > > >> > > compatibility? These are the two cases that come to
> > mind
> > > > >> > > > immediately
> > > > >> > > > > >> but
> > > > >> > > > > >> > if
> > > > >> > > > > >> > > any others occur to you feel free to document them
> as
> > > > >> well. This
> > > > >> > > > > >> coverage
> > > > >> > > > > >> > > can also help guide us in how to document the
> > interface
> > > > >> if/when
> > > > >> > > we
> > > > >> > > > > >> add it
> > > > >> > > > > >> > > to make it clear to plugin developers how to avoid
> > > > >> rendering
> > > > >> > > their
> > > > >> > > > > >> work
> > > > >> > > > > >> > > accidentally incompatibile with older versions of
> the
> > > > >> Connect
> > > > >> > > > runtime.
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > 3. There's also the pluggable ConnectRestExtension
> > > > >> > > > > >> > > and ConnectorClientConfigOverridePolicy interfaces.
> > > > Neither
> > > > >> > > > currently
> > > > >> > > > > >> > > exposes a config method but it wouldn't be out of
> the
> > > > >> question
> > > > >> > > to
> > > > >> > > > add
> > > > >> > > > > >> one
> > > > >> > > > > >> > > now to them with a default implementation that
> > returns an
> > > > >> empty
> > > > >> > > > > >> > ConfigDef.
> > > > >> > > > > >> > > Thoughts?
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > Cheers,
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > Chris
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale
> > via
> > > > >> dev <
> > > > >> > > > > >> > > dev@kafka.apache.org> wrote:
> > > > >> > > > > >> > >
> > > > >> > > > > >> > > > Hi Everyone,
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > I would like to start a discussion on KIP-1273:
> > Improve
> > > > >> > > Connect
> > > > >> > > > > >> > > > configurable components discoverability [1].
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > Summary:
> > > > >> > > > > >> > > > The idea is to introduce a common ConfigSpecifier
> > > > >> interface
> > > > >> > > for
> > > > >> > > > > >> Kafka
> > > > >> > > > > >> > > > Connect components that expose configuration
> > metadata.
> > > > By
> > > > >> > > > unifying
> > > > >> > > > > >> the
> > > > >> > > > > >> > > > existing config() method across connectors,
> > converters,
> > > > >> > > > > >> > transformations,
> > > > >> > > > > >> > > > and predicates under a single contract, it
> > simplifies
> > > > >> > > component
> > > > >> > > > > >> > > discovery,
> > > > >> > > > > >> > > > reduces code duplication and enables uniform
> > tooling
> > > > for
> > > > >> > > > > >> configuration
> > > > >> > > > > >> > > > introspection, validation, documentation, and UI
> > > > >> generation.
> > > > >> > > The
> > > > >> > > > > >> change
> > > > >> > > > > >> > > is
> > > > >> > > > > >> > > > fully backward compatible and requires no
> > modifications
> > > > >> to
> > > > >> > > > existing
> > > > >> > > > > >> > > > component implementations.
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > [1]
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> >
> > > > >> > > > > >>
> > > > >> > > >
> > > > >> > >
> > > > >>
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > > > Regards,
> > > > >> > > > > >> > > > Mario
> > > > >> > > > > >> > > >
> > > > >> > > > > >> > >
> > > > >> > > > > >> >
> > > > >> > > > > >>
> > > > >> > > > > >
> > > > >> > > >
> > > > >> > >
> > > > >>
> > > > >
> > > >
> >
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Mario Fiore Vitale via dev <de...@kafka.apache.org>.
Hi Chris and Mickael,
So, in this case, I like the second option, which is also more aligned with
my proposal to use "ConnectPlugin" as the name.
In the end, we will have the "ConnectPlugin" used for both connector and
worker plugins.
I'll do the changes on the KIP. Do you think we are ready for a vote then?
Thanks,
Mario.
Il giorno ven 13 feb 2026 alle ore 16:18 Mickael Maison <
mickael.maison@gmail.com> ha scritto:
> Hi,
>
> I agree on the "pick one of these two mutually-exclusive options" point.
> I'm fine with either option.
>
> Thanks,
> Mickael
>
> On Thu, Feb 12, 2026 at 8:29 PM Chris Egerton <fe...@gmail.com>
> wrote:
> >
> > Hi Mario,
> >
> > I think the idea with "ConnectorPlugin" is that it only applies to
> plugins
> > that are used directly by connectors, which excludes worker-level plugins
> > like the connector client override policy and REST extension interfaces.
> So
> > IMO we should pick one of these two mutually-exclusive options:
> >
> > 1. Name the parent interface "ConnectorPlugin" and only use it for
> > connector plugins but not worker plugins*
> > 2. Name the parent interface something else (maybe just "ConnectPlugin"?)
> > and use it for both connector and worker plugins
> >
> > * - Note that this doesn't preclude us from adding a config() method to
> the
> > worker plugin interfaces, just from using the same parent interface name
> as
> > the one for connector plugins
> >
> > I'd lean towards the second option but I'd accept either. I suspect
> Mickael
> > would prefer the first but will let him weigh in :)
> >
> > Cheers,
> >
> > Chris
> >
> > On Fri, Feb 6, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> > dev@kafka.apache.org> wrote:
> >
> > > Hi Everyone,
> > >
> > > I have updated the KIP document with the ConnectorPlugin. I'll
> appreciate
> > > any feedback.
> > >
> > > Thanks,
> > > Mario.
> > >
> > > Il giorno ven 6 feb 2026 alle ore 09:37 Mario Fiore Vitale <
> > > mfvitale@commonhaus.dev> ha scritto:
> > >
> > > > Hi Mickael,
> > > >
> > > > I got it. So I'll update the KIP with the ConnectorPlugin interface
> > > having
> > > > version() and config() methods.
> > > >
> > > > Thanks,
> > > > Mario
> > > >
> > > > Il giorno gio 5 feb 2026 alle ore 18:24 Mickael Maison <
> > > > mickael.maison@gmail.com> ha scritto:
> > > >
> > > >> Hi,
> > > >>
> > > >> Connectors, transformations, predicates are typically referred as
> > > >> "connector plugins".
> > > >> On the other hand ConnectRestExtension,
> > > >> ConnectorClientConfigOverridePolicy and ConfigProvider are "worker
> > > >> plugins".
> > > >> Yes it's confusing! The terms connectors and plugins are really
> > > overused.
> > > >>
> > > >> Mickael
> > > >>
> > > >> On Thu, Feb 5, 2026 at 5:58 PM Mario Fiore Vitale via dev
> > > >> <de...@kafka.apache.org> wrote:
> > > >> >
> > > >> > > Otherwise we could have a ConnectorPlugin interface (putting
> aside
> > > the
> > > >> > worker plugins for now as they don't have a config() method)
> bringing
> > > >> > config() and version() that all plugins interface would implement?
> > > >> >
> > > >> > So If I get it correctly, you are proposing to have a single
> interface
> > > >> > (ConnectorPlugin) that replaces the Versioned and the
> > > >> > ConfigSpecifier, merging them.
> > > >> > Sounds reasonable. In that case, I'll use a more generic name like
> > > >> > *ConnectPlugin
> > > >> > *since it will be used not only for connectors but also for
> > > >> transformation,
> > > >> > predicates, and converters.
> > > >> >
> > > >> > Mickael, Chris, WDYT?
> > > >> >
> > > >> > > I like the ConnectorPlugin interface idea. Mario, would that
> suit
> > > >> your use
> > > >> > case?
> > > >> >
> > > >> > The result is the same, so it is good.
> > > >> >
> > > >> > Thanks,
> > > >> > Mario.
> > > >> >
> > > >> > Il giorno gio 5 feb 2026 alle ore 16:53 Chris Egerton <
> > > >> > fearthecellos@gmail.com> ha scritto:
> > > >> >
> > > >> > > I agree that ConfigSpecifier is a little clunky, but IMO
> Configured
> > > >> is a
> > > >> > > little too be similar to the existing Configurable interface.
> > > >> > >
> > > >> > > I like the ConnectorPlugin interface idea. Mario, would that
> suit
> > > >> your use
> > > >> > > case?
> > > >> > >
> > > >> > > On Thu, Feb 5, 2026, 10:24 Mickael Maison <
> mickael.maison@gmail.com
> > > >
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Mario,
> > > >> > > >
> > > >> > > > Thanks for the KIP.
> > > >> > > >
> > > >> > > > Today the mechanism to discover plugins and their
> configurations
> > > is
> > > >> > > > the REST API.
> > > >> > > > You can retrieve the list of all installed connectors plugins
> > > >> > > > (connectors, transformations, predicates) via GET
> > > >> > > > /connector-plugins?connectorsOnly=false
> > > >> > > > You can get their configurations via GET
> > > >> > > /connector-plugins/{plugin}/config
> > > >> > > >
> > > >> > > > That said, I'm not against bringing consistency to the plugin
> > > APIs,
> > > >> > > > especially as the drawbacks are really minor.
> > > >> > > > In terms of naming I'm not a huge fan of ConfigSpecifier. I
> don't
> > > >> have
> > > >> > > > a great alternative to suggest.
> > > >> > > > To follow the naming of Versioned, maybe we can consider
> > > Configured?
> > > >> > > > Otherwise we could have a ConnectorPlugin interface (putting
> aside
> > > >> the
> > > >> > > > worker plugins for now as they don't have a config() method)
> > > >> bringing
> > > >> > > > config() and version() that all plugins interface would
> implement?
> > > >> > > >
> > > >> > > > Thanks,
> > > >> > > > Mickael
> > > >> > > >
> > > >> > > >
> > > >> > > > On Mon, Feb 2, 2026 at 9:48 AM Mario Fiore Vitale via dev
> > > >> > > > <de...@kafka.apache.org> wrote:
> > > >> > > > >
> > > >> > > > > Hi Everyone,
> > > >> > > > >
> > > >> > > > > Just a ping to bring the discussion up.
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > > Mario.
> > > >> > > > >
> > > >> > > > > Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore Vitale
> <
> > > >> > > > > mfvitale@commonhaus.dev> ha scritto:
> > > >> > > > >
> > > >> > > > > > Hi Chris,
> > > >> > > > > >
> > > >> > > > > > Thanks for the feedback. I updated the wrong Javadoc.
> > > >> > > > > >
> > > >> > > > > > Any other comments from anybody?
> > > >> > > > > >
> > > >> > > > > > Thanks,
> > > >> > > > > > Mario.
> > > >> > > > > >
> > > >> > > > > > Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
> > > >> > > > > > fearthecellos@gmail.com> ha scritto:
> > > >> > > > > >
> > > >> > > > > >> Hi Mario,
> > > >> > > > > >>
> > > >> > > > > >> Thanks for the updates, especially the comprehensive
> > > >> compatibility
> > > >> > > > > >> section!
> > > >> > > > > >> The KIP looks good to me, though you may want to
> double-check
> > > >> the
> > > >> > > > example
> > > >> > > > > >> Javadocs on ConnectRestExtension::config
> > > >> > > > > >> and ConnectorClientConfigOverridePolicy::config :)
> > > >> > > > > >>
> > > >> > > > > >> Cheers,
> > > >> > > > > >>
> > > >> > > > > >> Chris
> > > >> > > > > >>
> > > >> > > > > >> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via
> dev <
> > > >> > > > > >> dev@kafka.apache.org> wrote:
> > > >> > > > > >>
> > > >> > > > > >> > Hi Chris, thanks for your valuable feedback!
> > > >> > > > > >> >
> > > >> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config
> is a
> > > >> little
> > > >> > > > > >> > strange--is
> > > >> > > > > >> > it meant to refer to "this set of header converters" or
> > > >> should it
> > > >> > > > just
> > > >> > > > > >> be
> > > >> > > > > >> > "this header converter"?
> > > >> > > > > >> >
> > > >> > > > > >> > This was just taken from the current source. But I
> agree
> > > >> that it
> > > >> > > > should
> > > >> > > > > >> be
> > > >> > > > > >> > "this header converter".
> > > >> > > > > >> >
> > > >> > > > > >> > > For example, there's the obvious case
> > > >> > > > > >> > where a plugin instantiates an object with a
> left-hand-type
> > > >> of
> > > >> > > > > >> > ConfigSpecifier in its constructor. That plugin would
> be
> > > >> > > > incompatible
> > > >> > > > > >> with
> > > >> > > > > >> > older versions of Connect.
> > > >> > > > > >> >
> > > >> > > > > >> > Do you have an example of this? As you said, it's
> unlikely
> > > >> that
> > > >> > > this
> > > >> > > > > >> would
> > > >> > > > > >> > occur in practice, also to me.
> > > >> > > > > >> >
> > > >> > > > > >> > > If a plugin explicitly
> > > >> > > > > >> > implements ConfigSpecifier, would that break
> compatibility?
> > > >> > > > > >> >
> > > >> > > > > >> > In this case, it should.
> > > >> > > > > >> >
> > > >> > > > > >> > > If a plugin
> > > >> > > > > >> > doesn't explicitly implement ConfigSpecifier, would
> that
> > > >> break
> > > >> > > > > >> > compatibility?
> > > >> > > > > >> >
> > > >> > > > > >> > This should not break it.
> > > >> > > > > >> >
> > > >> > > > > >> > > These are the two cases that come to mind
> immediately but
> > > >> if
> > > >> > > > > >> > any others occur to you feel free to document them as
> well.
> > > >> > > > > >> >
> > > >> > > > > >> > No other cases come to mind. I'll update the
> compatibility
> > > >> section
> > > >> > > > with
> > > >> > > > > >> the
> > > >> > > > > >> > forward compatibility consideration.
> > > >> > > > > >> >
> > > >> > > > > >> > > There's also the pluggable ConnectRestExtension
> > > >> > > > > >> > and ConnectorClientConfigOverridePolicy interfaces.
> Neither
> > > >> > > > currently
> > > >> > > > > >> > exposes a config method but it wouldn't be out of the
> > > >> question to
> > > >> > > > add
> > > >> > > > > >> one
> > > >> > > > > >> > now to them with a default implementation that returns
> an
> > > >> empty
> > > >> > > > > >> ConfigDef.
> > > >> > > > > >> >
> > > >> > > > > >> > Well, I would say that since they are `Configurable`, I
> > > could
> > > >> > > expect
> > > >> > > > > >> that
> > > >> > > > > >> > maybe in some future there could also be the
> possibility to
> > > >> > > declare
> > > >> > > > a
> > > >> > > > > >> > specific configuration.
> > > >> > > > > >> > So your proposal to add it and implement it as a
> default
> > > >> seems
> > > >> > > good
> > > >> > > > to
> > > >> > > > > >> me.
> > > >> > > > > >> >
> > > >> > > > > >> > Regards,
> > > >> > > > > >> > Mario.
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton
> <
> > > >> > > > > >> > fearthecellos@gmail.com> ha scritto:
> > > >> > > > > >> >
> > > >> > > > > >> > > Hi Mario,
> > > >> > > > > >> > >
> > > >> > > > > >> > > Thanks for the KIP! My thoughts:
> > > >> > > > > >> > >
> > > >> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config
> is a
> > > >> little
> > > >> > > > > >> > strange--is
> > > >> > > > > >> > > it meant to refer to "this set of header converters"
> or
> > > >> should
> > > >> > > it
> > > >> > > > > >> just be
> > > >> > > > > >> > > "this header converter"?
> > > >> > > > > >> > >
> > > >> > > > > >> > > 2. The compatibility problem with Connect is always
> > > >> gnarly. The
> > > >> > > > > >> section
> > > >> > > > > >> > on
> > > >> > > > > >> > > backward compatibility is very well done in that it
> > > >> demonstrates
> > > >> > > > with
> > > >> > > > > >> > > certainty that older plugins will be able to run
> smoothly
> > > >> on
> > > >> > > newer
> > > >> > > > > >> > versions
> > > >> > > > > >> > > of the Connect runtime. However, there's also the
> > > question
> > > >> of
> > > >> > > > whether
> > > >> > > > > >> > newer
> > > >> > > > > >> > > plugins (compiled against a future version of the
> Connect
> > > >> API
> > > >> > > > with the
> > > >> > > > > >> > > change proposed in this KIP) will continue to be
> > > >> compatible with
> > > >> > > > older
> > > >> > > > > >> > > versions of the Connect runtime. For example,
> there's the
> > > >> > > obvious
> > > >> > > > case
> > > >> > > > > >> > > where a plugin instantiates an object with a
> > > >> left-hand-type of
> > > >> > > > > >> > > ConfigSpecifier in its constructor. That plugin
> would be
> > > >> > > > incompatible
> > > >> > > > > >> > with
> > > >> > > > > >> > > older versions of Connect. It's unlikely that this
> would
> > > >> occur
> > > >> > > in
> > > >> > > > > >> > practice
> > > >> > > > > >> > > and even if it does, I think it's fine to accept
> that as
> > > a
> > > >> > > > tradeoff.
> > > >> > > > > >> > > However, it'd be nice to see the compatibility
> section
> > > >> explore
> > > >> > > > exactly
> > > >> > > > > >> > what
> > > >> > > > > >> > > would render a plugin compiled against the newer
> Connect
> > > >> API
> > > >> > > > > >> incompatible
> > > >> > > > > >> > > with older versions of the Connect runtime. If a
> plugin
> > > >> > > explicitly
> > > >> > > > > >> > > implements ConfigSpecifier, would that break
> > > >> compatibility? If a
> > > >> > > > > >> plugin
> > > >> > > > > >> > > doesn't explicitly implement ConfigSpecifier, would
> that
> > > >> break
> > > >> > > > > >> > > compatibility? These are the two cases that come to
> mind
> > > >> > > > immediately
> > > >> > > > > >> but
> > > >> > > > > >> > if
> > > >> > > > > >> > > any others occur to you feel free to document them as
> > > >> well. This
> > > >> > > > > >> coverage
> > > >> > > > > >> > > can also help guide us in how to document the
> interface
> > > >> if/when
> > > >> > > we
> > > >> > > > > >> add it
> > > >> > > > > >> > > to make it clear to plugin developers how to avoid
> > > >> rendering
> > > >> > > their
> > > >> > > > > >> work
> > > >> > > > > >> > > accidentally incompatibile with older versions of the
> > > >> Connect
> > > >> > > > runtime.
> > > >> > > > > >> > >
> > > >> > > > > >> > > 3. There's also the pluggable ConnectRestExtension
> > > >> > > > > >> > > and ConnectorClientConfigOverridePolicy interfaces.
> > > Neither
> > > >> > > > currently
> > > >> > > > > >> > > exposes a config method but it wouldn't be out of the
> > > >> question
> > > >> > > to
> > > >> > > > add
> > > >> > > > > >> one
> > > >> > > > > >> > > now to them with a default implementation that
> returns an
> > > >> empty
> > > >> > > > > >> > ConfigDef.
> > > >> > > > > >> > > Thoughts?
> > > >> > > > > >> > >
> > > >> > > > > >> > > Cheers,
> > > >> > > > > >> > >
> > > >> > > > > >> > > Chris
> > > >> > > > > >> > >
> > > >> > > > > >> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale
> via
> > > >> dev <
> > > >> > > > > >> > > dev@kafka.apache.org> wrote:
> > > >> > > > > >> > >
> > > >> > > > > >> > > > Hi Everyone,
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > I would like to start a discussion on KIP-1273:
> Improve
> > > >> > > Connect
> > > >> > > > > >> > > > configurable components discoverability [1].
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > Summary:
> > > >> > > > > >> > > > The idea is to introduce a common ConfigSpecifier
> > > >> interface
> > > >> > > for
> > > >> > > > > >> Kafka
> > > >> > > > > >> > > > Connect components that expose configuration
> metadata.
> > > By
> > > >> > > > unifying
> > > >> > > > > >> the
> > > >> > > > > >> > > > existing config() method across connectors,
> converters,
> > > >> > > > > >> > transformations,
> > > >> > > > > >> > > > and predicates under a single contract, it
> simplifies
> > > >> > > component
> > > >> > > > > >> > > discovery,
> > > >> > > > > >> > > > reduces code duplication and enables uniform
> tooling
> > > for
> > > >> > > > > >> configuration
> > > >> > > > > >> > > > introspection, validation, documentation, and UI
> > > >> generation.
> > > >> > > The
> > > >> > > > > >> change
> > > >> > > > > >> > > is
> > > >> > > > > >> > > > fully backward compatible and requires no
> modifications
> > > >> to
> > > >> > > > existing
> > > >> > > > > >> > > > component implementations.
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > [1]
> > > >> > > > > >> > > >
> > > >> > > > > >> > > >
> > > >> > > > > >> > >
> > > >> > > > > >> >
> > > >> > > > > >>
> > > >> > > >
> > > >> > >
> > > >>
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> > > >> > > > > >> > > >
> > > >> > > > > >> > > > Regards,
> > > >> > > > > >> > > > Mario
> > > >> > > > > >> > > >
> > > >> > > > > >> > >
> > > >> > > > > >> >
> > > >> > > > > >>
> > > >> > > > > >
> > > >> > > >
> > > >> > >
> > > >>
> > > >
> > >
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Mickael Maison <mi...@gmail.com>.
Hi,
I agree on the "pick one of these two mutually-exclusive options" point.
I'm fine with either option.
Thanks,
Mickael
On Thu, Feb 12, 2026 at 8:29 PM Chris Egerton <fe...@gmail.com> wrote:
>
> Hi Mario,
>
> I think the idea with "ConnectorPlugin" is that it only applies to plugins
> that are used directly by connectors, which excludes worker-level plugins
> like the connector client override policy and REST extension interfaces. So
> IMO we should pick one of these two mutually-exclusive options:
>
> 1. Name the parent interface "ConnectorPlugin" and only use it for
> connector plugins but not worker plugins*
> 2. Name the parent interface something else (maybe just "ConnectPlugin"?)
> and use it for both connector and worker plugins
>
> * - Note that this doesn't preclude us from adding a config() method to the
> worker plugin interfaces, just from using the same parent interface name as
> the one for connector plugins
>
> I'd lean towards the second option but I'd accept either. I suspect Mickael
> would prefer the first but will let him weigh in :)
>
> Cheers,
>
> Chris
>
> On Fri, Feb 6, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> dev@kafka.apache.org> wrote:
>
> > Hi Everyone,
> >
> > I have updated the KIP document with the ConnectorPlugin. I'll appreciate
> > any feedback.
> >
> > Thanks,
> > Mario.
> >
> > Il giorno ven 6 feb 2026 alle ore 09:37 Mario Fiore Vitale <
> > mfvitale@commonhaus.dev> ha scritto:
> >
> > > Hi Mickael,
> > >
> > > I got it. So I'll update the KIP with the ConnectorPlugin interface
> > having
> > > version() and config() methods.
> > >
> > > Thanks,
> > > Mario
> > >
> > > Il giorno gio 5 feb 2026 alle ore 18:24 Mickael Maison <
> > > mickael.maison@gmail.com> ha scritto:
> > >
> > >> Hi,
> > >>
> > >> Connectors, transformations, predicates are typically referred as
> > >> "connector plugins".
> > >> On the other hand ConnectRestExtension,
> > >> ConnectorClientConfigOverridePolicy and ConfigProvider are "worker
> > >> plugins".
> > >> Yes it's confusing! The terms connectors and plugins are really
> > overused.
> > >>
> > >> Mickael
> > >>
> > >> On Thu, Feb 5, 2026 at 5:58 PM Mario Fiore Vitale via dev
> > >> <de...@kafka.apache.org> wrote:
> > >> >
> > >> > > Otherwise we could have a ConnectorPlugin interface (putting aside
> > the
> > >> > worker plugins for now as they don't have a config() method) bringing
> > >> > config() and version() that all plugins interface would implement?
> > >> >
> > >> > So If I get it correctly, you are proposing to have a single interface
> > >> > (ConnectorPlugin) that replaces the Versioned and the
> > >> > ConfigSpecifier, merging them.
> > >> > Sounds reasonable. In that case, I'll use a more generic name like
> > >> > *ConnectPlugin
> > >> > *since it will be used not only for connectors but also for
> > >> transformation,
> > >> > predicates, and converters.
> > >> >
> > >> > Mickael, Chris, WDYT?
> > >> >
> > >> > > I like the ConnectorPlugin interface idea. Mario, would that suit
> > >> your use
> > >> > case?
> > >> >
> > >> > The result is the same, so it is good.
> > >> >
> > >> > Thanks,
> > >> > Mario.
> > >> >
> > >> > Il giorno gio 5 feb 2026 alle ore 16:53 Chris Egerton <
> > >> > fearthecellos@gmail.com> ha scritto:
> > >> >
> > >> > > I agree that ConfigSpecifier is a little clunky, but IMO Configured
> > >> is a
> > >> > > little too be similar to the existing Configurable interface.
> > >> > >
> > >> > > I like the ConnectorPlugin interface idea. Mario, would that suit
> > >> your use
> > >> > > case?
> > >> > >
> > >> > > On Thu, Feb 5, 2026, 10:24 Mickael Maison <mickael.maison@gmail.com
> > >
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Mario,
> > >> > > >
> > >> > > > Thanks for the KIP.
> > >> > > >
> > >> > > > Today the mechanism to discover plugins and their configurations
> > is
> > >> > > > the REST API.
> > >> > > > You can retrieve the list of all installed connectors plugins
> > >> > > > (connectors, transformations, predicates) via GET
> > >> > > > /connector-plugins?connectorsOnly=false
> > >> > > > You can get their configurations via GET
> > >> > > /connector-plugins/{plugin}/config
> > >> > > >
> > >> > > > That said, I'm not against bringing consistency to the plugin
> > APIs,
> > >> > > > especially as the drawbacks are really minor.
> > >> > > > In terms of naming I'm not a huge fan of ConfigSpecifier. I don't
> > >> have
> > >> > > > a great alternative to suggest.
> > >> > > > To follow the naming of Versioned, maybe we can consider
> > Configured?
> > >> > > > Otherwise we could have a ConnectorPlugin interface (putting aside
> > >> the
> > >> > > > worker plugins for now as they don't have a config() method)
> > >> bringing
> > >> > > > config() and version() that all plugins interface would implement?
> > >> > > >
> > >> > > > Thanks,
> > >> > > > Mickael
> > >> > > >
> > >> > > >
> > >> > > > On Mon, Feb 2, 2026 at 9:48 AM Mario Fiore Vitale via dev
> > >> > > > <de...@kafka.apache.org> wrote:
> > >> > > > >
> > >> > > > > Hi Everyone,
> > >> > > > >
> > >> > > > > Just a ping to bring the discussion up.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > Mario.
> > >> > > > >
> > >> > > > > Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore Vitale <
> > >> > > > > mfvitale@commonhaus.dev> ha scritto:
> > >> > > > >
> > >> > > > > > Hi Chris,
> > >> > > > > >
> > >> > > > > > Thanks for the feedback. I updated the wrong Javadoc.
> > >> > > > > >
> > >> > > > > > Any other comments from anybody?
> > >> > > > > >
> > >> > > > > > Thanks,
> > >> > > > > > Mario.
> > >> > > > > >
> > >> > > > > > Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
> > >> > > > > > fearthecellos@gmail.com> ha scritto:
> > >> > > > > >
> > >> > > > > >> Hi Mario,
> > >> > > > > >>
> > >> > > > > >> Thanks for the updates, especially the comprehensive
> > >> compatibility
> > >> > > > > >> section!
> > >> > > > > >> The KIP looks good to me, though you may want to double-check
> > >> the
> > >> > > > example
> > >> > > > > >> Javadocs on ConnectRestExtension::config
> > >> > > > > >> and ConnectorClientConfigOverridePolicy::config :)
> > >> > > > > >>
> > >> > > > > >> Cheers,
> > >> > > > > >>
> > >> > > > > >> Chris
> > >> > > > > >>
> > >> > > > > >> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> > >> > > > > >> dev@kafka.apache.org> wrote:
> > >> > > > > >>
> > >> > > > > >> > Hi Chris, thanks for your valuable feedback!
> > >> > > > > >> >
> > >> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a
> > >> little
> > >> > > > > >> > strange--is
> > >> > > > > >> > it meant to refer to "this set of header converters" or
> > >> should it
> > >> > > > just
> > >> > > > > >> be
> > >> > > > > >> > "this header converter"?
> > >> > > > > >> >
> > >> > > > > >> > This was just taken from the current source. But I agree
> > >> that it
> > >> > > > should
> > >> > > > > >> be
> > >> > > > > >> > "this header converter".
> > >> > > > > >> >
> > >> > > > > >> > > For example, there's the obvious case
> > >> > > > > >> > where a plugin instantiates an object with a left-hand-type
> > >> of
> > >> > > > > >> > ConfigSpecifier in its constructor. That plugin would be
> > >> > > > incompatible
> > >> > > > > >> with
> > >> > > > > >> > older versions of Connect.
> > >> > > > > >> >
> > >> > > > > >> > Do you have an example of this? As you said, it's unlikely
> > >> that
> > >> > > this
> > >> > > > > >> would
> > >> > > > > >> > occur in practice, also to me.
> > >> > > > > >> >
> > >> > > > > >> > > If a plugin explicitly
> > >> > > > > >> > implements ConfigSpecifier, would that break compatibility?
> > >> > > > > >> >
> > >> > > > > >> > In this case, it should.
> > >> > > > > >> >
> > >> > > > > >> > > If a plugin
> > >> > > > > >> > doesn't explicitly implement ConfigSpecifier, would that
> > >> break
> > >> > > > > >> > compatibility?
> > >> > > > > >> >
> > >> > > > > >> > This should not break it.
> > >> > > > > >> >
> > >> > > > > >> > > These are the two cases that come to mind immediately but
> > >> if
> > >> > > > > >> > any others occur to you feel free to document them as well.
> > >> > > > > >> >
> > >> > > > > >> > No other cases come to mind. I'll update the compatibility
> > >> section
> > >> > > > with
> > >> > > > > >> the
> > >> > > > > >> > forward compatibility consideration.
> > >> > > > > >> >
> > >> > > > > >> > > There's also the pluggable ConnectRestExtension
> > >> > > > > >> > and ConnectorClientConfigOverridePolicy interfaces. Neither
> > >> > > > currently
> > >> > > > > >> > exposes a config method but it wouldn't be out of the
> > >> question to
> > >> > > > add
> > >> > > > > >> one
> > >> > > > > >> > now to them with a default implementation that returns an
> > >> empty
> > >> > > > > >> ConfigDef.
> > >> > > > > >> >
> > >> > > > > >> > Well, I would say that since they are `Configurable`, I
> > could
> > >> > > expect
> > >> > > > > >> that
> > >> > > > > >> > maybe in some future there could also be the possibility to
> > >> > > declare
> > >> > > > a
> > >> > > > > >> > specific configuration.
> > >> > > > > >> > So your proposal to add it and implement it as a default
> > >> seems
> > >> > > good
> > >> > > > to
> > >> > > > > >> me.
> > >> > > > > >> >
> > >> > > > > >> > Regards,
> > >> > > > > >> > Mario.
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
> > >> > > > > >> > fearthecellos@gmail.com> ha scritto:
> > >> > > > > >> >
> > >> > > > > >> > > Hi Mario,
> > >> > > > > >> > >
> > >> > > > > >> > > Thanks for the KIP! My thoughts:
> > >> > > > > >> > >
> > >> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a
> > >> little
> > >> > > > > >> > strange--is
> > >> > > > > >> > > it meant to refer to "this set of header converters" or
> > >> should
> > >> > > it
> > >> > > > > >> just be
> > >> > > > > >> > > "this header converter"?
> > >> > > > > >> > >
> > >> > > > > >> > > 2. The compatibility problem with Connect is always
> > >> gnarly. The
> > >> > > > > >> section
> > >> > > > > >> > on
> > >> > > > > >> > > backward compatibility is very well done in that it
> > >> demonstrates
> > >> > > > with
> > >> > > > > >> > > certainty that older plugins will be able to run smoothly
> > >> on
> > >> > > newer
> > >> > > > > >> > versions
> > >> > > > > >> > > of the Connect runtime. However, there's also the
> > question
> > >> of
> > >> > > > whether
> > >> > > > > >> > newer
> > >> > > > > >> > > plugins (compiled against a future version of the Connect
> > >> API
> > >> > > > with the
> > >> > > > > >> > > change proposed in this KIP) will continue to be
> > >> compatible with
> > >> > > > older
> > >> > > > > >> > > versions of the Connect runtime. For example, there's the
> > >> > > obvious
> > >> > > > case
> > >> > > > > >> > > where a plugin instantiates an object with a
> > >> left-hand-type of
> > >> > > > > >> > > ConfigSpecifier in its constructor. That plugin would be
> > >> > > > incompatible
> > >> > > > > >> > with
> > >> > > > > >> > > older versions of Connect. It's unlikely that this would
> > >> occur
> > >> > > in
> > >> > > > > >> > practice
> > >> > > > > >> > > and even if it does, I think it's fine to accept that as
> > a
> > >> > > > tradeoff.
> > >> > > > > >> > > However, it'd be nice to see the compatibility section
> > >> explore
> > >> > > > exactly
> > >> > > > > >> > what
> > >> > > > > >> > > would render a plugin compiled against the newer Connect
> > >> API
> > >> > > > > >> incompatible
> > >> > > > > >> > > with older versions of the Connect runtime. If a plugin
> > >> > > explicitly
> > >> > > > > >> > > implements ConfigSpecifier, would that break
> > >> compatibility? If a
> > >> > > > > >> plugin
> > >> > > > > >> > > doesn't explicitly implement ConfigSpecifier, would that
> > >> break
> > >> > > > > >> > > compatibility? These are the two cases that come to mind
> > >> > > > immediately
> > >> > > > > >> but
> > >> > > > > >> > if
> > >> > > > > >> > > any others occur to you feel free to document them as
> > >> well. This
> > >> > > > > >> coverage
> > >> > > > > >> > > can also help guide us in how to document the interface
> > >> if/when
> > >> > > we
> > >> > > > > >> add it
> > >> > > > > >> > > to make it clear to plugin developers how to avoid
> > >> rendering
> > >> > > their
> > >> > > > > >> work
> > >> > > > > >> > > accidentally incompatibile with older versions of the
> > >> Connect
> > >> > > > runtime.
> > >> > > > > >> > >
> > >> > > > > >> > > 3. There's also the pluggable ConnectRestExtension
> > >> > > > > >> > > and ConnectorClientConfigOverridePolicy interfaces.
> > Neither
> > >> > > > currently
> > >> > > > > >> > > exposes a config method but it wouldn't be out of the
> > >> question
> > >> > > to
> > >> > > > add
> > >> > > > > >> one
> > >> > > > > >> > > now to them with a default implementation that returns an
> > >> empty
> > >> > > > > >> > ConfigDef.
> > >> > > > > >> > > Thoughts?
> > >> > > > > >> > >
> > >> > > > > >> > > Cheers,
> > >> > > > > >> > >
> > >> > > > > >> > > Chris
> > >> > > > > >> > >
> > >> > > > > >> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via
> > >> dev <
> > >> > > > > >> > > dev@kafka.apache.org> wrote:
> > >> > > > > >> > >
> > >> > > > > >> > > > Hi Everyone,
> > >> > > > > >> > > >
> > >> > > > > >> > > > I would like to start a discussion on KIP-1273: Improve
> > >> > > Connect
> > >> > > > > >> > > > configurable components discoverability [1].
> > >> > > > > >> > > >
> > >> > > > > >> > > > Summary:
> > >> > > > > >> > > > The idea is to introduce a common ConfigSpecifier
> > >> interface
> > >> > > for
> > >> > > > > >> Kafka
> > >> > > > > >> > > > Connect components that expose configuration metadata.
> > By
> > >> > > > unifying
> > >> > > > > >> the
> > >> > > > > >> > > > existing config() method across connectors, converters,
> > >> > > > > >> > transformations,
> > >> > > > > >> > > > and predicates under a single contract, it simplifies
> > >> > > component
> > >> > > > > >> > > discovery,
> > >> > > > > >> > > > reduces code duplication and enables uniform tooling
> > for
> > >> > > > > >> configuration
> > >> > > > > >> > > > introspection, validation, documentation, and UI
> > >> generation.
> > >> > > The
> > >> > > > > >> change
> > >> > > > > >> > > is
> > >> > > > > >> > > > fully backward compatible and requires no modifications
> > >> to
> > >> > > > existing
> > >> > > > > >> > > > component implementations.
> > >> > > > > >> > > >
> > >> > > > > >> > > > [1]
> > >> > > > > >> > > >
> > >> > > > > >> > > >
> > >> > > > > >> > >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > >
> > >> > >
> > >>
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> > >> > > > > >> > > >
> > >> > > > > >> > > > Regards,
> > >> > > > > >> > > > Mario
> > >> > > > > >> > > >
> > >> > > > > >> > >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > > >
> > >> > > >
> > >> > >
> > >>
> > >
> >
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Chris Egerton <fe...@gmail.com>.
Hi Mario,
I think the idea with "ConnectorPlugin" is that it only applies to plugins
that are used directly by connectors, which excludes worker-level plugins
like the connector client override policy and REST extension interfaces. So
IMO we should pick one of these two mutually-exclusive options:
1. Name the parent interface "ConnectorPlugin" and only use it for
connector plugins but not worker plugins*
2. Name the parent interface something else (maybe just "ConnectPlugin"?)
and use it for both connector and worker plugins
* - Note that this doesn't preclude us from adding a config() method to the
worker plugin interfaces, just from using the same parent interface name as
the one for connector plugins
I'd lean towards the second option but I'd accept either. I suspect Mickael
would prefer the first but will let him weigh in :)
Cheers,
Chris
On Fri, Feb 6, 2026 at 5:54 AM Mario Fiore Vitale via dev <
dev@kafka.apache.org> wrote:
> Hi Everyone,
>
> I have updated the KIP document with the ConnectorPlugin. I'll appreciate
> any feedback.
>
> Thanks,
> Mario.
>
> Il giorno ven 6 feb 2026 alle ore 09:37 Mario Fiore Vitale <
> mfvitale@commonhaus.dev> ha scritto:
>
> > Hi Mickael,
> >
> > I got it. So I'll update the KIP with the ConnectorPlugin interface
> having
> > version() and config() methods.
> >
> > Thanks,
> > Mario
> >
> > Il giorno gio 5 feb 2026 alle ore 18:24 Mickael Maison <
> > mickael.maison@gmail.com> ha scritto:
> >
> >> Hi,
> >>
> >> Connectors, transformations, predicates are typically referred as
> >> "connector plugins".
> >> On the other hand ConnectRestExtension,
> >> ConnectorClientConfigOverridePolicy and ConfigProvider are "worker
> >> plugins".
> >> Yes it's confusing! The terms connectors and plugins are really
> overused.
> >>
> >> Mickael
> >>
> >> On Thu, Feb 5, 2026 at 5:58 PM Mario Fiore Vitale via dev
> >> <de...@kafka.apache.org> wrote:
> >> >
> >> > > Otherwise we could have a ConnectorPlugin interface (putting aside
> the
> >> > worker plugins for now as they don't have a config() method) bringing
> >> > config() and version() that all plugins interface would implement?
> >> >
> >> > So If I get it correctly, you are proposing to have a single interface
> >> > (ConnectorPlugin) that replaces the Versioned and the
> >> > ConfigSpecifier, merging them.
> >> > Sounds reasonable. In that case, I'll use a more generic name like
> >> > *ConnectPlugin
> >> > *since it will be used not only for connectors but also for
> >> transformation,
> >> > predicates, and converters.
> >> >
> >> > Mickael, Chris, WDYT?
> >> >
> >> > > I like the ConnectorPlugin interface idea. Mario, would that suit
> >> your use
> >> > case?
> >> >
> >> > The result is the same, so it is good.
> >> >
> >> > Thanks,
> >> > Mario.
> >> >
> >> > Il giorno gio 5 feb 2026 alle ore 16:53 Chris Egerton <
> >> > fearthecellos@gmail.com> ha scritto:
> >> >
> >> > > I agree that ConfigSpecifier is a little clunky, but IMO Configured
> >> is a
> >> > > little too be similar to the existing Configurable interface.
> >> > >
> >> > > I like the ConnectorPlugin interface idea. Mario, would that suit
> >> your use
> >> > > case?
> >> > >
> >> > > On Thu, Feb 5, 2026, 10:24 Mickael Maison <mickael.maison@gmail.com
> >
> >> > > wrote:
> >> > >
> >> > > > Hi Mario,
> >> > > >
> >> > > > Thanks for the KIP.
> >> > > >
> >> > > > Today the mechanism to discover plugins and their configurations
> is
> >> > > > the REST API.
> >> > > > You can retrieve the list of all installed connectors plugins
> >> > > > (connectors, transformations, predicates) via GET
> >> > > > /connector-plugins?connectorsOnly=false
> >> > > > You can get their configurations via GET
> >> > > /connector-plugins/{plugin}/config
> >> > > >
> >> > > > That said, I'm not against bringing consistency to the plugin
> APIs,
> >> > > > especially as the drawbacks are really minor.
> >> > > > In terms of naming I'm not a huge fan of ConfigSpecifier. I don't
> >> have
> >> > > > a great alternative to suggest.
> >> > > > To follow the naming of Versioned, maybe we can consider
> Configured?
> >> > > > Otherwise we could have a ConnectorPlugin interface (putting aside
> >> the
> >> > > > worker plugins for now as they don't have a config() method)
> >> bringing
> >> > > > config() and version() that all plugins interface would implement?
> >> > > >
> >> > > > Thanks,
> >> > > > Mickael
> >> > > >
> >> > > >
> >> > > > On Mon, Feb 2, 2026 at 9:48 AM Mario Fiore Vitale via dev
> >> > > > <de...@kafka.apache.org> wrote:
> >> > > > >
> >> > > > > Hi Everyone,
> >> > > > >
> >> > > > > Just a ping to bring the discussion up.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Mario.
> >> > > > >
> >> > > > > Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore Vitale <
> >> > > > > mfvitale@commonhaus.dev> ha scritto:
> >> > > > >
> >> > > > > > Hi Chris,
> >> > > > > >
> >> > > > > > Thanks for the feedback. I updated the wrong Javadoc.
> >> > > > > >
> >> > > > > > Any other comments from anybody?
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > > Mario.
> >> > > > > >
> >> > > > > > Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
> >> > > > > > fearthecellos@gmail.com> ha scritto:
> >> > > > > >
> >> > > > > >> Hi Mario,
> >> > > > > >>
> >> > > > > >> Thanks for the updates, especially the comprehensive
> >> compatibility
> >> > > > > >> section!
> >> > > > > >> The KIP looks good to me, though you may want to double-check
> >> the
> >> > > > example
> >> > > > > >> Javadocs on ConnectRestExtension::config
> >> > > > > >> and ConnectorClientConfigOverridePolicy::config :)
> >> > > > > >>
> >> > > > > >> Cheers,
> >> > > > > >>
> >> > > > > >> Chris
> >> > > > > >>
> >> > > > > >> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> >> > > > > >> dev@kafka.apache.org> wrote:
> >> > > > > >>
> >> > > > > >> > Hi Chris, thanks for your valuable feedback!
> >> > > > > >> >
> >> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a
> >> little
> >> > > > > >> > strange--is
> >> > > > > >> > it meant to refer to "this set of header converters" or
> >> should it
> >> > > > just
> >> > > > > >> be
> >> > > > > >> > "this header converter"?
> >> > > > > >> >
> >> > > > > >> > This was just taken from the current source. But I agree
> >> that it
> >> > > > should
> >> > > > > >> be
> >> > > > > >> > "this header converter".
> >> > > > > >> >
> >> > > > > >> > > For example, there's the obvious case
> >> > > > > >> > where a plugin instantiates an object with a left-hand-type
> >> of
> >> > > > > >> > ConfigSpecifier in its constructor. That plugin would be
> >> > > > incompatible
> >> > > > > >> with
> >> > > > > >> > older versions of Connect.
> >> > > > > >> >
> >> > > > > >> > Do you have an example of this? As you said, it's unlikely
> >> that
> >> > > this
> >> > > > > >> would
> >> > > > > >> > occur in practice, also to me.
> >> > > > > >> >
> >> > > > > >> > > If a plugin explicitly
> >> > > > > >> > implements ConfigSpecifier, would that break compatibility?
> >> > > > > >> >
> >> > > > > >> > In this case, it should.
> >> > > > > >> >
> >> > > > > >> > > If a plugin
> >> > > > > >> > doesn't explicitly implement ConfigSpecifier, would that
> >> break
> >> > > > > >> > compatibility?
> >> > > > > >> >
> >> > > > > >> > This should not break it.
> >> > > > > >> >
> >> > > > > >> > > These are the two cases that come to mind immediately but
> >> if
> >> > > > > >> > any others occur to you feel free to document them as well.
> >> > > > > >> >
> >> > > > > >> > No other cases come to mind. I'll update the compatibility
> >> section
> >> > > > with
> >> > > > > >> the
> >> > > > > >> > forward compatibility consideration.
> >> > > > > >> >
> >> > > > > >> > > There's also the pluggable ConnectRestExtension
> >> > > > > >> > and ConnectorClientConfigOverridePolicy interfaces. Neither
> >> > > > currently
> >> > > > > >> > exposes a config method but it wouldn't be out of the
> >> question to
> >> > > > add
> >> > > > > >> one
> >> > > > > >> > now to them with a default implementation that returns an
> >> empty
> >> > > > > >> ConfigDef.
> >> > > > > >> >
> >> > > > > >> > Well, I would say that since they are `Configurable`, I
> could
> >> > > expect
> >> > > > > >> that
> >> > > > > >> > maybe in some future there could also be the possibility to
> >> > > declare
> >> > > > a
> >> > > > > >> > specific configuration.
> >> > > > > >> > So your proposal to add it and implement it as a default
> >> seems
> >> > > good
> >> > > > to
> >> > > > > >> me.
> >> > > > > >> >
> >> > > > > >> > Regards,
> >> > > > > >> > Mario.
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
> >> > > > > >> > fearthecellos@gmail.com> ha scritto:
> >> > > > > >> >
> >> > > > > >> > > Hi Mario,
> >> > > > > >> > >
> >> > > > > >> > > Thanks for the KIP! My thoughts:
> >> > > > > >> > >
> >> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a
> >> little
> >> > > > > >> > strange--is
> >> > > > > >> > > it meant to refer to "this set of header converters" or
> >> should
> >> > > it
> >> > > > > >> just be
> >> > > > > >> > > "this header converter"?
> >> > > > > >> > >
> >> > > > > >> > > 2. The compatibility problem with Connect is always
> >> gnarly. The
> >> > > > > >> section
> >> > > > > >> > on
> >> > > > > >> > > backward compatibility is very well done in that it
> >> demonstrates
> >> > > > with
> >> > > > > >> > > certainty that older plugins will be able to run smoothly
> >> on
> >> > > newer
> >> > > > > >> > versions
> >> > > > > >> > > of the Connect runtime. However, there's also the
> question
> >> of
> >> > > > whether
> >> > > > > >> > newer
> >> > > > > >> > > plugins (compiled against a future version of the Connect
> >> API
> >> > > > with the
> >> > > > > >> > > change proposed in this KIP) will continue to be
> >> compatible with
> >> > > > older
> >> > > > > >> > > versions of the Connect runtime. For example, there's the
> >> > > obvious
> >> > > > case
> >> > > > > >> > > where a plugin instantiates an object with a
> >> left-hand-type of
> >> > > > > >> > > ConfigSpecifier in its constructor. That plugin would be
> >> > > > incompatible
> >> > > > > >> > with
> >> > > > > >> > > older versions of Connect. It's unlikely that this would
> >> occur
> >> > > in
> >> > > > > >> > practice
> >> > > > > >> > > and even if it does, I think it's fine to accept that as
> a
> >> > > > tradeoff.
> >> > > > > >> > > However, it'd be nice to see the compatibility section
> >> explore
> >> > > > exactly
> >> > > > > >> > what
> >> > > > > >> > > would render a plugin compiled against the newer Connect
> >> API
> >> > > > > >> incompatible
> >> > > > > >> > > with older versions of the Connect runtime. If a plugin
> >> > > explicitly
> >> > > > > >> > > implements ConfigSpecifier, would that break
> >> compatibility? If a
> >> > > > > >> plugin
> >> > > > > >> > > doesn't explicitly implement ConfigSpecifier, would that
> >> break
> >> > > > > >> > > compatibility? These are the two cases that come to mind
> >> > > > immediately
> >> > > > > >> but
> >> > > > > >> > if
> >> > > > > >> > > any others occur to you feel free to document them as
> >> well. This
> >> > > > > >> coverage
> >> > > > > >> > > can also help guide us in how to document the interface
> >> if/when
> >> > > we
> >> > > > > >> add it
> >> > > > > >> > > to make it clear to plugin developers how to avoid
> >> rendering
> >> > > their
> >> > > > > >> work
> >> > > > > >> > > accidentally incompatibile with older versions of the
> >> Connect
> >> > > > runtime.
> >> > > > > >> > >
> >> > > > > >> > > 3. There's also the pluggable ConnectRestExtension
> >> > > > > >> > > and ConnectorClientConfigOverridePolicy interfaces.
> Neither
> >> > > > currently
> >> > > > > >> > > exposes a config method but it wouldn't be out of the
> >> question
> >> > > to
> >> > > > add
> >> > > > > >> one
> >> > > > > >> > > now to them with a default implementation that returns an
> >> empty
> >> > > > > >> > ConfigDef.
> >> > > > > >> > > Thoughts?
> >> > > > > >> > >
> >> > > > > >> > > Cheers,
> >> > > > > >> > >
> >> > > > > >> > > Chris
> >> > > > > >> > >
> >> > > > > >> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via
> >> dev <
> >> > > > > >> > > dev@kafka.apache.org> wrote:
> >> > > > > >> > >
> >> > > > > >> > > > Hi Everyone,
> >> > > > > >> > > >
> >> > > > > >> > > > I would like to start a discussion on KIP-1273: Improve
> >> > > Connect
> >> > > > > >> > > > configurable components discoverability [1].
> >> > > > > >> > > >
> >> > > > > >> > > > Summary:
> >> > > > > >> > > > The idea is to introduce a common ConfigSpecifier
> >> interface
> >> > > for
> >> > > > > >> Kafka
> >> > > > > >> > > > Connect components that expose configuration metadata.
> By
> >> > > > unifying
> >> > > > > >> the
> >> > > > > >> > > > existing config() method across connectors, converters,
> >> > > > > >> > transformations,
> >> > > > > >> > > > and predicates under a single contract, it simplifies
> >> > > component
> >> > > > > >> > > discovery,
> >> > > > > >> > > > reduces code duplication and enables uniform tooling
> for
> >> > > > > >> configuration
> >> > > > > >> > > > introspection, validation, documentation, and UI
> >> generation.
> >> > > The
> >> > > > > >> change
> >> > > > > >> > > is
> >> > > > > >> > > > fully backward compatible and requires no modifications
> >> to
> >> > > > existing
> >> > > > > >> > > > component implementations.
> >> > > > > >> > > >
> >> > > > > >> > > > [1]
> >> > > > > >> > > >
> >> > > > > >> > > >
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > >
> >> > >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> >> > > > > >> > > >
> >> > > > > >> > > > Regards,
> >> > > > > >> > > > Mario
> >> > > > > >> > > >
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > >
> >> > >
> >>
> >
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Mario Fiore Vitale via dev <de...@kafka.apache.org>.
Hi Everyone,
I have updated the KIP document with the ConnectorPlugin. I'll appreciate
any feedback.
Thanks,
Mario.
Il giorno ven 6 feb 2026 alle ore 09:37 Mario Fiore Vitale <
mfvitale@commonhaus.dev> ha scritto:
> Hi Mickael,
>
> I got it. So I'll update the KIP with the ConnectorPlugin interface having
> version() and config() methods.
>
> Thanks,
> Mario
>
> Il giorno gio 5 feb 2026 alle ore 18:24 Mickael Maison <
> mickael.maison@gmail.com> ha scritto:
>
>> Hi,
>>
>> Connectors, transformations, predicates are typically referred as
>> "connector plugins".
>> On the other hand ConnectRestExtension,
>> ConnectorClientConfigOverridePolicy and ConfigProvider are "worker
>> plugins".
>> Yes it's confusing! The terms connectors and plugins are really overused.
>>
>> Mickael
>>
>> On Thu, Feb 5, 2026 at 5:58 PM Mario Fiore Vitale via dev
>> <de...@kafka.apache.org> wrote:
>> >
>> > > Otherwise we could have a ConnectorPlugin interface (putting aside the
>> > worker plugins for now as they don't have a config() method) bringing
>> > config() and version() that all plugins interface would implement?
>> >
>> > So If I get it correctly, you are proposing to have a single interface
>> > (ConnectorPlugin) that replaces the Versioned and the
>> > ConfigSpecifier, merging them.
>> > Sounds reasonable. In that case, I'll use a more generic name like
>> > *ConnectPlugin
>> > *since it will be used not only for connectors but also for
>> transformation,
>> > predicates, and converters.
>> >
>> > Mickael, Chris, WDYT?
>> >
>> > > I like the ConnectorPlugin interface idea. Mario, would that suit
>> your use
>> > case?
>> >
>> > The result is the same, so it is good.
>> >
>> > Thanks,
>> > Mario.
>> >
>> > Il giorno gio 5 feb 2026 alle ore 16:53 Chris Egerton <
>> > fearthecellos@gmail.com> ha scritto:
>> >
>> > > I agree that ConfigSpecifier is a little clunky, but IMO Configured
>> is a
>> > > little too be similar to the existing Configurable interface.
>> > >
>> > > I like the ConnectorPlugin interface idea. Mario, would that suit
>> your use
>> > > case?
>> > >
>> > > On Thu, Feb 5, 2026, 10:24 Mickael Maison <mi...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi Mario,
>> > > >
>> > > > Thanks for the KIP.
>> > > >
>> > > > Today the mechanism to discover plugins and their configurations is
>> > > > the REST API.
>> > > > You can retrieve the list of all installed connectors plugins
>> > > > (connectors, transformations, predicates) via GET
>> > > > /connector-plugins?connectorsOnly=false
>> > > > You can get their configurations via GET
>> > > /connector-plugins/{plugin}/config
>> > > >
>> > > > That said, I'm not against bringing consistency to the plugin APIs,
>> > > > especially as the drawbacks are really minor.
>> > > > In terms of naming I'm not a huge fan of ConfigSpecifier. I don't
>> have
>> > > > a great alternative to suggest.
>> > > > To follow the naming of Versioned, maybe we can consider Configured?
>> > > > Otherwise we could have a ConnectorPlugin interface (putting aside
>> the
>> > > > worker plugins for now as they don't have a config() method)
>> bringing
>> > > > config() and version() that all plugins interface would implement?
>> > > >
>> > > > Thanks,
>> > > > Mickael
>> > > >
>> > > >
>> > > > On Mon, Feb 2, 2026 at 9:48 AM Mario Fiore Vitale via dev
>> > > > <de...@kafka.apache.org> wrote:
>> > > > >
>> > > > > Hi Everyone,
>> > > > >
>> > > > > Just a ping to bring the discussion up.
>> > > > >
>> > > > > Thanks,
>> > > > > Mario.
>> > > > >
>> > > > > Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore Vitale <
>> > > > > mfvitale@commonhaus.dev> ha scritto:
>> > > > >
>> > > > > > Hi Chris,
>> > > > > >
>> > > > > > Thanks for the feedback. I updated the wrong Javadoc.
>> > > > > >
>> > > > > > Any other comments from anybody?
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Mario.
>> > > > > >
>> > > > > > Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
>> > > > > > fearthecellos@gmail.com> ha scritto:
>> > > > > >
>> > > > > >> Hi Mario,
>> > > > > >>
>> > > > > >> Thanks for the updates, especially the comprehensive
>> compatibility
>> > > > > >> section!
>> > > > > >> The KIP looks good to me, though you may want to double-check
>> the
>> > > > example
>> > > > > >> Javadocs on ConnectRestExtension::config
>> > > > > >> and ConnectorClientConfigOverridePolicy::config :)
>> > > > > >>
>> > > > > >> Cheers,
>> > > > > >>
>> > > > > >> Chris
>> > > > > >>
>> > > > > >> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
>> > > > > >> dev@kafka.apache.org> wrote:
>> > > > > >>
>> > > > > >> > Hi Chris, thanks for your valuable feedback!
>> > > > > >> >
>> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a
>> little
>> > > > > >> > strange--is
>> > > > > >> > it meant to refer to "this set of header converters" or
>> should it
>> > > > just
>> > > > > >> be
>> > > > > >> > "this header converter"?
>> > > > > >> >
>> > > > > >> > This was just taken from the current source. But I agree
>> that it
>> > > > should
>> > > > > >> be
>> > > > > >> > "this header converter".
>> > > > > >> >
>> > > > > >> > > For example, there's the obvious case
>> > > > > >> > where a plugin instantiates an object with a left-hand-type
>> of
>> > > > > >> > ConfigSpecifier in its constructor. That plugin would be
>> > > > incompatible
>> > > > > >> with
>> > > > > >> > older versions of Connect.
>> > > > > >> >
>> > > > > >> > Do you have an example of this? As you said, it's unlikely
>> that
>> > > this
>> > > > > >> would
>> > > > > >> > occur in practice, also to me.
>> > > > > >> >
>> > > > > >> > > If a plugin explicitly
>> > > > > >> > implements ConfigSpecifier, would that break compatibility?
>> > > > > >> >
>> > > > > >> > In this case, it should.
>> > > > > >> >
>> > > > > >> > > If a plugin
>> > > > > >> > doesn't explicitly implement ConfigSpecifier, would that
>> break
>> > > > > >> > compatibility?
>> > > > > >> >
>> > > > > >> > This should not break it.
>> > > > > >> >
>> > > > > >> > > These are the two cases that come to mind immediately but
>> if
>> > > > > >> > any others occur to you feel free to document them as well.
>> > > > > >> >
>> > > > > >> > No other cases come to mind. I'll update the compatibility
>> section
>> > > > with
>> > > > > >> the
>> > > > > >> > forward compatibility consideration.
>> > > > > >> >
>> > > > > >> > > There's also the pluggable ConnectRestExtension
>> > > > > >> > and ConnectorClientConfigOverridePolicy interfaces. Neither
>> > > > currently
>> > > > > >> > exposes a config method but it wouldn't be out of the
>> question to
>> > > > add
>> > > > > >> one
>> > > > > >> > now to them with a default implementation that returns an
>> empty
>> > > > > >> ConfigDef.
>> > > > > >> >
>> > > > > >> > Well, I would say that since they are `Configurable`, I could
>> > > expect
>> > > > > >> that
>> > > > > >> > maybe in some future there could also be the possibility to
>> > > declare
>> > > > a
>> > > > > >> > specific configuration.
>> > > > > >> > So your proposal to add it and implement it as a default
>> seems
>> > > good
>> > > > to
>> > > > > >> me.
>> > > > > >> >
>> > > > > >> > Regards,
>> > > > > >> > Mario.
>> > > > > >> >
>> > > > > >> >
>> > > > > >> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
>> > > > > >> > fearthecellos@gmail.com> ha scritto:
>> > > > > >> >
>> > > > > >> > > Hi Mario,
>> > > > > >> > >
>> > > > > >> > > Thanks for the KIP! My thoughts:
>> > > > > >> > >
>> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a
>> little
>> > > > > >> > strange--is
>> > > > > >> > > it meant to refer to "this set of header converters" or
>> should
>> > > it
>> > > > > >> just be
>> > > > > >> > > "this header converter"?
>> > > > > >> > >
>> > > > > >> > > 2. The compatibility problem with Connect is always
>> gnarly. The
>> > > > > >> section
>> > > > > >> > on
>> > > > > >> > > backward compatibility is very well done in that it
>> demonstrates
>> > > > with
>> > > > > >> > > certainty that older plugins will be able to run smoothly
>> on
>> > > newer
>> > > > > >> > versions
>> > > > > >> > > of the Connect runtime. However, there's also the question
>> of
>> > > > whether
>> > > > > >> > newer
>> > > > > >> > > plugins (compiled against a future version of the Connect
>> API
>> > > > with the
>> > > > > >> > > change proposed in this KIP) will continue to be
>> compatible with
>> > > > older
>> > > > > >> > > versions of the Connect runtime. For example, there's the
>> > > obvious
>> > > > case
>> > > > > >> > > where a plugin instantiates an object with a
>> left-hand-type of
>> > > > > >> > > ConfigSpecifier in its constructor. That plugin would be
>> > > > incompatible
>> > > > > >> > with
>> > > > > >> > > older versions of Connect. It's unlikely that this would
>> occur
>> > > in
>> > > > > >> > practice
>> > > > > >> > > and even if it does, I think it's fine to accept that as a
>> > > > tradeoff.
>> > > > > >> > > However, it'd be nice to see the compatibility section
>> explore
>> > > > exactly
>> > > > > >> > what
>> > > > > >> > > would render a plugin compiled against the newer Connect
>> API
>> > > > > >> incompatible
>> > > > > >> > > with older versions of the Connect runtime. If a plugin
>> > > explicitly
>> > > > > >> > > implements ConfigSpecifier, would that break
>> compatibility? If a
>> > > > > >> plugin
>> > > > > >> > > doesn't explicitly implement ConfigSpecifier, would that
>> break
>> > > > > >> > > compatibility? These are the two cases that come to mind
>> > > > immediately
>> > > > > >> but
>> > > > > >> > if
>> > > > > >> > > any others occur to you feel free to document them as
>> well. This
>> > > > > >> coverage
>> > > > > >> > > can also help guide us in how to document the interface
>> if/when
>> > > we
>> > > > > >> add it
>> > > > > >> > > to make it clear to plugin developers how to avoid
>> rendering
>> > > their
>> > > > > >> work
>> > > > > >> > > accidentally incompatibile with older versions of the
>> Connect
>> > > > runtime.
>> > > > > >> > >
>> > > > > >> > > 3. There's also the pluggable ConnectRestExtension
>> > > > > >> > > and ConnectorClientConfigOverridePolicy interfaces. Neither
>> > > > currently
>> > > > > >> > > exposes a config method but it wouldn't be out of the
>> question
>> > > to
>> > > > add
>> > > > > >> one
>> > > > > >> > > now to them with a default implementation that returns an
>> empty
>> > > > > >> > ConfigDef.
>> > > > > >> > > Thoughts?
>> > > > > >> > >
>> > > > > >> > > Cheers,
>> > > > > >> > >
>> > > > > >> > > Chris
>> > > > > >> > >
>> > > > > >> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via
>> dev <
>> > > > > >> > > dev@kafka.apache.org> wrote:
>> > > > > >> > >
>> > > > > >> > > > Hi Everyone,
>> > > > > >> > > >
>> > > > > >> > > > I would like to start a discussion on KIP-1273: Improve
>> > > Connect
>> > > > > >> > > > configurable components discoverability [1].
>> > > > > >> > > >
>> > > > > >> > > > Summary:
>> > > > > >> > > > The idea is to introduce a common ConfigSpecifier
>> interface
>> > > for
>> > > > > >> Kafka
>> > > > > >> > > > Connect components that expose configuration metadata. By
>> > > > unifying
>> > > > > >> the
>> > > > > >> > > > existing config() method across connectors, converters,
>> > > > > >> > transformations,
>> > > > > >> > > > and predicates under a single contract, it simplifies
>> > > component
>> > > > > >> > > discovery,
>> > > > > >> > > > reduces code duplication and enables uniform tooling for
>> > > > > >> configuration
>> > > > > >> > > > introspection, validation, documentation, and UI
>> generation.
>> > > The
>> > > > > >> change
>> > > > > >> > > is
>> > > > > >> > > > fully backward compatible and requires no modifications
>> to
>> > > > existing
>> > > > > >> > > > component implementations.
>> > > > > >> > > >
>> > > > > >> > > > [1]
>> > > > > >> > > >
>> > > > > >> > > >
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > >
>> > >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
>> > > > > >> > > >
>> > > > > >> > > > Regards,
>> > > > > >> > > > Mario
>> > > > > >> > > >
>> > > > > >> > >
>> > > > > >> >
>> > > > > >>
>> > > > > >
>> > > >
>> > >
>>
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Mario Fiore Vitale via dev <de...@kafka.apache.org>.
Hi Mickael,
I got it. So I'll update the KIP with the ConnectorPlugin interface having
version() and config() methods.
Thanks,
Mario
Il giorno gio 5 feb 2026 alle ore 18:24 Mickael Maison <
mickael.maison@gmail.com> ha scritto:
> Hi,
>
> Connectors, transformations, predicates are typically referred as
> "connector plugins".
> On the other hand ConnectRestExtension,
> ConnectorClientConfigOverridePolicy and ConfigProvider are "worker
> plugins".
> Yes it's confusing! The terms connectors and plugins are really overused.
>
> Mickael
>
> On Thu, Feb 5, 2026 at 5:58 PM Mario Fiore Vitale via dev
> <de...@kafka.apache.org> wrote:
> >
> > > Otherwise we could have a ConnectorPlugin interface (putting aside the
> > worker plugins for now as they don't have a config() method) bringing
> > config() and version() that all plugins interface would implement?
> >
> > So If I get it correctly, you are proposing to have a single interface
> > (ConnectorPlugin) that replaces the Versioned and the
> > ConfigSpecifier, merging them.
> > Sounds reasonable. In that case, I'll use a more generic name like
> > *ConnectPlugin
> > *since it will be used not only for connectors but also for
> transformation,
> > predicates, and converters.
> >
> > Mickael, Chris, WDYT?
> >
> > > I like the ConnectorPlugin interface idea. Mario, would that suit your
> use
> > case?
> >
> > The result is the same, so it is good.
> >
> > Thanks,
> > Mario.
> >
> > Il giorno gio 5 feb 2026 alle ore 16:53 Chris Egerton <
> > fearthecellos@gmail.com> ha scritto:
> >
> > > I agree that ConfigSpecifier is a little clunky, but IMO Configured is
> a
> > > little too be similar to the existing Configurable interface.
> > >
> > > I like the ConnectorPlugin interface idea. Mario, would that suit your
> use
> > > case?
> > >
> > > On Thu, Feb 5, 2026, 10:24 Mickael Maison <mi...@gmail.com>
> > > wrote:
> > >
> > > > Hi Mario,
> > > >
> > > > Thanks for the KIP.
> > > >
> > > > Today the mechanism to discover plugins and their configurations is
> > > > the REST API.
> > > > You can retrieve the list of all installed connectors plugins
> > > > (connectors, transformations, predicates) via GET
> > > > /connector-plugins?connectorsOnly=false
> > > > You can get their configurations via GET
> > > /connector-plugins/{plugin}/config
> > > >
> > > > That said, I'm not against bringing consistency to the plugin APIs,
> > > > especially as the drawbacks are really minor.
> > > > In terms of naming I'm not a huge fan of ConfigSpecifier. I don't
> have
> > > > a great alternative to suggest.
> > > > To follow the naming of Versioned, maybe we can consider Configured?
> > > > Otherwise we could have a ConnectorPlugin interface (putting aside
> the
> > > > worker plugins for now as they don't have a config() method) bringing
> > > > config() and version() that all plugins interface would implement?
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
> > > >
> > > > On Mon, Feb 2, 2026 at 9:48 AM Mario Fiore Vitale via dev
> > > > <de...@kafka.apache.org> wrote:
> > > > >
> > > > > Hi Everyone,
> > > > >
> > > > > Just a ping to bring the discussion up.
> > > > >
> > > > > Thanks,
> > > > > Mario.
> > > > >
> > > > > Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore Vitale <
> > > > > mfvitale@commonhaus.dev> ha scritto:
> > > > >
> > > > > > Hi Chris,
> > > > > >
> > > > > > Thanks for the feedback. I updated the wrong Javadoc.
> > > > > >
> > > > > > Any other comments from anybody?
> > > > > >
> > > > > > Thanks,
> > > > > > Mario.
> > > > > >
> > > > > > Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
> > > > > > fearthecellos@gmail.com> ha scritto:
> > > > > >
> > > > > >> Hi Mario,
> > > > > >>
> > > > > >> Thanks for the updates, especially the comprehensive
> compatibility
> > > > > >> section!
> > > > > >> The KIP looks good to me, though you may want to double-check
> the
> > > > example
> > > > > >> Javadocs on ConnectRestExtension::config
> > > > > >> and ConnectorClientConfigOverridePolicy::config :)
> > > > > >>
> > > > > >> Cheers,
> > > > > >>
> > > > > >> Chris
> > > > > >>
> > > > > >> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> > > > > >> dev@kafka.apache.org> wrote:
> > > > > >>
> > > > > >> > Hi Chris, thanks for your valuable feedback!
> > > > > >> >
> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a
> little
> > > > > >> > strange--is
> > > > > >> > it meant to refer to "this set of header converters" or
> should it
> > > > just
> > > > > >> be
> > > > > >> > "this header converter"?
> > > > > >> >
> > > > > >> > This was just taken from the current source. But I agree that
> it
> > > > should
> > > > > >> be
> > > > > >> > "this header converter".
> > > > > >> >
> > > > > >> > > For example, there's the obvious case
> > > > > >> > where a plugin instantiates an object with a left-hand-type of
> > > > > >> > ConfigSpecifier in its constructor. That plugin would be
> > > > incompatible
> > > > > >> with
> > > > > >> > older versions of Connect.
> > > > > >> >
> > > > > >> > Do you have an example of this? As you said, it's unlikely
> that
> > > this
> > > > > >> would
> > > > > >> > occur in practice, also to me.
> > > > > >> >
> > > > > >> > > If a plugin explicitly
> > > > > >> > implements ConfigSpecifier, would that break compatibility?
> > > > > >> >
> > > > > >> > In this case, it should.
> > > > > >> >
> > > > > >> > > If a plugin
> > > > > >> > doesn't explicitly implement ConfigSpecifier, would that break
> > > > > >> > compatibility?
> > > > > >> >
> > > > > >> > This should not break it.
> > > > > >> >
> > > > > >> > > These are the two cases that come to mind immediately but if
> > > > > >> > any others occur to you feel free to document them as well.
> > > > > >> >
> > > > > >> > No other cases come to mind. I'll update the compatibility
> section
> > > > with
> > > > > >> the
> > > > > >> > forward compatibility consideration.
> > > > > >> >
> > > > > >> > > There's also the pluggable ConnectRestExtension
> > > > > >> > and ConnectorClientConfigOverridePolicy interfaces. Neither
> > > > currently
> > > > > >> > exposes a config method but it wouldn't be out of the
> question to
> > > > add
> > > > > >> one
> > > > > >> > now to them with a default implementation that returns an
> empty
> > > > > >> ConfigDef.
> > > > > >> >
> > > > > >> > Well, I would say that since they are `Configurable`, I could
> > > expect
> > > > > >> that
> > > > > >> > maybe in some future there could also be the possibility to
> > > declare
> > > > a
> > > > > >> > specific configuration.
> > > > > >> > So your proposal to add it and implement it as a default seems
> > > good
> > > > to
> > > > > >> me.
> > > > > >> >
> > > > > >> > Regards,
> > > > > >> > Mario.
> > > > > >> >
> > > > > >> >
> > > > > >> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
> > > > > >> > fearthecellos@gmail.com> ha scritto:
> > > > > >> >
> > > > > >> > > Hi Mario,
> > > > > >> > >
> > > > > >> > > Thanks for the KIP! My thoughts:
> > > > > >> > >
> > > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a
> little
> > > > > >> > strange--is
> > > > > >> > > it meant to refer to "this set of header converters" or
> should
> > > it
> > > > > >> just be
> > > > > >> > > "this header converter"?
> > > > > >> > >
> > > > > >> > > 2. The compatibility problem with Connect is always gnarly.
> The
> > > > > >> section
> > > > > >> > on
> > > > > >> > > backward compatibility is very well done in that it
> demonstrates
> > > > with
> > > > > >> > > certainty that older plugins will be able to run smoothly on
> > > newer
> > > > > >> > versions
> > > > > >> > > of the Connect runtime. However, there's also the question
> of
> > > > whether
> > > > > >> > newer
> > > > > >> > > plugins (compiled against a future version of the Connect
> API
> > > > with the
> > > > > >> > > change proposed in this KIP) will continue to be compatible
> with
> > > > older
> > > > > >> > > versions of the Connect runtime. For example, there's the
> > > obvious
> > > > case
> > > > > >> > > where a plugin instantiates an object with a left-hand-type
> of
> > > > > >> > > ConfigSpecifier in its constructor. That plugin would be
> > > > incompatible
> > > > > >> > with
> > > > > >> > > older versions of Connect. It's unlikely that this would
> occur
> > > in
> > > > > >> > practice
> > > > > >> > > and even if it does, I think it's fine to accept that as a
> > > > tradeoff.
> > > > > >> > > However, it'd be nice to see the compatibility section
> explore
> > > > exactly
> > > > > >> > what
> > > > > >> > > would render a plugin compiled against the newer Connect API
> > > > > >> incompatible
> > > > > >> > > with older versions of the Connect runtime. If a plugin
> > > explicitly
> > > > > >> > > implements ConfigSpecifier, would that break compatibility?
> If a
> > > > > >> plugin
> > > > > >> > > doesn't explicitly implement ConfigSpecifier, would that
> break
> > > > > >> > > compatibility? These are the two cases that come to mind
> > > > immediately
> > > > > >> but
> > > > > >> > if
> > > > > >> > > any others occur to you feel free to document them as well.
> This
> > > > > >> coverage
> > > > > >> > > can also help guide us in how to document the interface
> if/when
> > > we
> > > > > >> add it
> > > > > >> > > to make it clear to plugin developers how to avoid rendering
> > > their
> > > > > >> work
> > > > > >> > > accidentally incompatibile with older versions of the
> Connect
> > > > runtime.
> > > > > >> > >
> > > > > >> > > 3. There's also the pluggable ConnectRestExtension
> > > > > >> > > and ConnectorClientConfigOverridePolicy interfaces. Neither
> > > > currently
> > > > > >> > > exposes a config method but it wouldn't be out of the
> question
> > > to
> > > > add
> > > > > >> one
> > > > > >> > > now to them with a default implementation that returns an
> empty
> > > > > >> > ConfigDef.
> > > > > >> > > Thoughts?
> > > > > >> > >
> > > > > >> > > Cheers,
> > > > > >> > >
> > > > > >> > > Chris
> > > > > >> > >
> > > > > >> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via dev
> <
> > > > > >> > > dev@kafka.apache.org> wrote:
> > > > > >> > >
> > > > > >> > > > Hi Everyone,
> > > > > >> > > >
> > > > > >> > > > I would like to start a discussion on KIP-1273: Improve
> > > Connect
> > > > > >> > > > configurable components discoverability [1].
> > > > > >> > > >
> > > > > >> > > > Summary:
> > > > > >> > > > The idea is to introduce a common ConfigSpecifier
> interface
> > > for
> > > > > >> Kafka
> > > > > >> > > > Connect components that expose configuration metadata. By
> > > > unifying
> > > > > >> the
> > > > > >> > > > existing config() method across connectors, converters,
> > > > > >> > transformations,
> > > > > >> > > > and predicates under a single contract, it simplifies
> > > component
> > > > > >> > > discovery,
> > > > > >> > > > reduces code duplication and enables uniform tooling for
> > > > > >> configuration
> > > > > >> > > > introspection, validation, documentation, and UI
> generation.
> > > The
> > > > > >> change
> > > > > >> > > is
> > > > > >> > > > fully backward compatible and requires no modifications to
> > > > existing
> > > > > >> > > > component implementations.
> > > > > >> > > >
> > > > > >> > > > [1]
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> > > > > >> > > >
> > > > > >> > > > Regards,
> > > > > >> > > > Mario
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > >
> > >
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Mickael Maison <mi...@gmail.com>.
Hi,
Connectors, transformations, predicates are typically referred as
"connector plugins".
On the other hand ConnectRestExtension,
ConnectorClientConfigOverridePolicy and ConfigProvider are "worker
plugins".
Yes it's confusing! The terms connectors and plugins are really overused.
Mickael
On Thu, Feb 5, 2026 at 5:58 PM Mario Fiore Vitale via dev
<de...@kafka.apache.org> wrote:
>
> > Otherwise we could have a ConnectorPlugin interface (putting aside the
> worker plugins for now as they don't have a config() method) bringing
> config() and version() that all plugins interface would implement?
>
> So If I get it correctly, you are proposing to have a single interface
> (ConnectorPlugin) that replaces the Versioned and the
> ConfigSpecifier, merging them.
> Sounds reasonable. In that case, I'll use a more generic name like
> *ConnectPlugin
> *since it will be used not only for connectors but also for transformation,
> predicates, and converters.
>
> Mickael, Chris, WDYT?
>
> > I like the ConnectorPlugin interface idea. Mario, would that suit your use
> case?
>
> The result is the same, so it is good.
>
> Thanks,
> Mario.
>
> Il giorno gio 5 feb 2026 alle ore 16:53 Chris Egerton <
> fearthecellos@gmail.com> ha scritto:
>
> > I agree that ConfigSpecifier is a little clunky, but IMO Configured is a
> > little too be similar to the existing Configurable interface.
> >
> > I like the ConnectorPlugin interface idea. Mario, would that suit your use
> > case?
> >
> > On Thu, Feb 5, 2026, 10:24 Mickael Maison <mi...@gmail.com>
> > wrote:
> >
> > > Hi Mario,
> > >
> > > Thanks for the KIP.
> > >
> > > Today the mechanism to discover plugins and their configurations is
> > > the REST API.
> > > You can retrieve the list of all installed connectors plugins
> > > (connectors, transformations, predicates) via GET
> > > /connector-plugins?connectorsOnly=false
> > > You can get their configurations via GET
> > /connector-plugins/{plugin}/config
> > >
> > > That said, I'm not against bringing consistency to the plugin APIs,
> > > especially as the drawbacks are really minor.
> > > In terms of naming I'm not a huge fan of ConfigSpecifier. I don't have
> > > a great alternative to suggest.
> > > To follow the naming of Versioned, maybe we can consider Configured?
> > > Otherwise we could have a ConnectorPlugin interface (putting aside the
> > > worker plugins for now as they don't have a config() method) bringing
> > > config() and version() that all plugins interface would implement?
> > >
> > > Thanks,
> > > Mickael
> > >
> > >
> > > On Mon, Feb 2, 2026 at 9:48 AM Mario Fiore Vitale via dev
> > > <de...@kafka.apache.org> wrote:
> > > >
> > > > Hi Everyone,
> > > >
> > > > Just a ping to bring the discussion up.
> > > >
> > > > Thanks,
> > > > Mario.
> > > >
> > > > Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore Vitale <
> > > > mfvitale@commonhaus.dev> ha scritto:
> > > >
> > > > > Hi Chris,
> > > > >
> > > > > Thanks for the feedback. I updated the wrong Javadoc.
> > > > >
> > > > > Any other comments from anybody?
> > > > >
> > > > > Thanks,
> > > > > Mario.
> > > > >
> > > > > Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
> > > > > fearthecellos@gmail.com> ha scritto:
> > > > >
> > > > >> Hi Mario,
> > > > >>
> > > > >> Thanks for the updates, especially the comprehensive compatibility
> > > > >> section!
> > > > >> The KIP looks good to me, though you may want to double-check the
> > > example
> > > > >> Javadocs on ConnectRestExtension::config
> > > > >> and ConnectorClientConfigOverridePolicy::config :)
> > > > >>
> > > > >> Cheers,
> > > > >>
> > > > >> Chris
> > > > >>
> > > > >> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> > > > >> dev@kafka.apache.org> wrote:
> > > > >>
> > > > >> > Hi Chris, thanks for your valuable feedback!
> > > > >> >
> > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> > > > >> > strange--is
> > > > >> > it meant to refer to "this set of header converters" or should it
> > > just
> > > > >> be
> > > > >> > "this header converter"?
> > > > >> >
> > > > >> > This was just taken from the current source. But I agree that it
> > > should
> > > > >> be
> > > > >> > "this header converter".
> > > > >> >
> > > > >> > > For example, there's the obvious case
> > > > >> > where a plugin instantiates an object with a left-hand-type of
> > > > >> > ConfigSpecifier in its constructor. That plugin would be
> > > incompatible
> > > > >> with
> > > > >> > older versions of Connect.
> > > > >> >
> > > > >> > Do you have an example of this? As you said, it's unlikely that
> > this
> > > > >> would
> > > > >> > occur in practice, also to me.
> > > > >> >
> > > > >> > > If a plugin explicitly
> > > > >> > implements ConfigSpecifier, would that break compatibility?
> > > > >> >
> > > > >> > In this case, it should.
> > > > >> >
> > > > >> > > If a plugin
> > > > >> > doesn't explicitly implement ConfigSpecifier, would that break
> > > > >> > compatibility?
> > > > >> >
> > > > >> > This should not break it.
> > > > >> >
> > > > >> > > These are the two cases that come to mind immediately but if
> > > > >> > any others occur to you feel free to document them as well.
> > > > >> >
> > > > >> > No other cases come to mind. I'll update the compatibility section
> > > with
> > > > >> the
> > > > >> > forward compatibility consideration.
> > > > >> >
> > > > >> > > There's also the pluggable ConnectRestExtension
> > > > >> > and ConnectorClientConfigOverridePolicy interfaces. Neither
> > > currently
> > > > >> > exposes a config method but it wouldn't be out of the question to
> > > add
> > > > >> one
> > > > >> > now to them with a default implementation that returns an empty
> > > > >> ConfigDef.
> > > > >> >
> > > > >> > Well, I would say that since they are `Configurable`, I could
> > expect
> > > > >> that
> > > > >> > maybe in some future there could also be the possibility to
> > declare
> > > a
> > > > >> > specific configuration.
> > > > >> > So your proposal to add it and implement it as a default seems
> > good
> > > to
> > > > >> me.
> > > > >> >
> > > > >> > Regards,
> > > > >> > Mario.
> > > > >> >
> > > > >> >
> > > > >> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
> > > > >> > fearthecellos@gmail.com> ha scritto:
> > > > >> >
> > > > >> > > Hi Mario,
> > > > >> > >
> > > > >> > > Thanks for the KIP! My thoughts:
> > > > >> > >
> > > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> > > > >> > strange--is
> > > > >> > > it meant to refer to "this set of header converters" or should
> > it
> > > > >> just be
> > > > >> > > "this header converter"?
> > > > >> > >
> > > > >> > > 2. The compatibility problem with Connect is always gnarly. The
> > > > >> section
> > > > >> > on
> > > > >> > > backward compatibility is very well done in that it demonstrates
> > > with
> > > > >> > > certainty that older plugins will be able to run smoothly on
> > newer
> > > > >> > versions
> > > > >> > > of the Connect runtime. However, there's also the question of
> > > whether
> > > > >> > newer
> > > > >> > > plugins (compiled against a future version of the Connect API
> > > with the
> > > > >> > > change proposed in this KIP) will continue to be compatible with
> > > older
> > > > >> > > versions of the Connect runtime. For example, there's the
> > obvious
> > > case
> > > > >> > > where a plugin instantiates an object with a left-hand-type of
> > > > >> > > ConfigSpecifier in its constructor. That plugin would be
> > > incompatible
> > > > >> > with
> > > > >> > > older versions of Connect. It's unlikely that this would occur
> > in
> > > > >> > practice
> > > > >> > > and even if it does, I think it's fine to accept that as a
> > > tradeoff.
> > > > >> > > However, it'd be nice to see the compatibility section explore
> > > exactly
> > > > >> > what
> > > > >> > > would render a plugin compiled against the newer Connect API
> > > > >> incompatible
> > > > >> > > with older versions of the Connect runtime. If a plugin
> > explicitly
> > > > >> > > implements ConfigSpecifier, would that break compatibility? If a
> > > > >> plugin
> > > > >> > > doesn't explicitly implement ConfigSpecifier, would that break
> > > > >> > > compatibility? These are the two cases that come to mind
> > > immediately
> > > > >> but
> > > > >> > if
> > > > >> > > any others occur to you feel free to document them as well. This
> > > > >> coverage
> > > > >> > > can also help guide us in how to document the interface if/when
> > we
> > > > >> add it
> > > > >> > > to make it clear to plugin developers how to avoid rendering
> > their
> > > > >> work
> > > > >> > > accidentally incompatibile with older versions of the Connect
> > > runtime.
> > > > >> > >
> > > > >> > > 3. There's also the pluggable ConnectRestExtension
> > > > >> > > and ConnectorClientConfigOverridePolicy interfaces. Neither
> > > currently
> > > > >> > > exposes a config method but it wouldn't be out of the question
> > to
> > > add
> > > > >> one
> > > > >> > > now to them with a default implementation that returns an empty
> > > > >> > ConfigDef.
> > > > >> > > Thoughts?
> > > > >> > >
> > > > >> > > Cheers,
> > > > >> > >
> > > > >> > > Chris
> > > > >> > >
> > > > >> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via dev <
> > > > >> > > dev@kafka.apache.org> wrote:
> > > > >> > >
> > > > >> > > > Hi Everyone,
> > > > >> > > >
> > > > >> > > > I would like to start a discussion on KIP-1273: Improve
> > Connect
> > > > >> > > > configurable components discoverability [1].
> > > > >> > > >
> > > > >> > > > Summary:
> > > > >> > > > The idea is to introduce a common ConfigSpecifier interface
> > for
> > > > >> Kafka
> > > > >> > > > Connect components that expose configuration metadata. By
> > > unifying
> > > > >> the
> > > > >> > > > existing config() method across connectors, converters,
> > > > >> > transformations,
> > > > >> > > > and predicates under a single contract, it simplifies
> > component
> > > > >> > > discovery,
> > > > >> > > > reduces code duplication and enables uniform tooling for
> > > > >> configuration
> > > > >> > > > introspection, validation, documentation, and UI generation.
> > The
> > > > >> change
> > > > >> > > is
> > > > >> > > > fully backward compatible and requires no modifications to
> > > existing
> > > > >> > > > component implementations.
> > > > >> > > >
> > > > >> > > > [1]
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> > > > >> > > >
> > > > >> > > > Regards,
> > > > >> > > > Mario
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > >
> >
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Mario Fiore Vitale via dev <de...@kafka.apache.org>.
> Otherwise we could have a ConnectorPlugin interface (putting aside the
worker plugins for now as they don't have a config() method) bringing
config() and version() that all plugins interface would implement?
So If I get it correctly, you are proposing to have a single interface
(ConnectorPlugin) that replaces the Versioned and the
ConfigSpecifier, merging them.
Sounds reasonable. In that case, I'll use a more generic name like
*ConnectPlugin
*since it will be used not only for connectors but also for transformation,
predicates, and converters.
Mickael, Chris, WDYT?
> I like the ConnectorPlugin interface idea. Mario, would that suit your use
case?
The result is the same, so it is good.
Thanks,
Mario.
Il giorno gio 5 feb 2026 alle ore 16:53 Chris Egerton <
fearthecellos@gmail.com> ha scritto:
> I agree that ConfigSpecifier is a little clunky, but IMO Configured is a
> little too be similar to the existing Configurable interface.
>
> I like the ConnectorPlugin interface idea. Mario, would that suit your use
> case?
>
> On Thu, Feb 5, 2026, 10:24 Mickael Maison <mi...@gmail.com>
> wrote:
>
> > Hi Mario,
> >
> > Thanks for the KIP.
> >
> > Today the mechanism to discover plugins and their configurations is
> > the REST API.
> > You can retrieve the list of all installed connectors plugins
> > (connectors, transformations, predicates) via GET
> > /connector-plugins?connectorsOnly=false
> > You can get their configurations via GET
> /connector-plugins/{plugin}/config
> >
> > That said, I'm not against bringing consistency to the plugin APIs,
> > especially as the drawbacks are really minor.
> > In terms of naming I'm not a huge fan of ConfigSpecifier. I don't have
> > a great alternative to suggest.
> > To follow the naming of Versioned, maybe we can consider Configured?
> > Otherwise we could have a ConnectorPlugin interface (putting aside the
> > worker plugins for now as they don't have a config() method) bringing
> > config() and version() that all plugins interface would implement?
> >
> > Thanks,
> > Mickael
> >
> >
> > On Mon, Feb 2, 2026 at 9:48 AM Mario Fiore Vitale via dev
> > <de...@kafka.apache.org> wrote:
> > >
> > > Hi Everyone,
> > >
> > > Just a ping to bring the discussion up.
> > >
> > > Thanks,
> > > Mario.
> > >
> > > Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore Vitale <
> > > mfvitale@commonhaus.dev> ha scritto:
> > >
> > > > Hi Chris,
> > > >
> > > > Thanks for the feedback. I updated the wrong Javadoc.
> > > >
> > > > Any other comments from anybody?
> > > >
> > > > Thanks,
> > > > Mario.
> > > >
> > > > Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
> > > > fearthecellos@gmail.com> ha scritto:
> > > >
> > > >> Hi Mario,
> > > >>
> > > >> Thanks for the updates, especially the comprehensive compatibility
> > > >> section!
> > > >> The KIP looks good to me, though you may want to double-check the
> > example
> > > >> Javadocs on ConnectRestExtension::config
> > > >> and ConnectorClientConfigOverridePolicy::config :)
> > > >>
> > > >> Cheers,
> > > >>
> > > >> Chris
> > > >>
> > > >> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> > > >> dev@kafka.apache.org> wrote:
> > > >>
> > > >> > Hi Chris, thanks for your valuable feedback!
> > > >> >
> > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> > > >> > strange--is
> > > >> > it meant to refer to "this set of header converters" or should it
> > just
> > > >> be
> > > >> > "this header converter"?
> > > >> >
> > > >> > This was just taken from the current source. But I agree that it
> > should
> > > >> be
> > > >> > "this header converter".
> > > >> >
> > > >> > > For example, there's the obvious case
> > > >> > where a plugin instantiates an object with a left-hand-type of
> > > >> > ConfigSpecifier in its constructor. That plugin would be
> > incompatible
> > > >> with
> > > >> > older versions of Connect.
> > > >> >
> > > >> > Do you have an example of this? As you said, it's unlikely that
> this
> > > >> would
> > > >> > occur in practice, also to me.
> > > >> >
> > > >> > > If a plugin explicitly
> > > >> > implements ConfigSpecifier, would that break compatibility?
> > > >> >
> > > >> > In this case, it should.
> > > >> >
> > > >> > > If a plugin
> > > >> > doesn't explicitly implement ConfigSpecifier, would that break
> > > >> > compatibility?
> > > >> >
> > > >> > This should not break it.
> > > >> >
> > > >> > > These are the two cases that come to mind immediately but if
> > > >> > any others occur to you feel free to document them as well.
> > > >> >
> > > >> > No other cases come to mind. I'll update the compatibility section
> > with
> > > >> the
> > > >> > forward compatibility consideration.
> > > >> >
> > > >> > > There's also the pluggable ConnectRestExtension
> > > >> > and ConnectorClientConfigOverridePolicy interfaces. Neither
> > currently
> > > >> > exposes a config method but it wouldn't be out of the question to
> > add
> > > >> one
> > > >> > now to them with a default implementation that returns an empty
> > > >> ConfigDef.
> > > >> >
> > > >> > Well, I would say that since they are `Configurable`, I could
> expect
> > > >> that
> > > >> > maybe in some future there could also be the possibility to
> declare
> > a
> > > >> > specific configuration.
> > > >> > So your proposal to add it and implement it as a default seems
> good
> > to
> > > >> me.
> > > >> >
> > > >> > Regards,
> > > >> > Mario.
> > > >> >
> > > >> >
> > > >> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
> > > >> > fearthecellos@gmail.com> ha scritto:
> > > >> >
> > > >> > > Hi Mario,
> > > >> > >
> > > >> > > Thanks for the KIP! My thoughts:
> > > >> > >
> > > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> > > >> > strange--is
> > > >> > > it meant to refer to "this set of header converters" or should
> it
> > > >> just be
> > > >> > > "this header converter"?
> > > >> > >
> > > >> > > 2. The compatibility problem with Connect is always gnarly. The
> > > >> section
> > > >> > on
> > > >> > > backward compatibility is very well done in that it demonstrates
> > with
> > > >> > > certainty that older plugins will be able to run smoothly on
> newer
> > > >> > versions
> > > >> > > of the Connect runtime. However, there's also the question of
> > whether
> > > >> > newer
> > > >> > > plugins (compiled against a future version of the Connect API
> > with the
> > > >> > > change proposed in this KIP) will continue to be compatible with
> > older
> > > >> > > versions of the Connect runtime. For example, there's the
> obvious
> > case
> > > >> > > where a plugin instantiates an object with a left-hand-type of
> > > >> > > ConfigSpecifier in its constructor. That plugin would be
> > incompatible
> > > >> > with
> > > >> > > older versions of Connect. It's unlikely that this would occur
> in
> > > >> > practice
> > > >> > > and even if it does, I think it's fine to accept that as a
> > tradeoff.
> > > >> > > However, it'd be nice to see the compatibility section explore
> > exactly
> > > >> > what
> > > >> > > would render a plugin compiled against the newer Connect API
> > > >> incompatible
> > > >> > > with older versions of the Connect runtime. If a plugin
> explicitly
> > > >> > > implements ConfigSpecifier, would that break compatibility? If a
> > > >> plugin
> > > >> > > doesn't explicitly implement ConfigSpecifier, would that break
> > > >> > > compatibility? These are the two cases that come to mind
> > immediately
> > > >> but
> > > >> > if
> > > >> > > any others occur to you feel free to document them as well. This
> > > >> coverage
> > > >> > > can also help guide us in how to document the interface if/when
> we
> > > >> add it
> > > >> > > to make it clear to plugin developers how to avoid rendering
> their
> > > >> work
> > > >> > > accidentally incompatibile with older versions of the Connect
> > runtime.
> > > >> > >
> > > >> > > 3. There's also the pluggable ConnectRestExtension
> > > >> > > and ConnectorClientConfigOverridePolicy interfaces. Neither
> > currently
> > > >> > > exposes a config method but it wouldn't be out of the question
> to
> > add
> > > >> one
> > > >> > > now to them with a default implementation that returns an empty
> > > >> > ConfigDef.
> > > >> > > Thoughts?
> > > >> > >
> > > >> > > Cheers,
> > > >> > >
> > > >> > > Chris
> > > >> > >
> > > >> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via dev <
> > > >> > > dev@kafka.apache.org> wrote:
> > > >> > >
> > > >> > > > Hi Everyone,
> > > >> > > >
> > > >> > > > I would like to start a discussion on KIP-1273: Improve
> Connect
> > > >> > > > configurable components discoverability [1].
> > > >> > > >
> > > >> > > > Summary:
> > > >> > > > The idea is to introduce a common ConfigSpecifier interface
> for
> > > >> Kafka
> > > >> > > > Connect components that expose configuration metadata. By
> > unifying
> > > >> the
> > > >> > > > existing config() method across connectors, converters,
> > > >> > transformations,
> > > >> > > > and predicates under a single contract, it simplifies
> component
> > > >> > > discovery,
> > > >> > > > reduces code duplication and enables uniform tooling for
> > > >> configuration
> > > >> > > > introspection, validation, documentation, and UI generation.
> The
> > > >> change
> > > >> > > is
> > > >> > > > fully backward compatible and requires no modifications to
> > existing
> > > >> > > > component implementations.
> > > >> > > >
> > > >> > > > [1]
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> > > >> > > >
> > > >> > > > Regards,
> > > >> > > > Mario
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> >
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Chris Egerton <fe...@gmail.com>.
I agree that ConfigSpecifier is a little clunky, but IMO Configured is a
little too be similar to the existing Configurable interface.
I like the ConnectorPlugin interface idea. Mario, would that suit your use
case?
On Thu, Feb 5, 2026, 10:24 Mickael Maison <mi...@gmail.com> wrote:
> Hi Mario,
>
> Thanks for the KIP.
>
> Today the mechanism to discover plugins and their configurations is
> the REST API.
> You can retrieve the list of all installed connectors plugins
> (connectors, transformations, predicates) via GET
> /connector-plugins?connectorsOnly=false
> You can get their configurations via GET /connector-plugins/{plugin}/config
>
> That said, I'm not against bringing consistency to the plugin APIs,
> especially as the drawbacks are really minor.
> In terms of naming I'm not a huge fan of ConfigSpecifier. I don't have
> a great alternative to suggest.
> To follow the naming of Versioned, maybe we can consider Configured?
> Otherwise we could have a ConnectorPlugin interface (putting aside the
> worker plugins for now as they don't have a config() method) bringing
> config() and version() that all plugins interface would implement?
>
> Thanks,
> Mickael
>
>
> On Mon, Feb 2, 2026 at 9:48 AM Mario Fiore Vitale via dev
> <de...@kafka.apache.org> wrote:
> >
> > Hi Everyone,
> >
> > Just a ping to bring the discussion up.
> >
> > Thanks,
> > Mario.
> >
> > Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore Vitale <
> > mfvitale@commonhaus.dev> ha scritto:
> >
> > > Hi Chris,
> > >
> > > Thanks for the feedback. I updated the wrong Javadoc.
> > >
> > > Any other comments from anybody?
> > >
> > > Thanks,
> > > Mario.
> > >
> > > Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
> > > fearthecellos@gmail.com> ha scritto:
> > >
> > >> Hi Mario,
> > >>
> > >> Thanks for the updates, especially the comprehensive compatibility
> > >> section!
> > >> The KIP looks good to me, though you may want to double-check the
> example
> > >> Javadocs on ConnectRestExtension::config
> > >> and ConnectorClientConfigOverridePolicy::config :)
> > >>
> > >> Cheers,
> > >>
> > >> Chris
> > >>
> > >> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> > >> dev@kafka.apache.org> wrote:
> > >>
> > >> > Hi Chris, thanks for your valuable feedback!
> > >> >
> > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> > >> > strange--is
> > >> > it meant to refer to "this set of header converters" or should it
> just
> > >> be
> > >> > "this header converter"?
> > >> >
> > >> > This was just taken from the current source. But I agree that it
> should
> > >> be
> > >> > "this header converter".
> > >> >
> > >> > > For example, there's the obvious case
> > >> > where a plugin instantiates an object with a left-hand-type of
> > >> > ConfigSpecifier in its constructor. That plugin would be
> incompatible
> > >> with
> > >> > older versions of Connect.
> > >> >
> > >> > Do you have an example of this? As you said, it's unlikely that this
> > >> would
> > >> > occur in practice, also to me.
> > >> >
> > >> > > If a plugin explicitly
> > >> > implements ConfigSpecifier, would that break compatibility?
> > >> >
> > >> > In this case, it should.
> > >> >
> > >> > > If a plugin
> > >> > doesn't explicitly implement ConfigSpecifier, would that break
> > >> > compatibility?
> > >> >
> > >> > This should not break it.
> > >> >
> > >> > > These are the two cases that come to mind immediately but if
> > >> > any others occur to you feel free to document them as well.
> > >> >
> > >> > No other cases come to mind. I'll update the compatibility section
> with
> > >> the
> > >> > forward compatibility consideration.
> > >> >
> > >> > > There's also the pluggable ConnectRestExtension
> > >> > and ConnectorClientConfigOverridePolicy interfaces. Neither
> currently
> > >> > exposes a config method but it wouldn't be out of the question to
> add
> > >> one
> > >> > now to them with a default implementation that returns an empty
> > >> ConfigDef.
> > >> >
> > >> > Well, I would say that since they are `Configurable`, I could expect
> > >> that
> > >> > maybe in some future there could also be the possibility to declare
> a
> > >> > specific configuration.
> > >> > So your proposal to add it and implement it as a default seems good
> to
> > >> me.
> > >> >
> > >> > Regards,
> > >> > Mario.
> > >> >
> > >> >
> > >> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
> > >> > fearthecellos@gmail.com> ha scritto:
> > >> >
> > >> > > Hi Mario,
> > >> > >
> > >> > > Thanks for the KIP! My thoughts:
> > >> > >
> > >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> > >> > strange--is
> > >> > > it meant to refer to "this set of header converters" or should it
> > >> just be
> > >> > > "this header converter"?
> > >> > >
> > >> > > 2. The compatibility problem with Connect is always gnarly. The
> > >> section
> > >> > on
> > >> > > backward compatibility is very well done in that it demonstrates
> with
> > >> > > certainty that older plugins will be able to run smoothly on newer
> > >> > versions
> > >> > > of the Connect runtime. However, there's also the question of
> whether
> > >> > newer
> > >> > > plugins (compiled against a future version of the Connect API
> with the
> > >> > > change proposed in this KIP) will continue to be compatible with
> older
> > >> > > versions of the Connect runtime. For example, there's the obvious
> case
> > >> > > where a plugin instantiates an object with a left-hand-type of
> > >> > > ConfigSpecifier in its constructor. That plugin would be
> incompatible
> > >> > with
> > >> > > older versions of Connect. It's unlikely that this would occur in
> > >> > practice
> > >> > > and even if it does, I think it's fine to accept that as a
> tradeoff.
> > >> > > However, it'd be nice to see the compatibility section explore
> exactly
> > >> > what
> > >> > > would render a plugin compiled against the newer Connect API
> > >> incompatible
> > >> > > with older versions of the Connect runtime. If a plugin explicitly
> > >> > > implements ConfigSpecifier, would that break compatibility? If a
> > >> plugin
> > >> > > doesn't explicitly implement ConfigSpecifier, would that break
> > >> > > compatibility? These are the two cases that come to mind
> immediately
> > >> but
> > >> > if
> > >> > > any others occur to you feel free to document them as well. This
> > >> coverage
> > >> > > can also help guide us in how to document the interface if/when we
> > >> add it
> > >> > > to make it clear to plugin developers how to avoid rendering their
> > >> work
> > >> > > accidentally incompatibile with older versions of the Connect
> runtime.
> > >> > >
> > >> > > 3. There's also the pluggable ConnectRestExtension
> > >> > > and ConnectorClientConfigOverridePolicy interfaces. Neither
> currently
> > >> > > exposes a config method but it wouldn't be out of the question to
> add
> > >> one
> > >> > > now to them with a default implementation that returns an empty
> > >> > ConfigDef.
> > >> > > Thoughts?
> > >> > >
> > >> > > Cheers,
> > >> > >
> > >> > > Chris
> > >> > >
> > >> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via dev <
> > >> > > dev@kafka.apache.org> wrote:
> > >> > >
> > >> > > > Hi Everyone,
> > >> > > >
> > >> > > > I would like to start a discussion on KIP-1273: Improve Connect
> > >> > > > configurable components discoverability [1].
> > >> > > >
> > >> > > > Summary:
> > >> > > > The idea is to introduce a common ConfigSpecifier interface for
> > >> Kafka
> > >> > > > Connect components that expose configuration metadata. By
> unifying
> > >> the
> > >> > > > existing config() method across connectors, converters,
> > >> > transformations,
> > >> > > > and predicates under a single contract, it simplifies component
> > >> > > discovery,
> > >> > > > reduces code duplication and enables uniform tooling for
> > >> configuration
> > >> > > > introspection, validation, documentation, and UI generation. The
> > >> change
> > >> > > is
> > >> > > > fully backward compatible and requires no modifications to
> existing
> > >> > > > component implementations.
> > >> > > >
> > >> > > > [1]
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> > >> > > >
> > >> > > > Regards,
> > >> > > > Mario
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Mickael Maison <mi...@gmail.com>.
Hi Mario,
Thanks for the KIP.
Today the mechanism to discover plugins and their configurations is
the REST API.
You can retrieve the list of all installed connectors plugins
(connectors, transformations, predicates) via GET
/connector-plugins?connectorsOnly=false
You can get their configurations via GET /connector-plugins/{plugin}/config
That said, I'm not against bringing consistency to the plugin APIs,
especially as the drawbacks are really minor.
In terms of naming I'm not a huge fan of ConfigSpecifier. I don't have
a great alternative to suggest.
To follow the naming of Versioned, maybe we can consider Configured?
Otherwise we could have a ConnectorPlugin interface (putting aside the
worker plugins for now as they don't have a config() method) bringing
config() and version() that all plugins interface would implement?
Thanks,
Mickael
On Mon, Feb 2, 2026 at 9:48 AM Mario Fiore Vitale via dev
<de...@kafka.apache.org> wrote:
>
> Hi Everyone,
>
> Just a ping to bring the discussion up.
>
> Thanks,
> Mario.
>
> Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore Vitale <
> mfvitale@commonhaus.dev> ha scritto:
>
> > Hi Chris,
> >
> > Thanks for the feedback. I updated the wrong Javadoc.
> >
> > Any other comments from anybody?
> >
> > Thanks,
> > Mario.
> >
> > Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
> > fearthecellos@gmail.com> ha scritto:
> >
> >> Hi Mario,
> >>
> >> Thanks for the updates, especially the comprehensive compatibility
> >> section!
> >> The KIP looks good to me, though you may want to double-check the example
> >> Javadocs on ConnectRestExtension::config
> >> and ConnectorClientConfigOverridePolicy::config :)
> >>
> >> Cheers,
> >>
> >> Chris
> >>
> >> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> >> dev@kafka.apache.org> wrote:
> >>
> >> > Hi Chris, thanks for your valuable feedback!
> >> >
> >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> >> > strange--is
> >> > it meant to refer to "this set of header converters" or should it just
> >> be
> >> > "this header converter"?
> >> >
> >> > This was just taken from the current source. But I agree that it should
> >> be
> >> > "this header converter".
> >> >
> >> > > For example, there's the obvious case
> >> > where a plugin instantiates an object with a left-hand-type of
> >> > ConfigSpecifier in its constructor. That plugin would be incompatible
> >> with
> >> > older versions of Connect.
> >> >
> >> > Do you have an example of this? As you said, it's unlikely that this
> >> would
> >> > occur in practice, also to me.
> >> >
> >> > > If a plugin explicitly
> >> > implements ConfigSpecifier, would that break compatibility?
> >> >
> >> > In this case, it should.
> >> >
> >> > > If a plugin
> >> > doesn't explicitly implement ConfigSpecifier, would that break
> >> > compatibility?
> >> >
> >> > This should not break it.
> >> >
> >> > > These are the two cases that come to mind immediately but if
> >> > any others occur to you feel free to document them as well.
> >> >
> >> > No other cases come to mind. I'll update the compatibility section with
> >> the
> >> > forward compatibility consideration.
> >> >
> >> > > There's also the pluggable ConnectRestExtension
> >> > and ConnectorClientConfigOverridePolicy interfaces. Neither currently
> >> > exposes a config method but it wouldn't be out of the question to add
> >> one
> >> > now to them with a default implementation that returns an empty
> >> ConfigDef.
> >> >
> >> > Well, I would say that since they are `Configurable`, I could expect
> >> that
> >> > maybe in some future there could also be the possibility to declare a
> >> > specific configuration.
> >> > So your proposal to add it and implement it as a default seems good to
> >> me.
> >> >
> >> > Regards,
> >> > Mario.
> >> >
> >> >
> >> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
> >> > fearthecellos@gmail.com> ha scritto:
> >> >
> >> > > Hi Mario,
> >> > >
> >> > > Thanks for the KIP! My thoughts:
> >> > >
> >> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> >> > strange--is
> >> > > it meant to refer to "this set of header converters" or should it
> >> just be
> >> > > "this header converter"?
> >> > >
> >> > > 2. The compatibility problem with Connect is always gnarly. The
> >> section
> >> > on
> >> > > backward compatibility is very well done in that it demonstrates with
> >> > > certainty that older plugins will be able to run smoothly on newer
> >> > versions
> >> > > of the Connect runtime. However, there's also the question of whether
> >> > newer
> >> > > plugins (compiled against a future version of the Connect API with the
> >> > > change proposed in this KIP) will continue to be compatible with older
> >> > > versions of the Connect runtime. For example, there's the obvious case
> >> > > where a plugin instantiates an object with a left-hand-type of
> >> > > ConfigSpecifier in its constructor. That plugin would be incompatible
> >> > with
> >> > > older versions of Connect. It's unlikely that this would occur in
> >> > practice
> >> > > and even if it does, I think it's fine to accept that as a tradeoff.
> >> > > However, it'd be nice to see the compatibility section explore exactly
> >> > what
> >> > > would render a plugin compiled against the newer Connect API
> >> incompatible
> >> > > with older versions of the Connect runtime. If a plugin explicitly
> >> > > implements ConfigSpecifier, would that break compatibility? If a
> >> plugin
> >> > > doesn't explicitly implement ConfigSpecifier, would that break
> >> > > compatibility? These are the two cases that come to mind immediately
> >> but
> >> > if
> >> > > any others occur to you feel free to document them as well. This
> >> coverage
> >> > > can also help guide us in how to document the interface if/when we
> >> add it
> >> > > to make it clear to plugin developers how to avoid rendering their
> >> work
> >> > > accidentally incompatibile with older versions of the Connect runtime.
> >> > >
> >> > > 3. There's also the pluggable ConnectRestExtension
> >> > > and ConnectorClientConfigOverridePolicy interfaces. Neither currently
> >> > > exposes a config method but it wouldn't be out of the question to add
> >> one
> >> > > now to them with a default implementation that returns an empty
> >> > ConfigDef.
> >> > > Thoughts?
> >> > >
> >> > > Cheers,
> >> > >
> >> > > Chris
> >> > >
> >> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via dev <
> >> > > dev@kafka.apache.org> wrote:
> >> > >
> >> > > > Hi Everyone,
> >> > > >
> >> > > > I would like to start a discussion on KIP-1273: Improve Connect
> >> > > > configurable components discoverability [1].
> >> > > >
> >> > > > Summary:
> >> > > > The idea is to introduce a common ConfigSpecifier interface for
> >> Kafka
> >> > > > Connect components that expose configuration metadata. By unifying
> >> the
> >> > > > existing config() method across connectors, converters,
> >> > transformations,
> >> > > > and predicates under a single contract, it simplifies component
> >> > > discovery,
> >> > > > reduces code duplication and enables uniform tooling for
> >> configuration
> >> > > > introspection, validation, documentation, and UI generation. The
> >> change
> >> > > is
> >> > > > fully backward compatible and requires no modifications to existing
> >> > > > component implementations.
> >> > > >
> >> > > > [1]
> >> > > >
> >> > > >
> >> > >
> >> >
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> >> > > >
> >> > > > Regards,
> >> > > > Mario
> >> > > >
> >> > >
> >> >
> >>
> >
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Mario Fiore Vitale via dev <de...@kafka.apache.org>.
Hi Everyone,
Just a ping to bring the discussion up.
Thanks,
Mario.
Il giorno mar 27 gen 2026 alle ore 09:17 Mario Fiore Vitale <
mfvitale@commonhaus.dev> ha scritto:
> Hi Chris,
>
> Thanks for the feedback. I updated the wrong Javadoc.
>
> Any other comments from anybody?
>
> Thanks,
> Mario.
>
> Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
> fearthecellos@gmail.com> ha scritto:
>
>> Hi Mario,
>>
>> Thanks for the updates, especially the comprehensive compatibility
>> section!
>> The KIP looks good to me, though you may want to double-check the example
>> Javadocs on ConnectRestExtension::config
>> and ConnectorClientConfigOverridePolicy::config :)
>>
>> Cheers,
>>
>> Chris
>>
>> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
>> dev@kafka.apache.org> wrote:
>>
>> > Hi Chris, thanks for your valuable feedback!
>> >
>> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
>> > strange--is
>> > it meant to refer to "this set of header converters" or should it just
>> be
>> > "this header converter"?
>> >
>> > This was just taken from the current source. But I agree that it should
>> be
>> > "this header converter".
>> >
>> > > For example, there's the obvious case
>> > where a plugin instantiates an object with a left-hand-type of
>> > ConfigSpecifier in its constructor. That plugin would be incompatible
>> with
>> > older versions of Connect.
>> >
>> > Do you have an example of this? As you said, it's unlikely that this
>> would
>> > occur in practice, also to me.
>> >
>> > > If a plugin explicitly
>> > implements ConfigSpecifier, would that break compatibility?
>> >
>> > In this case, it should.
>> >
>> > > If a plugin
>> > doesn't explicitly implement ConfigSpecifier, would that break
>> > compatibility?
>> >
>> > This should not break it.
>> >
>> > > These are the two cases that come to mind immediately but if
>> > any others occur to you feel free to document them as well.
>> >
>> > No other cases come to mind. I'll update the compatibility section with
>> the
>> > forward compatibility consideration.
>> >
>> > > There's also the pluggable ConnectRestExtension
>> > and ConnectorClientConfigOverridePolicy interfaces. Neither currently
>> > exposes a config method but it wouldn't be out of the question to add
>> one
>> > now to them with a default implementation that returns an empty
>> ConfigDef.
>> >
>> > Well, I would say that since they are `Configurable`, I could expect
>> that
>> > maybe in some future there could also be the possibility to declare a
>> > specific configuration.
>> > So your proposal to add it and implement it as a default seems good to
>> me.
>> >
>> > Regards,
>> > Mario.
>> >
>> >
>> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
>> > fearthecellos@gmail.com> ha scritto:
>> >
>> > > Hi Mario,
>> > >
>> > > Thanks for the KIP! My thoughts:
>> > >
>> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
>> > strange--is
>> > > it meant to refer to "this set of header converters" or should it
>> just be
>> > > "this header converter"?
>> > >
>> > > 2. The compatibility problem with Connect is always gnarly. The
>> section
>> > on
>> > > backward compatibility is very well done in that it demonstrates with
>> > > certainty that older plugins will be able to run smoothly on newer
>> > versions
>> > > of the Connect runtime. However, there's also the question of whether
>> > newer
>> > > plugins (compiled against a future version of the Connect API with the
>> > > change proposed in this KIP) will continue to be compatible with older
>> > > versions of the Connect runtime. For example, there's the obvious case
>> > > where a plugin instantiates an object with a left-hand-type of
>> > > ConfigSpecifier in its constructor. That plugin would be incompatible
>> > with
>> > > older versions of Connect. It's unlikely that this would occur in
>> > practice
>> > > and even if it does, I think it's fine to accept that as a tradeoff.
>> > > However, it'd be nice to see the compatibility section explore exactly
>> > what
>> > > would render a plugin compiled against the newer Connect API
>> incompatible
>> > > with older versions of the Connect runtime. If a plugin explicitly
>> > > implements ConfigSpecifier, would that break compatibility? If a
>> plugin
>> > > doesn't explicitly implement ConfigSpecifier, would that break
>> > > compatibility? These are the two cases that come to mind immediately
>> but
>> > if
>> > > any others occur to you feel free to document them as well. This
>> coverage
>> > > can also help guide us in how to document the interface if/when we
>> add it
>> > > to make it clear to plugin developers how to avoid rendering their
>> work
>> > > accidentally incompatibile with older versions of the Connect runtime.
>> > >
>> > > 3. There's also the pluggable ConnectRestExtension
>> > > and ConnectorClientConfigOverridePolicy interfaces. Neither currently
>> > > exposes a config method but it wouldn't be out of the question to add
>> one
>> > > now to them with a default implementation that returns an empty
>> > ConfigDef.
>> > > Thoughts?
>> > >
>> > > Cheers,
>> > >
>> > > Chris
>> > >
>> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via dev <
>> > > dev@kafka.apache.org> wrote:
>> > >
>> > > > Hi Everyone,
>> > > >
>> > > > I would like to start a discussion on KIP-1273: Improve Connect
>> > > > configurable components discoverability [1].
>> > > >
>> > > > Summary:
>> > > > The idea is to introduce a common ConfigSpecifier interface for
>> Kafka
>> > > > Connect components that expose configuration metadata. By unifying
>> the
>> > > > existing config() method across connectors, converters,
>> > transformations,
>> > > > and predicates under a single contract, it simplifies component
>> > > discovery,
>> > > > reduces code duplication and enables uniform tooling for
>> configuration
>> > > > introspection, validation, documentation, and UI generation. The
>> change
>> > > is
>> > > > fully backward compatible and requires no modifications to existing
>> > > > component implementations.
>> > > >
>> > > > [1]
>> > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
>> > > >
>> > > > Regards,
>> > > > Mario
>> > > >
>> > >
>> >
>>
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Mario Fiore Vitale via dev <de...@kafka.apache.org>.
Hi Chris,
Thanks for the feedback. I updated the wrong Javadoc.
Any other comments from anybody?
Thanks,
Mario.
Il giorno ven 23 gen 2026 alle ore 07:31 Chris Egerton <
fearthecellos@gmail.com> ha scritto:
> Hi Mario,
>
> Thanks for the updates, especially the comprehensive compatibility section!
> The KIP looks good to me, though you may want to double-check the example
> Javadocs on ConnectRestExtension::config
> and ConnectorClientConfigOverridePolicy::config :)
>
> Cheers,
>
> Chris
>
> On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
> dev@kafka.apache.org> wrote:
>
> > Hi Chris, thanks for your valuable feedback!
> >
> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> > strange--is
> > it meant to refer to "this set of header converters" or should it just be
> > "this header converter"?
> >
> > This was just taken from the current source. But I agree that it should
> be
> > "this header converter".
> >
> > > For example, there's the obvious case
> > where a plugin instantiates an object with a left-hand-type of
> > ConfigSpecifier in its constructor. That plugin would be incompatible
> with
> > older versions of Connect.
> >
> > Do you have an example of this? As you said, it's unlikely that this
> would
> > occur in practice, also to me.
> >
> > > If a plugin explicitly
> > implements ConfigSpecifier, would that break compatibility?
> >
> > In this case, it should.
> >
> > > If a plugin
> > doesn't explicitly implement ConfigSpecifier, would that break
> > compatibility?
> >
> > This should not break it.
> >
> > > These are the two cases that come to mind immediately but if
> > any others occur to you feel free to document them as well.
> >
> > No other cases come to mind. I'll update the compatibility section with
> the
> > forward compatibility consideration.
> >
> > > There's also the pluggable ConnectRestExtension
> > and ConnectorClientConfigOverridePolicy interfaces. Neither currently
> > exposes a config method but it wouldn't be out of the question to add one
> > now to them with a default implementation that returns an empty
> ConfigDef.
> >
> > Well, I would say that since they are `Configurable`, I could expect that
> > maybe in some future there could also be the possibility to declare a
> > specific configuration.
> > So your proposal to add it and implement it as a default seems good to
> me.
> >
> > Regards,
> > Mario.
> >
> >
> > Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
> > fearthecellos@gmail.com> ha scritto:
> >
> > > Hi Mario,
> > >
> > > Thanks for the KIP! My thoughts:
> > >
> > > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> > strange--is
> > > it meant to refer to "this set of header converters" or should it just
> be
> > > "this header converter"?
> > >
> > > 2. The compatibility problem with Connect is always gnarly. The section
> > on
> > > backward compatibility is very well done in that it demonstrates with
> > > certainty that older plugins will be able to run smoothly on newer
> > versions
> > > of the Connect runtime. However, there's also the question of whether
> > newer
> > > plugins (compiled against a future version of the Connect API with the
> > > change proposed in this KIP) will continue to be compatible with older
> > > versions of the Connect runtime. For example, there's the obvious case
> > > where a plugin instantiates an object with a left-hand-type of
> > > ConfigSpecifier in its constructor. That plugin would be incompatible
> > with
> > > older versions of Connect. It's unlikely that this would occur in
> > practice
> > > and even if it does, I think it's fine to accept that as a tradeoff.
> > > However, it'd be nice to see the compatibility section explore exactly
> > what
> > > would render a plugin compiled against the newer Connect API
> incompatible
> > > with older versions of the Connect runtime. If a plugin explicitly
> > > implements ConfigSpecifier, would that break compatibility? If a plugin
> > > doesn't explicitly implement ConfigSpecifier, would that break
> > > compatibility? These are the two cases that come to mind immediately
> but
> > if
> > > any others occur to you feel free to document them as well. This
> coverage
> > > can also help guide us in how to document the interface if/when we add
> it
> > > to make it clear to plugin developers how to avoid rendering their work
> > > accidentally incompatibile with older versions of the Connect runtime.
> > >
> > > 3. There's also the pluggable ConnectRestExtension
> > > and ConnectorClientConfigOverridePolicy interfaces. Neither currently
> > > exposes a config method but it wouldn't be out of the question to add
> one
> > > now to them with a default implementation that returns an empty
> > ConfigDef.
> > > Thoughts?
> > >
> > > Cheers,
> > >
> > > Chris
> > >
> > > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via dev <
> > > dev@kafka.apache.org> wrote:
> > >
> > > > Hi Everyone,
> > > >
> > > > I would like to start a discussion on KIP-1273: Improve Connect
> > > > configurable components discoverability [1].
> > > >
> > > > Summary:
> > > > The idea is to introduce a common ConfigSpecifier interface for Kafka
> > > > Connect components that expose configuration metadata. By unifying
> the
> > > > existing config() method across connectors, converters,
> > transformations,
> > > > and predicates under a single contract, it simplifies component
> > > discovery,
> > > > reduces code duplication and enables uniform tooling for
> configuration
> > > > introspection, validation, documentation, and UI generation. The
> change
> > > is
> > > > fully backward compatible and requires no modifications to existing
> > > > component implementations.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> > > >
> > > > Regards,
> > > > Mario
> > > >
> > >
> >
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Chris Egerton <fe...@gmail.com>.
Hi Mario,
Thanks for the updates, especially the comprehensive compatibility section!
The KIP looks good to me, though you may want to double-check the example
Javadocs on ConnectRestExtension::config
and ConnectorClientConfigOverridePolicy::config :)
Cheers,
Chris
On Thu, Jan 22, 2026 at 5:54 AM Mario Fiore Vitale via dev <
dev@kafka.apache.org> wrote:
> Hi Chris, thanks for your valuable feedback!
>
> > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> strange--is
> it meant to refer to "this set of header converters" or should it just be
> "this header converter"?
>
> This was just taken from the current source. But I agree that it should be
> "this header converter".
>
> > For example, there's the obvious case
> where a plugin instantiates an object with a left-hand-type of
> ConfigSpecifier in its constructor. That plugin would be incompatible with
> older versions of Connect.
>
> Do you have an example of this? As you said, it's unlikely that this would
> occur in practice, also to me.
>
> > If a plugin explicitly
> implements ConfigSpecifier, would that break compatibility?
>
> In this case, it should.
>
> > If a plugin
> doesn't explicitly implement ConfigSpecifier, would that break
> compatibility?
>
> This should not break it.
>
> > These are the two cases that come to mind immediately but if
> any others occur to you feel free to document them as well.
>
> No other cases come to mind. I'll update the compatibility section with the
> forward compatibility consideration.
>
> > There's also the pluggable ConnectRestExtension
> and ConnectorClientConfigOverridePolicy interfaces. Neither currently
> exposes a config method but it wouldn't be out of the question to add one
> now to them with a default implementation that returns an empty ConfigDef.
>
> Well, I would say that since they are `Configurable`, I could expect that
> maybe in some future there could also be the possibility to declare a
> specific configuration.
> So your proposal to add it and implement it as a default seems good to me.
>
> Regards,
> Mario.
>
>
> Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
> fearthecellos@gmail.com> ha scritto:
>
> > Hi Mario,
> >
> > Thanks for the KIP! My thoughts:
> >
> > 1. (Minor) The Javadoc for HeaderConverter::config is a little
> strange--is
> > it meant to refer to "this set of header converters" or should it just be
> > "this header converter"?
> >
> > 2. The compatibility problem with Connect is always gnarly. The section
> on
> > backward compatibility is very well done in that it demonstrates with
> > certainty that older plugins will be able to run smoothly on newer
> versions
> > of the Connect runtime. However, there's also the question of whether
> newer
> > plugins (compiled against a future version of the Connect API with the
> > change proposed in this KIP) will continue to be compatible with older
> > versions of the Connect runtime. For example, there's the obvious case
> > where a plugin instantiates an object with a left-hand-type of
> > ConfigSpecifier in its constructor. That plugin would be incompatible
> with
> > older versions of Connect. It's unlikely that this would occur in
> practice
> > and even if it does, I think it's fine to accept that as a tradeoff.
> > However, it'd be nice to see the compatibility section explore exactly
> what
> > would render a plugin compiled against the newer Connect API incompatible
> > with older versions of the Connect runtime. If a plugin explicitly
> > implements ConfigSpecifier, would that break compatibility? If a plugin
> > doesn't explicitly implement ConfigSpecifier, would that break
> > compatibility? These are the two cases that come to mind immediately but
> if
> > any others occur to you feel free to document them as well. This coverage
> > can also help guide us in how to document the interface if/when we add it
> > to make it clear to plugin developers how to avoid rendering their work
> > accidentally incompatibile with older versions of the Connect runtime.
> >
> > 3. There's also the pluggable ConnectRestExtension
> > and ConnectorClientConfigOverridePolicy interfaces. Neither currently
> > exposes a config method but it wouldn't be out of the question to add one
> > now to them with a default implementation that returns an empty
> ConfigDef.
> > Thoughts?
> >
> > Cheers,
> >
> > Chris
> >
> > On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via dev <
> > dev@kafka.apache.org> wrote:
> >
> > > Hi Everyone,
> > >
> > > I would like to start a discussion on KIP-1273: Improve Connect
> > > configurable components discoverability [1].
> > >
> > > Summary:
> > > The idea is to introduce a common ConfigSpecifier interface for Kafka
> > > Connect components that expose configuration metadata. By unifying the
> > > existing config() method across connectors, converters,
> transformations,
> > > and predicates under a single contract, it simplifies component
> > discovery,
> > > reduces code duplication and enables uniform tooling for configuration
> > > introspection, validation, documentation, and UI generation. The change
> > is
> > > fully backward compatible and requires no modifications to existing
> > > component implementations.
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> > >
> > > Regards,
> > > Mario
> > >
> >
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Mario Fiore Vitale via dev <de...@kafka.apache.org>.
Hi Chris, thanks for your valuable feedback!
> 1. (Minor) The Javadoc for HeaderConverter::config is a little strange--is
it meant to refer to "this set of header converters" or should it just be
"this header converter"?
This was just taken from the current source. But I agree that it should be
"this header converter".
> For example, there's the obvious case
where a plugin instantiates an object with a left-hand-type of
ConfigSpecifier in its constructor. That plugin would be incompatible with
older versions of Connect.
Do you have an example of this? As you said, it's unlikely that this would
occur in practice, also to me.
> If a plugin explicitly
implements ConfigSpecifier, would that break compatibility?
In this case, it should.
> If a plugin
doesn't explicitly implement ConfigSpecifier, would that break
compatibility?
This should not break it.
> These are the two cases that come to mind immediately but if
any others occur to you feel free to document them as well.
No other cases come to mind. I'll update the compatibility section with the
forward compatibility consideration.
> There's also the pluggable ConnectRestExtension
and ConnectorClientConfigOverridePolicy interfaces. Neither currently
exposes a config method but it wouldn't be out of the question to add one
now to them with a default implementation that returns an empty ConfigDef.
Well, I would say that since they are `Configurable`, I could expect that
maybe in some future there could also be the possibility to declare a
specific configuration.
So your proposal to add it and implement it as a default seems good to me.
Regards,
Mario.
Il giorno gio 22 gen 2026 alle ore 04:33 Chris Egerton <
fearthecellos@gmail.com> ha scritto:
> Hi Mario,
>
> Thanks for the KIP! My thoughts:
>
> 1. (Minor) The Javadoc for HeaderConverter::config is a little strange--is
> it meant to refer to "this set of header converters" or should it just be
> "this header converter"?
>
> 2. The compatibility problem with Connect is always gnarly. The section on
> backward compatibility is very well done in that it demonstrates with
> certainty that older plugins will be able to run smoothly on newer versions
> of the Connect runtime. However, there's also the question of whether newer
> plugins (compiled against a future version of the Connect API with the
> change proposed in this KIP) will continue to be compatible with older
> versions of the Connect runtime. For example, there's the obvious case
> where a plugin instantiates an object with a left-hand-type of
> ConfigSpecifier in its constructor. That plugin would be incompatible with
> older versions of Connect. It's unlikely that this would occur in practice
> and even if it does, I think it's fine to accept that as a tradeoff.
> However, it'd be nice to see the compatibility section explore exactly what
> would render a plugin compiled against the newer Connect API incompatible
> with older versions of the Connect runtime. If a plugin explicitly
> implements ConfigSpecifier, would that break compatibility? If a plugin
> doesn't explicitly implement ConfigSpecifier, would that break
> compatibility? These are the two cases that come to mind immediately but if
> any others occur to you feel free to document them as well. This coverage
> can also help guide us in how to document the interface if/when we add it
> to make it clear to plugin developers how to avoid rendering their work
> accidentally incompatibile with older versions of the Connect runtime.
>
> 3. There's also the pluggable ConnectRestExtension
> and ConnectorClientConfigOverridePolicy interfaces. Neither currently
> exposes a config method but it wouldn't be out of the question to add one
> now to them with a default implementation that returns an empty ConfigDef.
> Thoughts?
>
> Cheers,
>
> Chris
>
> On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via dev <
> dev@kafka.apache.org> wrote:
>
> > Hi Everyone,
> >
> > I would like to start a discussion on KIP-1273: Improve Connect
> > configurable components discoverability [1].
> >
> > Summary:
> > The idea is to introduce a common ConfigSpecifier interface for Kafka
> > Connect components that expose configuration metadata. By unifying the
> > existing config() method across connectors, converters, transformations,
> > and predicates under a single contract, it simplifies component
> discovery,
> > reduces code duplication and enables uniform tooling for configuration
> > introspection, validation, documentation, and UI generation. The change
> is
> > fully backward compatible and requires no modifications to existing
> > component implementations.
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
> >
> > Regards,
> > Mario
> >
>
Re: [DISCUSS] KIP-1273 Improve Connect configurable components discoverability
Posted by Chris Egerton <fe...@gmail.com>.
Hi Mario,
Thanks for the KIP! My thoughts:
1. (Minor) The Javadoc for HeaderConverter::config is a little strange--is
it meant to refer to "this set of header converters" or should it just be
"this header converter"?
2. The compatibility problem with Connect is always gnarly. The section on
backward compatibility is very well done in that it demonstrates with
certainty that older plugins will be able to run smoothly on newer versions
of the Connect runtime. However, there's also the question of whether newer
plugins (compiled against a future version of the Connect API with the
change proposed in this KIP) will continue to be compatible with older
versions of the Connect runtime. For example, there's the obvious case
where a plugin instantiates an object with a left-hand-type of
ConfigSpecifier in its constructor. That plugin would be incompatible with
older versions of Connect. It's unlikely that this would occur in practice
and even if it does, I think it's fine to accept that as a tradeoff.
However, it'd be nice to see the compatibility section explore exactly what
would render a plugin compiled against the newer Connect API incompatible
with older versions of the Connect runtime. If a plugin explicitly
implements ConfigSpecifier, would that break compatibility? If a plugin
doesn't explicitly implement ConfigSpecifier, would that break
compatibility? These are the two cases that come to mind immediately but if
any others occur to you feel free to document them as well. This coverage
can also help guide us in how to document the interface if/when we add it
to make it clear to plugin developers how to avoid rendering their work
accidentally incompatibile with older versions of the Connect runtime.
3. There's also the pluggable ConnectRestExtension
and ConnectorClientConfigOverridePolicy interfaces. Neither currently
exposes a config method but it wouldn't be out of the question to add one
now to them with a default implementation that returns an empty ConfigDef.
Thoughts?
Cheers,
Chris
On Wed, Jan 21, 2026 at 10:33 AM Mario Fiore Vitale via dev <
dev@kafka.apache.org> wrote:
> Hi Everyone,
>
> I would like to start a discussion on KIP-1273: Improve Connect
> configurable components discoverability [1].
>
> Summary:
> The idea is to introduce a common ConfigSpecifier interface for Kafka
> Connect components that expose configuration metadata. By unifying the
> existing config() method across connectors, converters, transformations,
> and predicates under a single contract, it simplifies component discovery,
> reduces code duplication and enables uniform tooling for configuration
> introspection, validation, documentation, and UI generation. The change is
> fully backward compatible and requires no modifications to existing
> component implementations.
>
> [1]
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1273%3A+Improve+Connect+configurable+components+discoverability
>
> Regards,
> Mario
>