You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by Michael Negodaev <ne...@gmail.com> on 2022/07/22 07:57:23 UTC

[DISCUSS] KIP-855: Add schema.namespace parameter to SetSchemaMetadata SMT in Kafka Connect

Hi all,

I would like to start the discussion on my design to add "schema.namespace"
parameter in SetSchemaMetadata Single Message Transform in Kafka Connect.

KIP URL: https://cwiki.apache.org/confluence/x/CiT1D

Thanks!
-Michael

Re: [DISCUSS] KIP-855: Add schema.namespace parameter to SetSchemaMetadata SMT in Kafka Connect

Posted by Chris Egerton <ch...@aiven.io.INVALID>.

Hi Michael,

Thanks for the KIP! I was initially a little skeptical about the usefulness
of this feature since it seemed designed exclusively with Avro in mind, but
the motivation section convinced me that this would be applicable to a
wider range of use cases, so I'm happy to move forward with it.

Here are my initial thoughts:

1. The concept of a "namespace" is never explicitly defined anywhere in the
KIP. Can you add a concrete definition for that term? And, related to
this--are we assuming that the delimiter between a name and a namespace is
always the '.' character?
2. Why should we allow users to specify both a namespace and a name with
the SetSchemaMetadata transform? Isn't it possible to get the same behavior
from that kind of configuration by just setting a name that's prefixed with
the desired namespace?
3. The KIP currently addresses the use case of modifying a schema's
namespace, but leaving its name unchanged. Do you think there's any value
to adding support for the reverse--modifying a schema's name, but leaving
its namespace unchanged?
4. Can you add some examples to the KIP of what a schema's name would look
like before and after passing through the SetSchemaMetadata transform with
the following combinations of configuration:
   - Namespace provided, no name provided
   - Name provided, no namespace provided
   - Namespace and name provided (if we decide not to support this case, we
can call out that this results in a configuration error)

Cheers,

Chris

On Tue, Aug 23, 2022 at 1:37 AM Michael Negodaev <ne...@gmail.com> wrote:

> Hi Mickael,
>
> Thank you for looking into this.
> This is definitely a typo, I've just corrected it. Thanks for finding this!
>
> Michael
>
> ср, 17 авг. 2022 г. в 15:28, Mickael Maison <mi...@gmail.com>:
>
> > Hi Michael,
> >
> > Thanks for the KIP! Sorry for the delay, I finally took some time to
> > take a look.
> >
> > In both the "Public Interfaces" and "Compatibility, Deprecation, and
> > Migration Plan" sections it mentions the new config is
> > "transforms.transformschema.schema.name". However if I understand
> > correctly the config you propose adding is actually
> > "transforms.transformschema.schema.namespace". Is this a typo or am I
> > missing something?
> >
> > Thanks,
> > Mickael
> >
> > On Fri, Jul 22, 2022 at 9:57 AM Michael Negodaev <ne...@gmail.com>
> > wrote:
> > >
> > > Hi all,
> > >
> > > I would like to start the discussion on my design to add
> > "schema.namespace"
> > > parameter in SetSchemaMetadata Single Message Transform in Kafka
> Connect.
> > >
> > > KIP URL: https://cwiki.apache.org/confluence/x/CiT1D
> > >
> > > Thanks!
> > > -Michael
> >
>

Re: [DISCUSS] KIP-855: Add schema.namespace parameter to SetSchemaMetadata SMT in Kafka Connect

Posted by Michael Negodaev <ne...@gmail.com>.

Hi Mickael,

Thank you for looking into this.
This is definitely a typo, I've just corrected it. Thanks for finding this!

Michael

ср, 17 авг. 2022 г. в 15:28, Mickael Maison <mi...@gmail.com>:

> Hi Michael,
>
> Thanks for the KIP! Sorry for the delay, I finally took some time to
> take a look.
>
> In both the "Public Interfaces" and "Compatibility, Deprecation, and
> Migration Plan" sections it mentions the new config is
> "transforms.transformschema.schema.name". However if I understand
> correctly the config you propose adding is actually
> "transforms.transformschema.schema.namespace". Is this a typo or am I
> missing something?
>
> Thanks,
> Mickael
>
> On Fri, Jul 22, 2022 at 9:57 AM Michael Negodaev <ne...@gmail.com>
> wrote:
> >
> > Hi all,
> >
> > I would like to start the discussion on my design to add
> "schema.namespace"
> > parameter in SetSchemaMetadata Single Message Transform in Kafka Connect.
> >
> > KIP URL: https://cwiki.apache.org/confluence/x/CiT1D
> >
> > Thanks!
> > -Michael
>

Re: [DISCUSS] KIP-855: Add schema.namespace parameter to SetSchemaMetadata SMT in Kafka Connect

Posted by Mickael Maison <mi...@gmail.com>.

Hi Michael,

Thanks for the KIP! Sorry for the delay, I finally took some time to
take a look.

In both the "Public Interfaces" and "Compatibility, Deprecation, and
Migration Plan" sections it mentions the new config is
"transforms.transformschema.schema.name". However if I understand
correctly the config you propose adding is actually
"transforms.transformschema.schema.namespace". Is this a typo or am I
missing something?

Thanks,
Mickael

On Fri, Jul 22, 2022 at 9:57 AM Michael Negodaev <ne...@gmail.com> wrote:
>
> Hi all,
>
> I would like to start the discussion on my design to add "schema.namespace"
> parameter in SetSchemaMetadata Single Message Transform in Kafka Connect.
>
> KIP URL: https://cwiki.apache.org/confluence/x/CiT1D
>
> Thanks!
> -Michael