[DISCUSS] KIP-683: Add recursive support to Connect Cast and ReplaceField transforms, and support for casting complex types to either a native or JSON string

[Joshua Grisham <gr...@gmail.com>]

Hello again all!

Today I have typed up the KIP for this change which I am proposing as well
for two of the Connect transforms, Cast and ReplaceField.

Please see here for more information:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-683%3A+Add+recursive+support+to+Connect+Cast+and+ReplaceField+transforms%2C+and+support+for+casting+complex+types+to+either+a+native+or+JSON+string

There is already a PR available as well to review for specifics on how I
have proposed to implement these improvements:

https://github.com/apache/kafka/pull/9493

This does change a bit of the structure for both transforms and it would be
good to have a little bit of input if anyone thinks this is a bad idea or
sees some problem that I have not discovered already myself.

Any questions or feedback, please feel free to share!

Best regards,
Joshua Grisham

Re: [DISCUSS] KIP-683: Add recursive support to Connect Cast and ReplaceField transforms, and support for casting complex types to either a native or JSON string

[Joshua Grisham <gr...@gmail.com>]

It's me again, dusting another one off. 😁
This time, I thought I would bring up KIP-683 again.

In the end I guess this change is more about a design decision.  The
current versions of Connect Cast and ReplaceField are elegantly simple and
straight-forward and maybe serve as good examples of what kinds of things
can be done with a SMT.  But they are *so close* to being a bit more
practical for slightly more complex "real-world scenarios" (which is what I
tried to add some of here with KIP-683). So then the question becomes:
should a little more complexity be added to these standard transforms so
that they can be used in more complex scenarios, or should they continue to
serve as simple transforms and examples that can custom transforms can be
built from?

I noticed there were a few conflicts now since it has been > 6 months since
I originally wrote the proposed changes. But it mostly seemed like there is
an added feature in Cast to convert from binary to base64 string, which
should be relatively simple to merge into my code if this is a direction
that people would agree to go for?

Any thoughts?

Best regards,
Joshua


On Sun, Nov 8, 2020 at 4:33 PM Joshua Grisham <gr...@gmail.com> wrote:

> Hello again all!
>
> Today I have typed up the KIP for this change which I am proposing as well
> for two of the Connect transforms, Cast and ReplaceField.
>
> Please see here for more information:
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-683%3A+Add+recursive+support+to+Connect+Cast+and+ReplaceField+transforms%2C+and+support+for+casting+complex+types+to+either+a+native+or+JSON+string
>
> There is already a PR available as well to review for specifics on how I
> have proposed to implement these improvements:
>
> https://github.com/apache/kafka/pull/9493
>
> This does change a bit of the structure for both transforms and it would
> be good to have a little bit of input if anyone thinks this is a bad idea
> or sees some problem that I have not discovered already myself.
>
> Any questions or feedback, please feel free to share!
>
> Best regards,
> Joshua Grisham
>
>
>