You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@paimon.apache.org by umesh dangat <um...@gmail.com> on 2024/08/06 23:07:01 UTC
Support debezium avro format in cdc action + complex avro type support
Hello,
Checking in plans to merge this PR
https://github.com/apache/paimon/pull/3323?
We at Yelp are using this patch to enable us to consume data into paimon
from our kafka sources that are debezium avro encoded and use a confluent
schema registry.
Also I added another patch on top of this branch to get complex avro types
working, in particular support for Map, Array and Nested Record types.
https://github.com/zhuangchong/flink-table-store/pull/1
I would appreciate it if this is looked at as well.
Re: Support debezium avro format in cdc action + complex avro type support
Posted by Kerwin <zh...@gmail.com>.
Thank you very much for using and contributing, and I'll be actively
pushing to merge this PR as well!
--
Best Wish
— Kerwin (zhuangchong)
Jingsong Li <ji...@gmail.com> 于2024年8月7日周三 11:39写道：
> Very thanks for your contribution!
>
> Without a doubt, we should take a step forward on Avro support.
>
> We will take a review this week.
>
> https://github.com/zhuangchong/flink-table-store/pull/1
>
> Can you just create a PR to paimon repo? I will rebase master for your
> PR if pull-3323 has been merged.
>
> Best,
> Jingsong
>
> On Wed, Aug 7, 2024 at 7:08 AM umesh dangat <um...@gmail.com> wrote:
> >
> > Hello,
> >
> > Checking in plans to merge this PR
> > https://github.com/apache/paimon/pull/3323?
> >
> > We at Yelp are using this patch to enable us to consume data into paimon
> > from our kafka sources that are debezium avro encoded and use a confluent
> > schema registry.
> >
> > Also I added another patch on top of this branch to get complex avro
> types
> > working, in particular support for Map, Array and Nested Record types.
> > https://github.com/zhuangchong/flink-table-store/pull/1
> >
> > I would appreciate it if this is looked at as well.
>
Re: Support debezium avro format in cdc action + complex avro type support
Posted by Jingsong Li <ji...@gmail.com>.
Very thanks for your contribution!
Without a doubt, we should take a step forward on Avro support.
We will take a review this week.
https://github.com/zhuangchong/flink-table-store/pull/1
Can you just create a PR to paimon repo? I will rebase master for your
PR if pull-3323 has been merged.
Best,
Jingsong
On Wed, Aug 7, 2024 at 7:08 AM umesh dangat <um...@gmail.com> wrote:
>
> Hello,
>
> Checking in plans to merge this PR
> https://github.com/apache/paimon/pull/3323?
>
> We at Yelp are using this patch to enable us to consume data into paimon
> from our kafka sources that are debezium avro encoded and use a confluent
> schema registry.
>
> Also I added another patch on top of this branch to get complex avro types
> working, in particular support for Map, Array and Nested Record types.
> https://github.com/zhuangchong/flink-table-store/pull/1
>
> I would appreciate it if this is looked at as well.