Connecting the new DataSource API (FLIP-27) with DynamicTableSource

[Yuval Itzchakov <yu...@gmail.com>]

Hi,

I've implemented a new custom source using the new DataSource API (
https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/stream/sources.html)
and I want to connect it to the new DynamicTableSource API.

However, in ScanTableSource, the getScanRuntimeProvider method returns a
ScanRuntimeProvider interface, which only has an implementation for the old
SourceFunction[T], and not for the new Source[T] class:

[image: image.png]

Does the Table API not yet support the new Source API? or am I missing
something?

-- 
Best Regards,
Yuval Itzchakov.

Re: Connecting the new DataSource API (FLIP-27) with DynamicTableSource

[Jingsong Li <ji...@gmail.com>]

Hi Yuval,

Yes, The new table source does not support the new Source API in Flink
1.11. The integration is introduced in Flink master (1.12):

https://cwiki.apache.org/confluence/display/FLINK/FLIP-146%3A+Improve+new+TableSource+and+TableSink+interfaces

Best,
Jingsong

On Sun, Nov 1, 2020 at 10:54 PM Yuval Itzchakov <yu...@gmail.com> wrote:

> Hi,
>
> I've implemented a new custom source using the new DataSource API (
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/stream/sources.html)
> and I want to connect it to the new DynamicTableSource API.
>
> However, in ScanTableSource, the getScanRuntimeProvider method returns a
> ScanRuntimeProvider interface, which only has an implementation for the old
> SourceFunction[T], and not for the new Source[T] class:
>
> [image: image.png]
>
> Does the Table API not yet support the new Source API? or am I missing
> something?
>
> --
> Best Regards,
> Yuval Itzchakov.
>


-- 
Best, Jingsong Lee