Re: Roadmap for Execution Mode (Batch/Streaming) and interaction with Table/SQL APIs

[Aljoscha Krettek <al...@apache.org>]

Also cc'ing dev@flink.apache.org

On 2021/01/06 09:19, burkaygur wrote:
>1) How do these changes impact the Table and SQL APIs? Are they completely
>orthogonal or can we get the benefits of the new Batch Mode with Flink SQL
>as well?

The answer here is a bit complicated. The Table API/SQL already use 
similar techniques under the hood for BATCH execution. Our recent 
changes just made similar functionality available for the DataStream 
API. So in a way the changes are orthogonal.

However, the changes are relevant when you want to interoperate with the 
DataStream and Table API. There it becomes relevant that both parts can 
do STREAMING/BATCH execution well. We're not 100% there yet on this 
front but we're tracking some work under FLIP-136 [1].

[1] https://cwiki.apache.org/confluence/x/0DN4CQ

>2) What is the best ticket to follow the roadmap & track the progress of
>this whole project. Specifically the parts about bootstrapping of state. I
>would love to help contribute to it.

I would say the best way to follow progress is the dev mailing list and 
the the FLIP overview page [2]. That's not super intuitive and can be 
hard to follow for outsiders. Sometimes, people such as myself will 
write blog posts on the Flink website or private blogs that try and 
shine a light on the development so it might help to follow some 
relevant people from the project on Twitter, where such posts are often 
announced.

Specifically about state bootstrapping, we don't have many concrete 
thoughts yet. It would help if you could talk about some of the 
requirements that you would have for this.

Best,
Aljoscha

[2] 
https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals

Best,
Aljoscha