Advantages of running Flink on YuniKorn?

[Bowen Li <bo...@gmail.com>]

Hi folks,

Wondering what're the advantages of running Flink (especially Flink
streaming apps) on YuniKorn v.s. default or other scheduler of K8S?

Asking since 1) I didn't find much about it on
https://yunikorn.apache.org/docs/user_guide/workloads/run_flink  2) we
quite well understand the advantages of running batch jobs (e.g. Spark,
etc) on YK, but not so much on streaming jobs.

Would be great if someone can shed lights on it.

Thanks,
Bowen

Re: Advantages of running Flink on YuniKorn?

[Weiwei Yang <ww...@apache.org>]

Hi Bowen

You might want to take a look at this session:
https://www.youtube.com/watch?v=4hghJCuZk5M. This is given by Alibaba Flink
team, and explained why they are leveraging YuniKorn for their use cases.
They were some features they are extremely interested: hierarchy queues,
resource fairness, performance, job level priority, job preemption. Note,
not all these features are ready in upstream today.

Thanks

On Sun, Oct 17, 2021 at 11:06 PM Bowen Li <bo...@gmail.com> wrote:

> Hi folks,
>
> Wondering what're the advantages of running Flink (especially Flink
> streaming apps) on YuniKorn v.s. default or other scheduler of K8S?
>
> Asking since 1) I didn't find much about it on
> https://yunikorn.apache.org/docs/user_guide/workloads/run_flink  2) we
> quite well understand the advantages of running batch jobs (e.g. Spark,
> etc) on YK, but not so much on streaming jobs.
>
> Would be great if someone can shed lights on it.
>
> Thanks,
> Bowen
>