multiple pipeline deployment using flink k8s operator

[Sigalit Eliazov <e....@gmail.com>]

Hi all,
we just started using the flink k8s operator to deploy our flink cluster.
From what we understand we are only able to start a flink cluster per job.
So in our case when we have 2 jobs we have to create 2 different clusters.
obviously we would prefer to deploy these 2 job which relate to the same
use case in the same cluster with 1 job manager and 2 task managers (one
per job)

Is this possible via the operator?
Did we miss something understanding the configuration?

thanks
Sigalit

Re: multiple pipeline deployment using flink k8s operator

[Yang Wang <da...@gmail.com>]

The current application mode has the limitation that only one job could be
submitted when HA enabled[1].
So a feasible solution is to use the session mode[2], it will be supported
in the coming release-1.0.0.

However, I am afraid it still could not satisfy your requirement "2 task
managers (one per job)". Unless each TaskManager only has one slot.


[1].
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/overview/#application-mode
[2].
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resource/overview/#flinksessionjob


Best,
Yang

Sigalit Eliazov <e....@gmail.com> 于2022年6月1日周三 14:54写道：

> Hi all,
> we just started using the flink k8s operator to deploy our flink cluster.
> From what we understand we are only able to start a flink cluster per job.
> So in our case when we have 2 jobs we have to create 2 different clusters.
> obviously we would prefer to deploy these 2 job which relate to the same
> use case in the same cluster with 1 job manager and 2 task managers (one
> per job)
>
> Is this possible via the operator?
> Did we miss something understanding the configuration?
>
> thanks
> Sigalit
>