Running Flink Cluster on PaaS

[KristoffSC <kr...@gmail.com>]

Hi all,
are there any obstacles from running Flink Cluster on PaaS like OpenShift
for example?
Where for example, task manager could be reasigned to different physical
box?
Especially when Flink will be in form of Docker Job Cluster.

Regards,
Krzysztof



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Running Flink Cluster on PaaS

[Yang Wang <da...@gmail.com>]

> I always wonder what do you guys mean by "Standalone Flink session" or
"Standalone Cluster" ...

"Standalone Flink session" usually means an empty Flink cluster is started
and
could accept multiple jobs submission from the Flink client or webui. Even
all
the jobs finished, the session cluster will still be there until you
manually stop it.

"Standalone Flink per-job" means a dedicated Flink cluster is started for
only
one the job. It has better isolation. Usually this mode is used in container
environment. And users build in their jars and dependencies in the image.
You could check how to build your custom image and run a per-job cluster
here[1].


[1].
https://github.com/apache/flink/blob/release-1.10/flink-container/kubernetes/README.md#deploy-flink-job-cluster


Best,
Yang

KristoffSC <kr...@gmail.com> 于2020年2月21日周五 下午4:03写道：

> Thank you Yang Wang,
>
> Regarding [1] and a sentence from that doc.
> "This page describes deploying a standalone Flink session"
>
> I always wonder what do you guys mean by "Standalone Flink session" or
> "Standalone Cluster" that can be found here [2].
>
> I'm using a Docker with Job Cluster approach, I know that there is also a
> Session Cluster docker images. I understand the differences, but I'm not
> sure what you are referring to using those to terms from [1] and [2].
>
> Thanks,
> Krzysztof
>
> [1]
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/kubernetes.html#cluster_setup.html
> [2]
>
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/jobmanager_high_availability.html
>
>
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>

Re: Running Flink Cluster on PaaS

[KristoffSC <kr...@gmail.com>]

Thank you Yang Wang,

Regarding [1] and a sentence from that doc.
"This page describes deploying a standalone Flink session"

I always wonder what do you guys mean by "Standalone Flink session" or
"Standalone Cluster" that can be found here [2].

I'm using a Docker with Job Cluster approach, I know that there is also a
Session Cluster docker images. I understand the differences, but I'm not
sure what you are referring to using those to terms from [1] and [2].

Thanks,
Krzysztof

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/kubernetes.html#cluster_setup.html
[2]
https://ci.apache.org/projects/flink/flink-docs-stable/ops/jobmanager_high_availability.html





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Running Flink Cluster on PaaS

[Yang Wang <da...@gmail.com>]

Hi KristoffSC,

Currently, Flink could run in containerized environment and i think many
users and companies
deploy their production workloads on Kubernetes. For OpenShift, i think it
is just like K8s.
You could deploy standalone Flink session/per-job cluster by applying some
yamls[1]. Also you
could have a taste on the native integration. The community has not tested
on a OpenShift
environment and you could share your feedback after use.

>>  task manager could be reasigned to different physical box?
It does not matter where the taskmanager is assigned. When the network from
taskmanager to
jobmanager is normal, then it will register to the jobmanager again.

[1].
https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/kubernetes.html
[2].
https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/native_kubernetes.html


KristoffSC <kr...@gmail.com> 于2020年2月21日周五 上午6:43写道：

> Hi all,
> are there any obstacles from running Flink Cluster on PaaS like OpenShift
> for example?
> Where for example, task manager could be reasigned to different physical
> box?
> Especially when Flink will be in form of Docker Job Cluster.
>
> Regards,
> Krzysztof
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>