You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Almog Rozencwajg <Al...@niceactimize.com> on 2022/03/27 06:49:42 UTC
Deploy a Flink session cluster natively on K8s with multi AZ
Hi,
From the documentation, deploy a Flink session cluster natively on K8S is by running a shell script.
https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/resource-providers/native_kubernetes/?web=1&wdLOR=cFF7B7F0D-8BEC-4BAA-B06D-3D80A62B24AE#starting-a-flink-session-on-kubernetes
In the example, we can pass args in the command, is there a way to do it by using the flink-conf.yaml?
Is there another way to deploy a session cluster natively on K8S not only by running a command?
We want to support multi availability zone. If we are deploying the cluster in a standalone mode, we can configure the deployment of the job manager and task manager and using K8s pod topology spread constraints to achieve it.
If we are working with native K8s mode, is there a way to do it?
Thanks,
Almog
Confidentiality: This communication and any attachments are intended for the above-named persons only and may be confidential and/or legally privileged. Any opinions expressed in this communication are not necessarily those of NICE Actimize. If this communication has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender by e-mail immediately.
Monitoring: NICE Actimize may monitor incoming and outgoing e-mails.
Viruses: Although we have taken steps toward ensuring that this e-mail and attachments are free from any virus, we advise that in keeping with good computing practice the recipient should ensure they are actually virus free.
Re: Deploy a Flink session cluster natively on K8s with multi AZ
Posted by Yang Wang <da...@gmail.com>.
> In the example, we can pass args in the command, is there a way to do it
by using the flink-conf.yaml?
Yes. All the changes in the $FLINK_HOME/conf/flink-conf.yaml at the client
side will also be picked up when deploying a native K8s cluster.
For your use case, I am also suggesting the flink-kubernetes-operator. It
is a more k8s-native way.
Best,
Yang
Gyula Fóra <gy...@gmail.com> 于2022年3月27日周日 15:13写道：
> Hi!
>
> I think the Flink Kubernetes Operator (
> https://github.com/apache/flink-kubernetes-operator) project is exactly
> what you are looking for.
>
> This is a relatively new addition to Flink that supports k8s application
> and session deployments with lifecycle management through kubernetes native
> tooling.
>
> Cheers,
> Gyula
>
> On Sun, 27 Mar 2022 at 08:50, Almog Rozencwajg <
> Almog.Rozencwajg@niceactimize.com> wrote:
>
>> Hi,
>>
>>
>>
>> From the documentation, deploy a Flink session cluster natively on K8S is
>> by running a shell script.
>>
>>
>> https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/resource-providers/native_kubernetes/?web=1&wdLOR=cFF7B7F0D-8BEC-4BAA-B06D-3D80A62B24AE#starting-a-flink-session-on-kubernetes
>>
>> In the example, we can pass args in the command, is there a way to do it
>> by using the flink-conf.yaml?
>> Is there another way to deploy a session cluster natively on K8S not only
>> by running a command?
>>
>>
>> We want to support multi availability zone. If we are deploying the
>> cluster in a standalone mode, we can configure the deployment of the job
>> manager and task manager and using K8s pod topology spread constraints to
>> achieve it.
>> If we are working with native K8s mode, is there a way to do it?
>>
>>
>>
>> Thanks,
>>
>> Almog
>>
>>
>>
>>
>> Confidentiality: This communication and any attachments are intended for
>> the above-named persons only and may be confidential and/or legally
>> privileged. Any opinions expressed in this communication are not
>> necessarily those of NICE Actimize. If this communication has come to you
>> in error you must take no action based on it, nor must you copy or show it
>> to anyone; please delete/destroy and inform the sender by e-mail
>> immediately.
>> Monitoring: NICE Actimize may monitor incoming and outgoing e-mails.
>> Viruses: Although we have taken steps toward ensuring that this e-mail
>> and attachments are free from any virus, we advise that in keeping with
>> good computing practice the recipient should ensure they are actually virus
>> free.
>>
>
Re: Deploy a Flink session cluster natively on K8s with multi AZ
Posted by Gyula Fóra <gy...@gmail.com>.
Hi!
I think the Flink Kubernetes Operator (
https://github.com/apache/flink-kubernetes-operator) project is exactly
what you are looking for.
This is a relatively new addition to Flink that supports k8s application
and session deployments with lifecycle management through kubernetes native
tooling.
Cheers,
Gyula
On Sun, 27 Mar 2022 at 08:50, Almog Rozencwajg <
Almog.Rozencwajg@niceactimize.com> wrote:
> Hi,
>
>
>
> From the documentation, deploy a Flink session cluster natively on K8S is
> by running a shell script.
>
>
> https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/resource-providers/native_kubernetes/?web=1&wdLOR=cFF7B7F0D-8BEC-4BAA-B06D-3D80A62B24AE#starting-a-flink-session-on-kubernetes
>
> In the example, we can pass args in the command, is there a way to do it
> by using the flink-conf.yaml?
> Is there another way to deploy a session cluster natively on K8S not only
> by running a command?
>
>
> We want to support multi availability zone. If we are deploying the
> cluster in a standalone mode, we can configure the deployment of the job
> manager and task manager and using K8s pod topology spread constraints to
> achieve it.
> If we are working with native K8s mode, is there a way to do it?
>
>
>
> Thanks,
>
> Almog
>
>
>
>
> Confidentiality: This communication and any attachments are intended for
> the above-named persons only and may be confidential and/or legally
> privileged. Any opinions expressed in this communication are not
> necessarily those of NICE Actimize. If this communication has come to you
> in error you must take no action based on it, nor must you copy or show it
> to anyone; please delete/destroy and inform the sender by e-mail
> immediately.
> Monitoring: NICE Actimize may monitor incoming and outgoing e-mails.
> Viruses: Although we have taken steps toward ensuring that this e-mail and
> attachments are free from any virus, we advise that in keeping with good
> computing practice the recipient should ensure they are actually virus free.
>