You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Dhiru <us...@yahoo.com> on 2021/09/30 00:41:01 UTC

Flink application mode with no ui , how to start job using k8s ?

Hi ,
   My requirement is to create Flink cluster application Mode on k8s and do not want to expose UI, my requirement is to start the long-running  job which can be instantiated at boot time of flink and keep running
use these resource files from jobmanager-application-ha.yaml and taskmanager-job-deployment.yaml for creating cluster (https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/#application-cluster-resource-definitions)

a)  I need to start job during run time , I can bundle my jar with Flink image, so that can  instantiate jar 
b) Can I apply HPA (horizontal pod autoscalar) for task manager, will this work so that according to workload instance of  taskmanager goes up and down.
--kumar

Re: Flink application mode with no ui , how to start job using k8s ?

Posted by Dhiru <us...@yahoo.com>.

 Thanks Dawid,       if I am not exposing UI , how I am going to run my job we need to submit jar someway, I do not want my flink image tightly coupled with my jar 
    On Monday, October 4, 2021, 09:52:31 AM EDT, Dawid Wysakowicz <dw...@apache.org> wrote:  
 
  
Hi Dhiru,
 
For the question about auto scaling I'd recommend you this[1] blogpost from my colleague. I believe he explains it quite well how to do it.
 
Besides that I am not sure what is your other question. Are you asking how to start the jobmanager without the UI? Can't you just simply not expose the port?
 
Best,
 
Dawid
 
[1] https://flink.apache.org/2021/05/06/reactive-mode.html
 
 On 30/09/2021 02:41, Dhiru wrote:
  
 
    Hi , 
     My requirement is to create Flink cluster application Mode on k8s and do not want to expose UI, my requirement is to start the long-running  job which can be instantiated at boot time of flink and keep running 
  use these resource files from jobmanager-application-ha.yaml and taskmanager-job-deployment.yaml for creating cluster (https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/#application-cluster-resource-definitions) 
  
  a)  I need to start job during run time , I can bundle my jar with Flink image, so that can  instantiate jar  
  b) Can I apply HPA (horizontal pod autoscalar) for task manager, will this work so that according to workload instance of  taskmanager goes up and down. 
  --kumar

Re: Flink application mode with no ui , how to start job using k8s ?

Posted by Dawid Wysakowicz <dw...@apache.org>.

Hi Dhiru,

For the question about auto scaling I'd recommend you this[1] blogpost
from my colleague. I believe he explains it quite well how to do it.

Besides that I am not sure what is your other question. Are you asking
how to start the jobmanager without the UI? Can't you just simply not
expose the port?

Best,

Dawid

[1] https://flink.apache.org/2021/05/06/reactive-mode.html

On 30/09/2021 02:41, Dhiru wrote:
> Hi ,
>
>    My requirement is to create Flink cluster application Mode on k8s
> and do not want to expose UI, my requirement is to start the
> long-running  job which can be instantiated at boot time of flink and
> keep running
>
> use these resource files from jobmanager-application-ha.yaml
> and taskmanager-job-deployment.yaml for creating cluster
> (https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/#application-cluster-resource-definitions
> <https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/#application-cluster-resource-definitions>)
>
>
> a)  I need to start job during run time , I can bundle my jar with
> Flink image, so that can  instantiate jar 
>
> b) Can I apply HPA (horizontal pod autoscalar) for task manager, will
> this work so that according to workload instance of  taskmanager goes
> up and down.
>
> --kumar
>
>
>