You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Qihua Yang <ya...@gmail.com> on 2021/09/28 21:56:46 UTC
Flink run different jars
Hi,
Is that possible to run a flink app without a job? What I am trying to do
is I build multiple jars. And switch jar to run different jobs.
I am not sure if flink supports this mode. I saw rest API can upload jar,
cancel job and run a jar.
Right now I can upload a jar to flink. But when I cancel a job, flink will
restart automatically. I checked log. It show below logs. Can anyone help
me out?
Caused by:
org.apache.flink.client.deployment.application.UnsuccessfulExecutionException:
Application Status: CANCELED
at
org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:73)
... 41 common frames omitted
Caused by: org.apache.flink.runtime.client.JobCancellationException: Job
was cancelled.
at
org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:149)
at
org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:64)
... 41 common frames omitted
Thanks!
Re: Flink run different jars
Posted by Qihua Yang <ya...@gmail.com>.
Thanks a lot Yangze. That is very helpful!
On Tue, Sep 28, 2021 at 11:11 PM Yangze Guo <ka...@gmail.com> wrote:
> You need to edit the conf/workers. Example of the config[1] and the
> process[2].
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/#starting-and-stopping-a-cluster
> [2]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/#starting-a-standalone-cluster-session-mode
>
> Best,
> Yangze Guo
>
> On Wed, Sep 29, 2021 at 1:02 PM Qihua Yang <ya...@gmail.com> wrote:
> >
> > Hi Yangze,
> >
> > Thanks a lot for your reply. References are very helpful!
> > Another quick question. Reference 1 can start a standalone cluster
> (session Mode). That cluster has a jobManager. I can submit job to run. How
> about taskManger? Do I need to manually start multiple taskManagers?
> > Is there a complete example to show the process?
> >
> > Thanks,
> > Qihua
> >
> >
> > On Tue, Sep 28, 2021 at 7:02 PM Yangze Guo <ka...@gmail.com> wrote:
> >>
> >> Hi, Qihua
> >>
> >> IIUC, what you want might be a standalone cluster[1] or session
> cluster[2][3].
> >>
> >> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/
> >> [2]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#session-mode
> >> [3]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#session-mode
> >>
> >> Best,
> >> Yangze Guo
> >>
> >> On Wed, Sep 29, 2021 at 5:57 AM Qihua Yang <ya...@gmail.com> wrote:
> >> >
> >> > Hi,
> >> >
> >> > Is that possible to run a flink app without a job? What I am trying
> to do is I build multiple jars. And switch jar to run different jobs.
> >> > I am not sure if flink supports this mode. I saw rest API can upload
> jar, cancel job and run a jar.
> >> > Right now I can upload a jar to flink. But when I cancel a job, flink
> will restart automatically. I checked log. It show below logs. Can anyone
> help me out?
> >> >
> >> > Caused by:
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException:
> Application Status: CANCELED
> >> > at
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:73)
> >> > ... 41 common frames omitted
> >> > Caused by: org.apache.flink.runtime.client.JobCancellationException:
> Job was cancelled.
> >> > at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:149)
> >> > at
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:64)
> >> > ... 41 common frames omitted
> >> >
> >> > Thanks!
>
Re: Flink run different jars
Posted by Yangze Guo <ka...@gmail.com>.
You need to edit the conf/workers. Example of the config[1] and the process[2].
[1] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/#starting-and-stopping-a-cluster
[2] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/#starting-a-standalone-cluster-session-mode
Best,
Yangze Guo
On Wed, Sep 29, 2021 at 1:02 PM Qihua Yang <ya...@gmail.com> wrote:
>
> Hi Yangze,
>
> Thanks a lot for your reply. References are very helpful!
> Another quick question. Reference 1 can start a standalone cluster (session Mode). That cluster has a jobManager. I can submit job to run. How about taskManger? Do I need to manually start multiple taskManagers?
> Is there a complete example to show the process?
>
> Thanks,
> Qihua
>
>
> On Tue, Sep 28, 2021 at 7:02 PM Yangze Guo <ka...@gmail.com> wrote:
>>
>> Hi, Qihua
>>
>> IIUC, what you want might be a standalone cluster[1] or session cluster[2][3].
>>
>> [1] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/
>> [2] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#session-mode
>> [3] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#session-mode
>>
>> Best,
>> Yangze Guo
>>
>> On Wed, Sep 29, 2021 at 5:57 AM Qihua Yang <ya...@gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > Is that possible to run a flink app without a job? What I am trying to do is I build multiple jars. And switch jar to run different jobs.
>> > I am not sure if flink supports this mode. I saw rest API can upload jar, cancel job and run a jar.
>> > Right now I can upload a jar to flink. But when I cancel a job, flink will restart automatically. I checked log. It show below logs. Can anyone help me out?
>> >
>> > Caused by: org.apache.flink.client.deployment.application.UnsuccessfulExecutionException: Application Status: CANCELED
>> > at org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:73)
>> > ... 41 common frames omitted
>> > Caused by: org.apache.flink.runtime.client.JobCancellationException: Job was cancelled.
>> > at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:149)
>> > at org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:64)
>> > ... 41 common frames omitted
>> >
>> > Thanks!
Re: Flink run different jars
Posted by Qihua Yang <ya...@gmail.com>.
Hi Yangze,
Thanks a lot for your reply. References are very helpful!
Another quick question. Reference 1 can start a standalone cluster (session
Mode). That cluster has a jobManager. I can submit job to run. How about
taskManger? Do I need to manually start multiple taskManagers?
Is there a complete example to show the process?
Thanks,
Qihua
On Tue, Sep 28, 2021 at 7:02 PM Yangze Guo <ka...@gmail.com> wrote:
> Hi, Qihua
>
> IIUC, what you want might be a standalone cluster[1] or session
> cluster[2][3].
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/
> [2]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#session-mode
> [3]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#session-mode
>
> Best,
> Yangze Guo
>
> On Wed, Sep 29, 2021 at 5:57 AM Qihua Yang <ya...@gmail.com> wrote:
> >
> > Hi,
> >
> > Is that possible to run a flink app without a job? What I am trying to
> do is I build multiple jars. And switch jar to run different jobs.
> > I am not sure if flink supports this mode. I saw rest API can upload
> jar, cancel job and run a jar.
> > Right now I can upload a jar to flink. But when I cancel a job, flink
> will restart automatically. I checked log. It show below logs. Can anyone
> help me out?
> >
> > Caused by:
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException:
> Application Status: CANCELED
> > at
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:73)
> > ... 41 common frames omitted
> > Caused by: org.apache.flink.runtime.client.JobCancellationException: Job
> was cancelled.
> > at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:149)
> > at
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:64)
> > ... 41 common frames omitted
> >
> > Thanks!
>
Re: Flink run different jars
Posted by Yangze Guo <ka...@gmail.com>.
Hi, Qihua
IIUC, what you want might be a standalone cluster[1] or session cluster[2][3].
[1] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/
[2] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#session-mode
[3] https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#session-mode
Best,
Yangze Guo
On Wed, Sep 29, 2021 at 5:57 AM Qihua Yang <ya...@gmail.com> wrote:
>
> Hi,
>
> Is that possible to run a flink app without a job? What I am trying to do is I build multiple jars. And switch jar to run different jobs.
> I am not sure if flink supports this mode. I saw rest API can upload jar, cancel job and run a jar.
> Right now I can upload a jar to flink. But when I cancel a job, flink will restart automatically. I checked log. It show below logs. Can anyone help me out?
>
> Caused by: org.apache.flink.client.deployment.application.UnsuccessfulExecutionException: Application Status: CANCELED
> at org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:73)
> ... 41 common frames omitted
> Caused by: org.apache.flink.runtime.client.JobCancellationException: Job was cancelled.
> at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:149)
> at org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:64)
> ... 41 common frames omitted
>
> Thanks!