You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "r. r." <ro...@abv.bg> on 2017/09/28 06:31:16 UTC
how many 'run -c' commands to start?
Hello
I successfully ran a job with 'flink run -c', but this is for the local
setup.
How should i proceed with a cluster? Will flink automagically instantiate
the job on all servers - i hope i don't have to start 'flink run -c' on all
machines.
New to flink and bigdata, so sorry for the probably silly question
Thanks!
Rob
Re: how many 'run -c' commands to start?
Posted by "r. r." <ro...@abv.bg>.
Thanks, Chesnay, that was indeed the problem.
It also explains why -p5 was not working for me from the cmdline
Best regards
Robert
>-------- Оригинално писмо --------
>От: Chesnay Schepler chesnay@apache.org
>Относно: Re: how many 'run -c' commands to start?
>До: "r. r." <ro...@abv.bg>
>Изпратено на: 01.10.2017 00:58
> The order in which you pass program arguments is quite important. Can
> you try again putting --detached directly after run?
> I.e. flink run --detached -c ...
>
> The reason being that arguments after the jar are treated as arguments
> for the user-program, and not as arguments for the job execution.
>
> On 29.09.2017 17:23, r. r. wrote:
> > Sure, here is the cmdline output:
> >
> > flink-1.3.2/bin/flink run -c com.corp.flink.KafkaJob quickstart.jar --topic InputQ --bootstrap.servers localhost:9092 --zookeeper.connect localhost:2181 --group.id Consumers -p 5 --detached
> > Cluster configuration: Standalone cluster with JobManager at localhost/127.0.0.1:6123
> > Using address localhost:6123 to connect to JobManager.
> > JobManager web interface address http://localhost:8081
> > Starting execution of program
> > Submitting job with JobID: 5cc74547361cec2ff9874764dac9ee91. Waiting for job completion.
> > Connected to JobManager at Actor[akka.tcp://flink@localhost:6123/user/jobmanager#-100905913] with leader session id 00000000-0000-0000-0000-000000000000.
> > 09/28/2017 16:30:40 Job execution switched to status RUNNING.
> > 09/28/2017 16:30:40 Source: Custom Source -> Map(1/5) switched to SCHEDULED
> > 09/28/2017 16:30:40 Source: Custom Source -> Map(2/5) switched to SCHEDULED
> > 09/28/2017 16:30:40 Source: Custom Source -> Map(3/5) switched to SCHEDULED
> > 09/28/2017 16:30:40 Source: Custom Source -> Map(4/5) switched to SCHEDULED
> > 09/28/2017 16:30:40 Source: Custom Source -> Map(5/5) switched to SCHEDULED
> > 09/28/2017 16:30:40 Sink: Unnamed(1/5) switched to SCHEDULED
> > 09/28/2017 16:30:40 Sink: Unnamed(2/5) switched to SCHEDULED
> > 09/28/2017 16:30:40 Sink: Unnamed(3/5) switched to SCHEDULED
> > 09/28/2017 16:30:40 Sink: Unnamed(4/5) switched to SCHEDULED
> > 09/28/2017 16:30:40 Sink: Unnamed(5/5) switched to SCHEDULED
> > ...
> >
> > I thought --detach will put the process in the background, and give me back the cmdline, but maybe I got the meaning behind this option wrong?
> >
> > Thank you!
> >
> >
> >
> >
> >
> >
> >
> > >-------- Оригинално писмо --------
> >
> > >От: Chesnay Schepler chesnay@apache.org
> >
> > >Относно: Re: how many 'run -c' commands to start?
> >
> > >До: user@flink.apache.org
> >
> > >Изпратено на: 29.09.2017 18:01
> >
> >
> >
> >
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> The only nodes that matter are those on which the Flink processes, i.e
> >
> >> Job- and TaskManagers, are being run.
> >
> >>
> >
> >> To prevent a JobManager node failure from causing the job to fail you have to look into an
> >
> >> HA setup.
> >
> >> (The jobmanager is responsible for distributing/coordinating work)
> >
> >>
> >
> >> In case of a TaskManager node failure the job will fail and
> >
> >> restart, provided that enough TaskManagers are
> >
> >> still alive to satisfy the resource requirements of the job.
> >
> >>
> >
> >>
> >
> >> Can you elaborate a bit more what happened when you used the --detached param?
> >
> >>
> >
> >> On 28.09.2017 16:33, r. r. wrote:
> >
> >>
> >
> >>
> >
> >>
> >
> >> Thank you, Chesnay
> >
> >> to make sure - should the node where the job has been submitted goes down, the processing will continue, I hope?
> >
> >> Do I need to ensure this by configuration?
> >
> >
> >> btw I added --detached param to the run cmd, but it didn't go into background process as I would've expected. Am I guessing wrong?
> >
> >
> >> Thanks!
> >
> >> Rob
> >
> >
> >> >-------- Оригинално писмо --------
> >
> >
> >> >От: Chesnay Schepler chesnay@apache.org
> >
> >
> >> >Относно: Re: how many 'run -c' commands to start?
> >
> >
> >> >До: user@flink.apache.org
> >
> >
> >> >Изпратено на: 28.09.2017 15:05
> >
> >
> >>
> >
> >>
> >
> >>
> >
> >> Hi!
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> Given a Flink cluster, you would only call `flink run ...` to submit a
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> job once; for simplicity i would submit it on the node where you started
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> the cluster. Flink will automatically distribute job across the cluster,
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> in smaller independent parts known as Tasks.
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> Regards,
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> Chesnay
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> On 28.09.2017 08:31, r. r. wrote:
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> Hello
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> I successfully ran a job with 'flink run -c', but this is for the local
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> setup.
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> How should i proceed with a cluster? Will flink automagically instantiate
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> the job on all servers - i hope i don't have to start 'flink run -c' on all
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> machines.
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> New to flink and bigdata, so sorry for the probably silly question
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> Thanks!
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >> Rob
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
> >
> >>
Re: how many 'run -c' commands to start?
Posted by Chesnay Schepler <ch...@apache.org>.
The order in which you pass program arguments is quite important. Can
you try again putting --detached directly after run?
I.e. flink run --detached -c ...
The reason being that arguments after the jar are treated as arguments
for the user-program, and not as arguments for the job execution.
On 29.09.2017 17:23, r. r. wrote:
> Sure, here is the cmdline output:
>
> flink-1.3.2/bin/flink run -c com.corp.flink.KafkaJob quickstart.jar --topic InputQ --bootstrap.servers localhost:9092 --zookeeper.connect localhost:2181 --group.id Consumers -p 5 --detached
> Cluster configuration: Standalone cluster with JobManager at localhost/127.0.0.1:6123
> Using address localhost:6123 to connect to JobManager.
> JobManager web interface address http://localhost:8081
> Starting execution of program
> Submitting job with JobID: 5cc74547361cec2ff9874764dac9ee91. Waiting for job completion.
> Connected to JobManager at Actor[akka.tcp://flink@localhost:6123/user/jobmanager#-100905913] with leader session id 00000000-0000-0000-0000-000000000000.
> 09/28/2017 16:30:40 Job execution switched to status RUNNING.
> 09/28/2017 16:30:40 Source: Custom Source -> Map(1/5) switched to SCHEDULED
> 09/28/2017 16:30:40 Source: Custom Source -> Map(2/5) switched to SCHEDULED
> 09/28/2017 16:30:40 Source: Custom Source -> Map(3/5) switched to SCHEDULED
> 09/28/2017 16:30:40 Source: Custom Source -> Map(4/5) switched to SCHEDULED
> 09/28/2017 16:30:40 Source: Custom Source -> Map(5/5) switched to SCHEDULED
> 09/28/2017 16:30:40 Sink: Unnamed(1/5) switched to SCHEDULED
> 09/28/2017 16:30:40 Sink: Unnamed(2/5) switched to SCHEDULED
> 09/28/2017 16:30:40 Sink: Unnamed(3/5) switched to SCHEDULED
> 09/28/2017 16:30:40 Sink: Unnamed(4/5) switched to SCHEDULED
> 09/28/2017 16:30:40 Sink: Unnamed(5/5) switched to SCHEDULED
> ...
>
> I thought --detach will put the process in the background, and give me back the cmdline, but maybe I got the meaning behind this option wrong?
>
> Thank you!
>
>
>
>
>
>
>
> >-------- Оригинално писмо --------
>
> >От: Chesnay Schepler chesnay@apache.org
>
> >Относно: Re: how many 'run -c' commands to start?
>
> >До: user@flink.apache.org
>
> >Изпратено на: 29.09.2017 18:01
>
>
>
>
>
>>
>
>>
>
>>
>
>>
>
>> The only nodes that matter are those on which the Flink processes, i.e
>
>> Job- and TaskManagers, are being run.
>
>>
>
>> To prevent a JobManager node failure from causing the job to fail you have to look into an
>
>> HA setup.
>
>> (The jobmanager is responsible for distributing/coordinating work)
>
>>
>
>> In case of a TaskManager node failure the job will fail and
>
>> restart, provided that enough TaskManagers are
>
>> still alive to satisfy the resource requirements of the job.
>
>>
>
>>
>
>> Can you elaborate a bit more what happened when you used the --detached param?
>
>>
>
>> On 28.09.2017 16:33, r. r. wrote:
>
>>
>
>>
>
>>
>
>> Thank you, Chesnay
>
>> to make sure - should the node where the job has been submitted goes down, the processing will continue, I hope?
>
>> Do I need to ensure this by configuration?
>
>
>> btw I added --detached param to the run cmd, but it didn't go into background process as I would've expected. Am I guessing wrong?
>
>
>> Thanks!
>
>> Rob
>
>
>> >-------- Оригинално писмо --------
>
>
>> >От: Chesnay Schepler chesnay@apache.org
>
>
>> >Относно: Re: how many 'run -c' commands to start?
>
>
>> >До: user@flink.apache.org
>
>
>> >Изпратено на: 28.09.2017 15:05
>
>
>>
>
>>
>
>>
>
>> Hi!
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> Given a Flink cluster, you would only call `flink run ...` to submit a
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> job once; for simplicity i would submit it on the node where you started
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> the cluster. Flink will automatically distribute job across the cluster,
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> in smaller independent parts known as Tasks.
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> Regards,
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> Chesnay
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> On 28.09.2017 08:31, r. r. wrote:
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> Hello
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> I successfully ran a job with 'flink run -c', but this is for the local
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> setup.
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> How should i proceed with a cluster? Will flink automagically instantiate
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> the job on all servers - i hope i don't have to start 'flink run -c' on all
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> machines.
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> New to flink and bigdata, so sorry for the probably silly question
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> Thanks!
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> Rob
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
Re: how many 'run -c' commands to start?
Posted by "r. r." <ro...@abv.bg>.
Sure, here is the cmdline output:
flink-1.3.2/bin/flink run -c com.corp.flink.KafkaJob quickstart.jar --topic InputQ --bootstrap.servers localhost:9092 --zookeeper.connect localhost:2181 --group.id Consumers -p 5 --detached
Cluster configuration: Standalone cluster with JobManager at localhost/127.0.0.1:6123
Using address localhost:6123 to connect to JobManager.
JobManager web interface address http://localhost:8081
Starting execution of program
Submitting job with JobID: 5cc74547361cec2ff9874764dac9ee91. Waiting for job completion.
Connected to JobManager at Actor[akka.tcp://flink@localhost:6123/user/jobmanager#-100905913] with leader session id 00000000-0000-0000-0000-000000000000.
09/28/2017 16:30:40 Job execution switched to status RUNNING.
09/28/2017 16:30:40 Source: Custom Source -> Map(1/5) switched to SCHEDULED
09/28/2017 16:30:40 Source: Custom Source -> Map(2/5) switched to SCHEDULED
09/28/2017 16:30:40 Source: Custom Source -> Map(3/5) switched to SCHEDULED
09/28/2017 16:30:40 Source: Custom Source -> Map(4/5) switched to SCHEDULED
09/28/2017 16:30:40 Source: Custom Source -> Map(5/5) switched to SCHEDULED
09/28/2017 16:30:40 Sink: Unnamed(1/5) switched to SCHEDULED
09/28/2017 16:30:40 Sink: Unnamed(2/5) switched to SCHEDULED
09/28/2017 16:30:40 Sink: Unnamed(3/5) switched to SCHEDULED
09/28/2017 16:30:40 Sink: Unnamed(4/5) switched to SCHEDULED
09/28/2017 16:30:40 Sink: Unnamed(5/5) switched to SCHEDULED
...
I thought --detach will put the process in the background, and give me back the cmdline, but maybe I got the meaning behind this option wrong?
Thank you!
>-------- Оригинално писмо --------
>От: Chesnay Schepler chesnay@apache.org
>Относно: Re: how many 'run -c' commands to start?
>До: user@flink.apache.org
>Изпратено на: 29.09.2017 18:01
>
>
>
>
>
> The only nodes that matter are those on which the Flink processes, i.e
> Job- and TaskManagers, are being run.
>
> To prevent a JobManager node failure from causing the job to fail you have to look into an
> HA setup.
> (The jobmanager is responsible for distributing/coordinating work)
>
> In case of a TaskManager node failure the job will fail and
> restart, provided that enough TaskManagers are
> still alive to satisfy the resource requirements of the job.
>
>
> Can you elaborate a bit more what happened when you used the --detached param?
>
> On 28.09.2017 16:33, r. r. wrote:
>
>
>
> Thank you, Chesnay
> to make sure - should the node where the job has been submitted goes down, the processing will continue, I hope?
> Do I need to ensure this by configuration?
>
> btw I added --detached param to the run cmd, but it didn't go into background process as I would've expected. Am I guessing wrong?
>
> Thanks!
> Rob
>
> >-------- Оригинално писмо --------
>
> >От: Chesnay Schepler chesnay@apache.org
>
> >Относно: Re: how many 'run -c' commands to start?
>
> >До: user@flink.apache.org
>
> >Изпратено на: 28.09.2017 15:05
>
>
>
>
> Hi!
>
>
>
>
>
>
>
>
>
>
> Given a Flink cluster, you would only call `flink run ...` to submit a
>
>
>
>
>
> job once; for simplicity i would submit it on the node where you started
>
>
>
>
>
> the cluster. Flink will automatically distribute job across the cluster,
>
>
>
>
>
> in smaller independent parts known as Tasks.
>
>
>
>
>
>
>
>
>
>
> Regards,
>
>
>
>
>
> Chesnay
>
>
>
>
>
>
>
>
>
>
> On 28.09.2017 08:31, r. r. wrote:
>
>
>
>
>
>
> Hello
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> I successfully ran a job with 'flink run -c', but this is for the local
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> setup.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> How should i proceed with a cluster? Will flink automagically instantiate
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> the job on all servers - i hope i don't have to start 'flink run -c' on all
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> machines.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> New to flink and bigdata, so sorry for the probably silly question
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Thanks!
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Rob
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Re: how many 'run -c' commands to start?
Posted by Chesnay Schepler <ch...@apache.org>.
The only nodes that matter are those on which the Flink processes, i.e
Job- and TaskManagers
<https://ci.apache.org/projects/flink/flink-docs-release-1.4/concepts/runtime.html#job-managers-task-managers-clients>,
are being run.
To prevent a JobManager node failure from causing the job to fail you
have to look into an HA setup
<https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/jobmanager_high_availability.html>.
(The jobmanager is responsible for distributing/coordinating work)
In case of a TaskManager node failure the job will fail and restart
<https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/restart_strategies.html>,
provided that enough TaskManagers are
still alive to satisfy the resource requirements of the job.
Can you elaborate a bit more what happened when you used the --detached
param?
On 28.09.2017 16:33, r. r. wrote:
> Thank you, Chesnay
> to make sure - should the node where the job has been submitted goes down, the processing will continue, I hope?
> Do I need to ensure this by configuration?
>
> btw I added --detached param to the run cmd, but it didn't go into background process as I would've expected. Am I guessing wrong?
>
> Thanks!
> Rob
>
>
>
>
>
>
> >-------- Оригинално писмо --------
>
> >От: Chesnay Schepler chesnay@apache.org
>
> >Относно: Re: how many 'run -c' commands to start?
>
> >До: user@flink.apache.org
>
> >Изпратено на: 28.09.2017 15:05
>
>
>
>
>> Hi!
>
>
>> Given a Flink cluster, you would only call `flink run ...` to submit a
>
>> job once; for simplicity i would submit it on the node where you started
>
>> the cluster. Flink will automatically distribute job across the cluster,
>
>> in smaller independent parts known as Tasks.
>
>
>> Regards,
>
>> Chesnay
>
>
>> On 28.09.2017 08:31, r. r. wrote:
>
>>> Hello
>
>
>>> I successfully ran a job with 'flink run -c', but this is for the local
>
>
>>> setup.
>
>
>>> How should i proceed with a cluster? Will flink automagically instantiate
>
>
>>> the job on all servers - i hope i don't have to start 'flink run -c' on all
>
>
>>> machines.
>
>
>>> New to flink and bigdata, so sorry for the probably silly question
>
>
>
>
>>> Thanks!
>
>
>>> Rob
>
>
Re: how many 'run -c' commands to start?
Posted by "r. r." <ro...@abv.bg>.
Thank you, Chesnay
to make sure - should the node where the job has been submitted goes down, the processing will continue, I hope?
Do I need to ensure this by configuration?
btw I added --detached param to the run cmd, but it didn't go into background process as I would've expected. Am I guessing wrong?
Thanks!
Rob
>-------- Оригинално писмо --------
>От: Chesnay Schepler chesnay@apache.org
>Относно: Re: how many 'run -c' commands to start?
>До: user@flink.apache.org
>Изпратено на: 28.09.2017 15:05
> Hi!
>
> Given a Flink cluster, you would only call `flink run ...` to submit a
> job once; for simplicity i would submit it on the node where you started
> the cluster. Flink will automatically distribute job across the cluster,
> in smaller independent parts known as Tasks.
>
> Regards,
> Chesnay
>
> On 28.09.2017 08:31, r. r. wrote:
> > Hello
> >
> > I successfully ran a job with 'flink run -c', but this is for the local
> >
> > setup.
> >
> > How should i proceed with a cluster? Will flink automagically instantiate
> >
> > the job on all servers - i hope i don't have to start 'flink run -c' on all
> >
> > machines.
> >
> > New to flink and bigdata, so sorry for the probably silly question
> >
> >
> >
> > Thanks!
> >
> > Rob
> >
> >
Re: how many 'run -c' commands to start?
Posted by Chesnay Schepler <ch...@apache.org>.
Hi!
Given a Flink cluster, you would only call `flink run ...` to submit a
job once; for simplicity i would submit it on the node where you started
the cluster. Flink will automatically distribute job across the cluster,
in smaller independent parts known as Tasks.
Regards,
Chesnay
On 28.09.2017 08:31, r. r. wrote:
> Hello
>
> I successfully ran a job with 'flink run -c', but this is for the local
>
> setup.
>
> How should i proceed with a cluster? Will flink automagically instantiate
>
> the job on all servers - i hope i don't have to start 'flink run -c' on all
>
> machines.
>
> New to flink and bigdata, so sorry for the probably silly question
>
>
>
> Thanks!
>
> Rob
>
>