Clarity on Flink 1.5 Rescale mechanism

[Sampath Bhat <sa...@gmail.com>]

Hello
In flink 1.5 release notes -
https://flink.apache.org/news/2018/05/25/release-1.5.0.html#release-notes

Various Other Features and Improvements:
Applications can be rescaled without manually triggering a savepoint. Under
the hood, Flink will still take a savepoint, stop the application, and
rescale it to the new parallelism.

What exactly does this statement mean?
How to rescale the application ?
If i'm running my flink job application with parallelism of 5 and then I
execute the following command
flink run <jar_path> -p 10 then will the application rescale to 10
parallelism?

Adding on what will be the behavior in flink 1.5 if I increase the number
of task manager in a flink cluster?

So if i have a flink job running on flink cluster having 1 task manager and
i increase the number of task manager to 2 will the flink rescale the flink
job also or will the flink job be unaffected?

Re: Clarity on Flink 1.5 Rescale mechanism

[Stefan Richter <s....@data-artisans.com>]

Hi,

it means that you can now modify the parallelism of a running job with a new „modify“ command in the CLI, see [1]. Adding task manager will add their offered slots to the pool of available slots, it will not automatically change the parallelism.

Best,
Stefan

[1] https://ci.apache.org/projects/flink/flink-docs-master/ops/cli.html <https://ci.apache.org/projects/flink/flink-docs-master/ops/cli.html>

> Am 12.06.2018 um 11:50 schrieb Sampath Bhat <sa...@gmail.com>:
> 
> Hello
> In flink 1.5 release notes -
> https://flink.apache.org/news/2018/05/25/release-1.5.0.html#release-notes
> 
> Various Other Features and Improvements:
> Applications can be rescaled without manually triggering a savepoint. Under
> the hood, Flink will still take a savepoint, stop the application, and
> rescale it to the new parallelism.
> 
> What exactly does this statement mean?
> How to rescale the application ?
> If i'm running my flink job application with parallelism of 5 and then I
> execute the following command
> flink run <jar_path> -p 10 then will the application rescale to 10
> parallelism?
> 
> Adding on what will be the behavior in flink 1.5 if I increase the number
> of task manager in a flink cluster?
> 
> So if i have a flink job running on flink cluster having 1 task manager and
> i increase the number of task manager to 2 will the flink rescale the flink
> job also or will the flink job be unaffected?

Re: Clarity on Flink 1.5 Rescale mechanism

[Stefan Richter <s....@data-artisans.com>]

Hi,

it means that you can now modify the parallelism of a running job with a new „modify“ command in the CLI, see [1]. Adding task manager will add their offered slots to the pool of available slots, it will not automatically change the parallelism.

Best,
Stefan

[1] https://ci.apache.org/projects/flink/flink-docs-master/ops/cli.html <https://ci.apache.org/projects/flink/flink-docs-master/ops/cli.html>

> Am 12.06.2018 um 11:50 schrieb Sampath Bhat <sa...@gmail.com>:
> 
> Hello
> In flink 1.5 release notes -
> https://flink.apache.org/news/2018/05/25/release-1.5.0.html#release-notes
> 
> Various Other Features and Improvements:
> Applications can be rescaled without manually triggering a savepoint. Under
> the hood, Flink will still take a savepoint, stop the application, and
> rescale it to the new parallelism.
> 
> What exactly does this statement mean?
> How to rescale the application ?
> If i'm running my flink job application with parallelism of 5 and then I
> execute the following command
> flink run <jar_path> -p 10 then will the application rescale to 10
> parallelism?
> 
> Adding on what will be the behavior in flink 1.5 if I increase the number
> of task manager in a flink cluster?
> 
> So if i have a flink job running on flink cluster having 1 task manager and
> i increase the number of task manager to 2 will the flink rescale the flink
> job also or will the flink job be unaffected?