[jira] [Assigned] (FLINK-13093) Provide an easy way to modify max parallelism using the State Processor API

["Tzu-Li (Gordon) Tai (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-13093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tzu-Li (Gordon) Tai reassigned FLINK-13093:
-------------------------------------------

    Assignee: Seth Wiesman

> Provide an easy way to modify max parallelism using the State Processor API
> ---------------------------------------------------------------------------
>
>                 Key: FLINK-13093
>                 URL: https://issues.apache.org/jira/browse/FLINK-13093
>             Project: Flink
>          Issue Type: Sub-task
>            Reporter: Tzu-Li (Gordon) Tai
>            Assignee: Seth Wiesman
>            Priority: Major
>
> Currently, the State Processor API does not easily allow one to modify the max parallelism of a job. To do that with the current state of the API, one would have to read all state that exists in a loaded {{ExistingSavepoint}}, extract them as {{DataSet}}s, and then create a {{NewSavepoint}} that has the new max parallelism with all the extracted data sets bootstrapped as new state.
> It would be nice if the user could simply do something like the following (API is TBD):
> {code}
> ExistingSavepoint savepoint = Savepoint.load("path", env, backend);
> savepoint.modifyMaxParallelism("newPath", newParallelism);
> {code}
> Under the hood, a batch job is launched that repartitions all existing operator state using the new max parallelism, and writes to the re-partitioned state data into the new savepoint path.
> The API can be designed such that the user cannot modify the max parallelism and add / new operators at the same time, to not over complicate the batch job.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)