[jira] [Comment Edited] (FLINK-33944) Apache Flink: Process to restore more than one job on job manager startup from the respective savepoints

["Martijn Visser (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-33944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800796#comment-17800796 ] 

Martijn Visser edited comment on FLINK-33944 at 12/27/23 2:51 PM:
------------------------------------------------------------------

[~vrangana@in.ibm.com] Given that Application Mode is positioned to have a 1:1 mapping between a Flink Job Manager and a Flink Application, this feels like the wrong way to deploy this? I don't think we should go this route. I think you should consider Session Mode https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/overview/#application-mode


was (Author: martijnvisser):
[~vrangana@in.ibm.com] Given that Application Mode is positioned to have a 1:1 mapping between a Flink Job Manager and a Flink Application, this feels like the wrong way to deploy this? I don't think we should go this route. 

> Apache Flink: Process to restore more than one job on job manager startup from the respective savepoints
> --------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-33944
>                 URL: https://issues.apache.org/jira/browse/FLINK-33944
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Checkpointing
>    Affects Versions: 1.18.0
>            Reporter: Vijay
>            Priority: Major
>
>  
> We are using Flink (1.18) version for our Flink cluster. The job manager has been deployed in "Application mode" and we are looking for a process to restore multiple jobs (using their respective savepoint directories) when the job manager is started. Currently, we have the option to restore only one job while running "standalone-job.sh" using the --fromSavepoint and --allowNonRestoredState. However, we need a way to trigger multiple job executions via Java client (from its respective savepoint location) on Jobmanager startup.
> Note: We are not using a Kubernetes native deployment, but we are using k8s standalone mode of deployment.
> Additional Query: If there is a process to restore multiple jobs from its respective savepoints on "Application mode" of deployment, is the same supported on Session mode of deployment or not?
> *Expected process:*
>  # Before starting with the Flink/application image upgrade, trigger the savepoints for all the current running jobs.
>  # Once the savepoints process completed for all jobs, will trigger the scale down of job manager and task manager instances.
>  # Update the image version on the k8s deployment with the update application image.
>  # After image version is updated, scale up the job manager and task manager.
>  # We need a process to restore the previously running jobs from the savepoint dir and start all the jobs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)