[jira] [Commented] (FLINK-29464) JobID generation logic could lead to state loss

["Gyula Fora (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-29464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610847#comment-17610847 ] 

Gyula Fora commented on FLINK-29464:
------------------------------------

cc [~thw] 

> JobID generation logic could lead to state loss
> -----------------------------------------------
>
>                 Key: FLINK-29464
>                 URL: https://issues.apache.org/jira/browse/FLINK-29464
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kubernetes Operator
>    Affects Versions: kubernetes-operator-1.2.0
>            Reporter: Gyula Fora
>            Assignee: Gyula Fora
>            Priority: Blocker
>             Fix For: kubernetes-operator-1.2.0
>
>
> The recently added job id override logic (https://issues.apache.org/jira/browse/FLINK-29109) can under certain cases lead to state loss.
> State loss scenario:
> 1. Either first deployment / Stateless upgrade mode used -> new jobId will be generated and set in jobStatus
> 2. Operator/deployment fails during or directly after successful submission -> status is not persisted with the generated jobId
> 3. User submits a spec update with last-state upgrade
> 4.  If the job was never observed (due to a failure or early spec update) a last-state upgrade would be performed, deleting the Deployment and simply submitting the job.
> 5. The current logic would then generate a new jobid (because it's still empty) leading to a failure to recover the state from HA -> data loss
> There are multiple ways to solve this issue:
>  a ) Record status after generating a jobid
>  b ) Only ever set the status during stateless deployment
>  c ) Verify no HA data is present before setting the jobid when empty
> Probably the most robust solution is a).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)