You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "SwathiChandrashekar (Jira)" <ji...@apache.org> on 2022/12/11 12:22:00 UTC
[jira] [Updated] (FLINK-30361) Flink cluster deleted while updating the replicas
[ https://issues.apache.org/jira/browse/FLINK-30361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
SwathiChandrashekar updated FLINK-30361:
----------------------------------------
Description:
Whenever we try to update the replicas of the task manager for a flink standalone cluster using the flink CR, any change in CR triggers a redeploy of the flink cluster ( delete + create of all the components - JM and TM ).
This might not be required for replica update and this should not affect the existing pods and only a new TM pod will added during a scale up and a TM pod should be deleted during a scale down.
Example tried --> Change the TM replicas from 2 to 3.
bq. PS C:\Users\cswathi\Documents\flink-OSS-operator> kubectl get pods -w
bq.
bq. NAME READY STATUS RESTARTS AGE
bq.
bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 ContainerCreating 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 ContainerCreating 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 ContainerCreating 0 1s
bq.
bq. flink-kubernetes-operator-676897686f-5fc8r 2/2 Running 0 18m
bq.
bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 1/1 Running 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 1/1 Running 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 1/1 Running 0 13s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 1/1 Terminating 0 65s
bq.
bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 1/1 Terminating 0 65s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 1/1 Terminating 0 65s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 1/1 Terminating 0 66s
bq.
bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 1/1 Terminating 0 66s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 1/1 Terminating 0 66s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 Terminating 0 66s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 Terminating 0 66s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 Terminating 0 66s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 Terminating 0 67s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 Terminating 0 67s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 Terminating 0 67s
bq.
bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 Terminating 0 67s
bq.
bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 Terminating 0 67s
bq.
bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 Terminating 0 67s
bq.
bq. basic-session-deployment-only-example-588474bf97-nng85 0/1 Pending 0 0s
bq.
bq. basic-session-deployment-only-example-588474bf97-nng85 0/1 Pending 0 0s
bq.
bq. basic-session-deployment-only-example-588474bf97-nng85 0/1 ContainerCreating 0 0s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 Pending 0 0s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 Pending 0 0s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 Pending 0 0s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 Pending 0 0s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 Pending 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 Pending 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 ContainerCreating 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 ContainerCreating 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 ContainerCreating 0 1s
bq.
bq. basic-session-deployment-only-example-588474bf97-nng85 0/1 ContainerCreating 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 ContainerCreating 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 ContainerCreating 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 ContainerCreating 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 1/1 Running 0 1s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 1/1 Running 0 1s
bq.
bq. basic-session-deployment-only-example-588474bf97-nng85 1/1 Running 0 2s
bq.
bq. basic-session-deployment-only-example-taskmanager-77854fbb57v4t 1/1 Running 0 17s
was:
Whenever we try to update the replicas of the task manager for a flink standalone cluster using the flink CR, any change in CR triggers a redeploy of the flink cluster ( delete + create of all the components - JM and TM ).
This might not be required for replica update and this should not affect the existing pods and only a new TM pod will added during a scale up and a TM pod should be deleted during a scale down.
Example tried --> Change the TM replicas from 2 to 3.
```
PS C:\Users\cswathi\Documents\hilo\flink-OSS-operator> kubectl get pods -w
NAME READY STATUS RESTARTS AGE
basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 ContainerCreating 0 1s
basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 ContainerCreating 0 1s
basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 ContainerCreating 0 1s
flink-kubernetes-operator-676897686f-5fc8r 2/2 Running 0 18m
basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 1/1 Running 0 1s
basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 1/1 Running 0 1s
basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 1/1 Running 0 13s
basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 1/1 Terminating 0 65s
basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 1/1 Terminating 0 65s
basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 1/1 Terminating 0 65s
basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 1/1 Terminating 0 66s
basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 1/1 Terminating 0 66s
basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 1/1 Terminating 0 66s
basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 Terminating 0 66s
basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 Terminating 0 66s
basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 Terminating 0 66s
basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 Terminating 0 67s
basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 Terminating 0 67s
basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 Terminating 0 67s
basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 Terminating 0 67s
basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 Terminating 0 67s
basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 Terminating 0 67s
basic-session-deployment-only-example-588474bf97-nng85 0/1 Pending 0 0s
basic-session-deployment-only-example-588474bf97-nng85 0/1 Pending 0 0s
basic-session-deployment-only-example-588474bf97-nng85 0/1 ContainerCreating 0 0s
basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 Pending 0 0s
basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 Pending 0 0s
basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 Pending 0 0s
basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 Pending 0 0s
basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 Pending 0 1s
basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 Pending 0 1s
basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 ContainerCreating 0 1s
basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 ContainerCreating 0 1s
basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 ContainerCreating 0 1s
basic-session-deployment-only-example-588474bf97-nng85 0/1 ContainerCreating 0 1s
basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 ContainerCreating 0 1s
basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 ContainerCreating 0 1s
basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 ContainerCreating 0 1s
basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 1/1 Running 0 1s
basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 1/1 Running 0 1s
basic-session-deployment-only-example-588474bf97-nng85 1/1 Running 0 2s
basic-session-deployment-only-example-taskmanager-77854fbb57v4t 1/1 Running 0 17s
```
> Flink cluster deleted while updating the replicas
> -------------------------------------------------
>
> Key: FLINK-30361
> URL: https://issues.apache.org/jira/browse/FLINK-30361
> Project: Flink
> Issue Type: Bug
> Components: Kubernetes Operator
> Affects Versions: kubernetes-operator-1.2.0
> Reporter: SwathiChandrashekar
> Priority: Major
>
> Whenever we try to update the replicas of the task manager for a flink standalone cluster using the flink CR, any change in CR triggers a redeploy of the flink cluster ( delete + create of all the components - JM and TM ).
> This might not be required for replica update and this should not affect the existing pods and only a new TM pod will added during a scale up and a TM pod should be deleted during a scale down.
> Example tried --> Change the TM replicas from 2 to 3.
> bq. PS C:\Users\cswathi\Documents\flink-OSS-operator> kubectl get pods -w
> bq.
> bq. NAME READY STATUS RESTARTS AGE
> bq.
> bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 ContainerCreating 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 ContainerCreating 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 ContainerCreating 0 1s
> bq.
> bq. flink-kubernetes-operator-676897686f-5fc8r 2/2 Running 0 18m
> bq.
> bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 1/1 Running 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 1/1 Running 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 1/1 Running 0 13s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 1/1 Terminating 0 65s
> bq.
> bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 1/1 Terminating 0 65s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 1/1 Terminating 0 65s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 1/1 Terminating 0 66s
> bq.
> bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 1/1 Terminating 0 66s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 1/1 Terminating 0 66s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 Terminating 0 66s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 Terminating 0 66s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb7vzvd 0/1 Terminating 0 66s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 Terminating 0 67s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 Terminating 0 67s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbg6vzs 0/1 Terminating 0 67s
> bq.
> bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 Terminating 0 67s
> bq.
> bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 Terminating 0 67s
> bq.
> bq. basic-session-deployment-only-example-5dbbdf5dd8-cq8nb 0/1 Terminating 0 67s
> bq.
> bq. basic-session-deployment-only-example-588474bf97-nng85 0/1 Pending 0 0s
> bq.
> bq. basic-session-deployment-only-example-588474bf97-nng85 0/1 Pending 0 0s
> bq.
> bq. basic-session-deployment-only-example-588474bf97-nng85 0/1 ContainerCreating 0 0s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 Pending 0 0s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 Pending 0 0s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 Pending 0 0s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 Pending 0 0s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 Pending 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 Pending 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 ContainerCreating 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 ContainerCreating 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 ContainerCreating 0 1s
> bq.
> bq. basic-session-deployment-only-example-588474bf97-nng85 0/1 ContainerCreating 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 0/1 ContainerCreating 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb57v4t 0/1 ContainerCreating 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 0/1 ContainerCreating 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbbrfgvz 1/1 Running 0 1s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb5ddxv 1/1 Running 0 1s
> bq.
> bq. basic-session-deployment-only-example-588474bf97-nng85 1/1 Running 0 2s
> bq.
> bq. basic-session-deployment-only-example-taskmanager-77854fbb57v4t 1/1 Running 0 17s
--
This message was sent by Atlassian Jira
(v8.20.10#820010)