You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "kiranchavala (via GitHub)" <gi...@apache.org> on 2026/02/24 12:37:48 UTC

[I] CKS cluster remains in Alert state if the scaling fails due to capacity issue on the hypervisor host [cloudstack]

kiranchavala opened a new issue, #12699:
URL: https://github.com/apache/cloudstack/issues/12699

   ### problem
   
   CKS cluster remains in Alert state if the scaling fails due to capacity issue on the hypervisor host
   
   ### versions
   
   ACS 4.22
   
   ### The steps to reproduce the bug
   
   Have cloudstack environment with 2 kvm host in a cluster 
   
   1. Launch a Cks cluster with size 2 ( worker nodes) 
   
   Worker nodes deployed on kvm host 2 
   
   2. CKS cluster in running state 
   
   3. Deploy other vm's in the cloudstack environment  so that capacity of kvm host have reached 
   
   4. Scale the CKS cluster to size 3 
   
   5. Scaling of the CKS cluster fails due to capacity issue 
   
   The new worker node will be in stopped state 
   
   6. CKS cluster will be in Alert state 
   
   ```
   
   2026-02-24 11:12:14,223 DEBUG [c.c.k.c.KubernetesClusterManagerImpl] (Kubernetes-Cluster-State-Scanner-1:[ctx-c196e036]) (logid:43979d1a) Found VM: VM instance {"id":16,"instanceName":"i-2-16-VM","state":"Stopped","type":"User","uuid":"47386d74-3c9f-49aa-b102-1c10537c8350"} in the Kubernetes cluster KubernetesCluster {"id":2,"name":"test","uuid":"e155ab23-68ca-4c3e-b8c5-7175a3f65fda"} in state: Stopped while expected to be in state: Running. So moving the cluster to Alert state for reconciliation
   2026-02-24 11:12:14,224 DEBUG [c.c.k.c.KubernetesClusterManagerImpl] (Kubernetes-Cluster-State-Scanner-1:[ctx-c196e036]) (logid:43979d1a) Found VM: VM instance {"id":9,"instanceName":"i-2-9-VM","state":"Running","type":"User","uuid":"ebf0a5a6-01b7-462a-bad6-1f61887f0f41"} in the Kubernetes cluster KubernetesCluster {"id":2,"name":"test","uuid":"e155ab23-68ca-4c3e-b8c5-7175a3f65fda"} in state: Running while expected to be in state: Stopped. So moving the cluster to Alert state for reconciliation
   ```
   
   7. Cannot remove the worker node which is stopped state 
   
   Exception thrown 
   
   <img width="1623" height="528" alt="Image" src="https://github.com/user-attachments/assets/32b8bad2-db9c-4686-ac57-3c26b9f9d378" />
   
   
   ### What to do about it?
   
   CKS cluster should go back to running state since the scaling failed  due to insufficent capacity issue 
   
   Currently, we are checking only for resource limit during scaling operation with this pr 
   
   https://github.com/apache/cloudstack/pull/12167
   
   We should also check host capacity before scaling 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] CKS cluster remains in Alert state if the scaling fails due to capacity issue on the hypervisor host [cloudstack]

Posted by "kiranchavala (via GitHub)" <gi...@apache.org>.
kiranchavala commented on issue #12699:
URL: https://github.com/apache/cloudstack/issues/12699#issuecomment-3957021265

   Closing the issue as its related to the pr 
   
   https://github.com/apache/cloudstack/pull/12386


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] CKS cluster remains in Alert state if the scaling fails due to capacity issue on the hypervisor host [cloudstack]

Posted by "kiranchavala (via GitHub)" <gi...@apache.org>.
kiranchavala closed issue #12699: CKS cluster remains in Alert state if the scaling fails due to capacity issue on the hypervisor host
URL: https://github.com/apache/cloudstack/issues/12699


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org