You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Ricardo Pertuz <ri...@kuasar.co.INVALID> on 2023/05/20 00:37:38 UTC

Cannot start vm after being running - just after upgrade to 4.18

Hi guys,





We did an upgrade from 4.15 to 4.18 and everything went smoothly! We can create new vms without any issue however if I stop previously created vms they won’t start again. Any Clue?





 * 
Hyp KVM

 * 
ACS 4.18.0

 * 
Primary Storage Ceph





Logs





2023-05-19 19:17:00,878 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Deploy avoids pods: null, clusters: null, hosts: null





2023-05-19 19:17:00,882 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) VM start attempt #1





2023-05-19 19:17:00,887 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Root volume is ready, need to place VM in volume's cluster





2023-05-19 19:17:00,887 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Vol[1607|name=ROOT-686|vm=686|ROOT] is READY, changing deployment plan to use this pool's dcId: 2 , podId: null , and clusterId: null





2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding pods to avoid lists for non-explicit VM deployment: []





2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding clusters to avoid lists for non-explicit VM deployment: []





2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding hosts to avoid lists for non-explicit VM deployment: []





2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) DeploymentPlanner allocation algorithm: null





2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Trying to allocate a host and storage pools from dc:2, pod:null,cluster:null, requested cpu: 4800, requested ram: (8.00 GB) 8589934592





2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Is ROOT volume READY (pool already allocated)?: Yes





2023-05-19 19:17:00,924 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Deploy avoids pods: [], clusters: [], hosts: []





2023-05-19 19:17:00,924 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Deploy hosts with priorities {} , hosts have NORMAL priority by default





2023-05-19 19:17:00,926 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) This VM has last host_id specified, trying to choose the same host: 2





2023-05-19 19:17:00,933 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Host {id: 2, name:co-centro1a-phy-k1-hyp-01.kuasar.cloud, uuid: ba87d8ff-21e0-4f20-8bda-b8fa76453a1c} is KVM hypervisor type, no max guest limit check needed





2023-05-19 19:17:00,950 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Host: 2 doesn't have cpu capability (cpu:128, speed:2399) to support requested CPU: 2 and requested speed: 2400





2023-05-19 19:17:00,950 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) The last host of this VM does not have enough capacity





2023-05-19 19:17:00,950 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Cannot choose the last host to deploy this VM





2023-05-19 19:17:00,955 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Searching all possible resources under this Zone: 2





2023-05-19 19:17:00,963 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Zone: 2





2023-05-19 19:17:00,976 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Removing from the clusterId list these clusters from avoid set: []





2023-05-19 19:17:01,010 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) The clusterId list for the given offering tag: [1]





2023-05-19 19:17:01,017 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Checking resources in Cluster: 1 under Pod: 3





2023-05-19 19:17:01,022 INFO  [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9)  Guest VM is requested with Custom[UEFI] Boot Type false





2023-05-19 19:17:01,023 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Looking for hosts in dc: 2  pod:3  cluster:1





2023-05-19 19:17:01,023 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Looking for hosts having tag specified on SvcOffering:k1





2023-05-19 19:17:01,033 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Hosts with tag 'k1' are:[Host {"id":2,"name":"co-centro1a-phy-k1-hyp-01.kuasar.cloud","type":"Routing","uuid":"ba87d8ff-21e0-4f20-8bda-b8fa76453a1c"}]





2023-05-19 19:17:01,037 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) FirstFitAllocator has 1 hosts to check for allocation: [Host {"id":2,"name":"co-centro1a-phy-k1-hyp-01.kuasar.cloud","type":"Routing","uuid":"ba87d8ff-21e0-4f20-8bda-b8fa76453a1c"}]





2023-05-19 19:17:01,047 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Found 1 hosts for allocation after prioritization: [Host {"id":2,"name":"co-centro1a-phy-k1-hyp-01.kuasar.cloud","type":"Routing","uuid":"ba87d8ff-21e0-4f20-8bda-b8fa76453a1c"}]





2023-05-19 19:17:01,047 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Looking for speed=4800Mhz, Ram=8192 MB





2023-05-19 19:17:01,047 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Host {id: 2, name:co-centro1a-phy-k1-hyp-01.kuasar.cloud, uuid: ba87d8ff-21e0-4f20-8bda-b8fa76453a1c} is KVM hypervisor type, no max guest limit check needed





2023-05-19 19:17:01,059 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Host: 2 doesn't have cpu capability (cpu:128, speed:2399) to support requested CPU: 2 and requested speed: 2400





2023-05-19 19:17:01,059 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Checking if host: 2 has enough capacity for requested CPU: 4800 and requested RAM: (8.00 GB) 8589934592 , cpuOverprovisioningFactor: 6.0





2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Hosts's actual total CPU: 307072 and CPU after applying overprovisioning: 1842432





2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Free CPU: 1623976 , Requested CPU: 4800





2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Free RAM: (726.06 GB) 779597905920 , Requested RAM: (8.00 GB) 8589934592





2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Host has enough CPU and RAM available





2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) STATS: Can alloc CPU from host: 2, used: 199256, reserved: 19200, actual total: 307072, total with overprovisioning: 1842432; requested cpu:4800,alloc_from_last_host?:false ,considerReservedCapacity?: true





2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) STATS: Can alloc MEM from host: 2, used: (248.74 GB) 267084890112, reserved: (32.00 GB) 34359738368, total: (1006.80 GB) 1081042534400; requested mem: (8.00 GB) 8589934592, alloc_from_last_host?: false , considerReservedCapacity?: true





2023-05-19 19:17:01,063 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Not using host 2; host has cpu capability? false, host has capacity?true





2023-05-19 19:17:01,064 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Host Allocator returning 0 suitable hosts





2023-05-19 19:17:01,064 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) No suitable hosts found





2023-05-19 19:17:01,064 INFO  [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Re-ordering hosts [] by priorities {}





2023-05-19 19:17:01,064 INFO  [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Hosts after re-ordering are: []





2023-05-19 19:17:01,064 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) No suitable hosts found under this Cluster: 1





2023-05-19 19:17:01,066 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Could not find suitable Deployment Destination for this VM under any clusters, returning.





2023-05-19 19:17:01,070 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Searching all possible resources under this Zone: 2





2023-05-19 19:17:01,073 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Zone: 2





2023-05-19 19:17:01,079 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Removing from the clusterId list these clusters from avoid set: [1]





2023-05-19 19:17:01,083 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) The clusterId list for the given offering tag: []





2023-05-19 19:17:01,083 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) No clusters found after removing disabled clusters and clusters in avoid list, returning.





2023-05-19 19:17:01,085 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) VM start attempt #2





2023-05-19 19:17:01,103 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding pods to avoid lists for non-explicit VM deployment: []





2023-05-19 19:17:01,104 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding clusters to avoid lists for non-explicit VM deployment: []





2023-05-19 19:17:01,104 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding hosts to avoid lists for non-explicit VM deployment: []





2023-05-19 19:17:01,104 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) DeploymentPlanner allocation algorithm: null





2023-05-19 19:17:01,104 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Trying to allocate a host and storage pools from dc:2, pod:3,cluster:null, requested cpu: 4800, requested ram: (8.00 GB) 8589934592





2023-05-19 19:17:01,104 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Is ROOT volume READY (pool already allocated)?: No





2023-05-19 19:17:01,117 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Deploy avoids pods: [], clusters: [1], hosts: [2]





2023-05-19 19:17:01,117 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Deploy hosts with priorities {} , hosts have NORMAL priority by default





2023-05-19 19:17:01,119 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) This VM has last host_id specified, trying to choose the same host: 2





2023-05-19 19:17:01,126 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) The last host of this VM is in avoid set





2023-05-19 19:17:01,126 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Cannot choose the last host to deploy this VM





2023-05-19 19:17:01,128 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Searching resources only under specified Pod: 3





2023-05-19 19:17:01,133 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Pod: 3





2023-05-19 19:17:01,141 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Removing from the clusterId list these clusters from avoid set: [1]





2023-05-19 19:17:01,145 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) The clusterId list for the given offering tag: []





2023-05-19 19:17:01,145 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) No clusters found after removing disabled clusters and clusters in avoid list, returning.





2023-05-19 19:17:01,179 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) VM instance {"id":686,"instanceName":"i-31-686-VM","type":"User","uuid":"d7a8a80a-bef4-4778-9f3c-fc9c06b2cfe1"} state transited from [Starting] to [Stopped] with event [OperationFailed]. VM's original host: Host {"id":2,"name":"co-centro1a-phy-k1-hyp-01.kuasar.cloud","type":"Routing","uuid":"ba87d8ff-21e0-4f20-8bda-b8fa76453a1c"}, new host: null, host before state transition: null





2023-05-19 19:17:01,198 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Invocation exception, caused by:com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":686,"instanceName":"i-31-686-VM","type":"User","uuid":"d7a8a80a-bef4-4778-9f3c-fc9c06b2cfe1"}Scope=interfacecom.cloud.dc.DataCenter; id=2





2023-05-19 19:17:01,198 INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Rethrow exceptioncom.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":686,"instanceName":"i-31-686-VM","type":"User","uuid":"d7a8a80a-bef4-4778-9f3c-fc9c06b2cfe1"}Scope=interfacecom.cloud.dc.DataCenter; id=2





2023-05-19 19:17:01,198 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097) (logid:6edb3ce9) Done with run of VM work job:com.cloud.vm.VmWorkStart for VM 686, job origin: 18095





BR,




Ricardo Pertuz

Re: Cannot start vm after being running - just after upgrade to 4.18

Posted by Ricardo Pertuz <ri...@kuasar.co.INVALID>.
I could solved it, this is the issue:

(cpu:128, speed:2399) to support requested CPU: 2 and requested speed: 2400

I had to update the serviceoffering directly in the database

don't know exactly why it detects now that my hyps have 2399 when I check on the hardware and have 2400.

BR,

Ricardo Pertuz



19 de mayo de 2023, 19:37, "Ricardo Pertuz" <ri...@kuasar.co.invalid> escribió:


> 
> Hi guys,
> 
> We did an upgrade from 4.15 to 4.18 and everything went smoothly! We can create new vms without any issue however if I stop previously created vms they won’t start again. Any Clue?
> 
>  * 
> Hyp KVM
> 
>  * 
> ACS 4.18.0
> 
>  * 
> Primary Storage Ceph
> 
> Logs
> 
> 2023-05-19 19:17:00,878 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Deploy avoids pods: null, clusters: null, hosts: null
> 
> 2023-05-19 19:17:00,882 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) VM start attempt #1
> 
> 2023-05-19 19:17:00,887 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Root volume is ready, need to place VM in volume's cluster
> 
> 2023-05-19 19:17:00,887 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Vol[1607|name=ROOT-686|vm=686|ROOT] is READY, changing deployment plan to use this pool's dcId: 2 , podId: null , and clusterId: null
> 
> 2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding pods to avoid lists for non-explicit VM deployment: []
> 
> 2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding clusters to avoid lists for non-explicit VM deployment: []
> 
> 2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding hosts to avoid lists for non-explicit VM deployment: []
> 
> 2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) DeploymentPlanner allocation algorithm: null
> 
> 2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Trying to allocate a host and storage pools from dc:2, pod:null,cluster:null, requested cpu: 4800, requested ram: (8.00 GB) 8589934592
> 
> 2023-05-19 19:17:00,909 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Is ROOT volume READY (pool already allocated)?: Yes
> 
> 2023-05-19 19:17:00,924 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Deploy avoids pods: [], clusters: [], hosts: []
> 
> 2023-05-19 19:17:00,924 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Deploy hosts with priorities {} , hosts have NORMAL priority by default
> 
> 2023-05-19 19:17:00,926 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) This VM has last host_id specified, trying to choose the same host: 2
> 
> 2023-05-19 19:17:00,933 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Host {id: 2, name:co-centro1a-phy-k1-hyp-01.kuasar.cloud, uuid: ba87d8ff-21e0-4f20-8bda-b8fa76453a1c} is KVM hypervisor type, no max guest limit check needed
> 
> 2023-05-19 19:17:00,950 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Host: 2 doesn't have cpu capability (cpu:128, speed:2399) to support requested CPU: 2 and requested speed: 2400
> 
> 2023-05-19 19:17:00,950 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) The last host of this VM does not have enough capacity
> 
> 2023-05-19 19:17:00,950 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Cannot choose the last host to deploy this VM
> 
> 2023-05-19 19:17:00,955 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Searching all possible resources under this Zone: 2
> 
> 2023-05-19 19:17:00,963 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Zone: 2
> 
> 2023-05-19 19:17:00,976 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Removing from the clusterId list these clusters from avoid set: []
> 
> 2023-05-19 19:17:01,010 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) The clusterId list for the given offering tag: [1]
> 
> 2023-05-19 19:17:01,017 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Checking resources in Cluster: 1 under Pod: 3
> 
> 2023-05-19 19:17:01,022 INFO  [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9)  Guest VM is requested with Custom[UEFI] Boot Type false
> 
> 2023-05-19 19:17:01,023 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Looking for hosts in dc: 2  pod:3  cluster:1
> 
> 2023-05-19 19:17:01,023 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Looking for hosts having tag specified on SvcOffering:k1
> 
> 2023-05-19 19:17:01,033 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Hosts with tag 'k1' are:[Host {"id":2,"name":"co-centro1a-phy-k1-hyp-01.kuasar.cloud","type":"Routing","uuid":"ba87d8ff-21e0-4f20-8bda-b8fa76453a1c"}]
> 
> 2023-05-19 19:17:01,037 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) FirstFitAllocator has 1 hosts to check for allocation: [Host {"id":2,"name":"co-centro1a-phy-k1-hyp-01.kuasar.cloud","type":"Routing","uuid":"ba87d8ff-21e0-4f20-8bda-b8fa76453a1c"}]
> 
> 2023-05-19 19:17:01,047 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Found 1 hosts for allocation after prioritization: [Host {"id":2,"name":"co-centro1a-phy-k1-hyp-01.kuasar.cloud","type":"Routing","uuid":"ba87d8ff-21e0-4f20-8bda-b8fa76453a1c"}]
> 
> 2023-05-19 19:17:01,047 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Looking for speed=4800Mhz, Ram=8192 MB
> 
> 2023-05-19 19:17:01,047 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Host {id: 2, name:co-centro1a-phy-k1-hyp-01.kuasar.cloud, uuid: ba87d8ff-21e0-4f20-8bda-b8fa76453a1c} is KVM hypervisor type, no max guest limit check needed
> 
> 2023-05-19 19:17:01,059 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Host: 2 doesn't have cpu capability (cpu:128, speed:2399) to support requested CPU: 2 and requested speed: 2400
> 
> 2023-05-19 19:17:01,059 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Checking if host: 2 has enough capacity for requested CPU: 4800 and requested RAM: (8.00 GB) 8589934592 , cpuOverprovisioningFactor: 6.0
> 
> 2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Hosts's actual total CPU: 307072 and CPU after applying overprovisioning: 1842432
> 
> 2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Free CPU: 1623976 , Requested CPU: 4800
> 
> 2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Free RAM: (726.06 GB) 779597905920 , Requested RAM: (8.00 GB) 8589934592
> 
> 2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Host has enough CPU and RAM available
> 
> 2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) STATS: Can alloc CPU from host: 2, used: 199256, reserved: 19200, actual total: 307072, total with overprovisioning: 1842432; requested cpu:4800,alloc_from_last_host?:false ,considerReservedCapacity?: true
> 
> 2023-05-19 19:17:01,063 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) STATS: Can alloc MEM from host: 2, used: (248.74 GB) 267084890112, reserved: (32.00 GB) 34359738368, total: (1006.80 GB) 1081042534400; requested mem: (8.00 GB) 8589934592, alloc_from_last_host?: false , considerReservedCapacity?: true
> 
> 2023-05-19 19:17:01,063 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Not using host 2; host has cpu capability? false, host has capacity?true
> 
> 2023-05-19 19:17:01,064 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611 FirstFitRoutingAllocator) (logid:6edb3ce9) Host Allocator returning 0 suitable hosts
> 
> 2023-05-19 19:17:01,064 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) No suitable hosts found
> 
> 2023-05-19 19:17:01,064 INFO  [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Re-ordering hosts [] by priorities {}
> 
> 2023-05-19 19:17:01,064 INFO  [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Hosts after re-ordering are: []
> 
> 2023-05-19 19:17:01,064 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) No suitable hosts found under this Cluster: 1
> 
> 2023-05-19 19:17:01,066 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Could not find suitable Deployment Destination for this VM under any clusters, returning.
> 
> 2023-05-19 19:17:01,070 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Searching all possible resources under this Zone: 2
> 
> 2023-05-19 19:17:01,073 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Zone: 2
> 
> 2023-05-19 19:17:01,079 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Removing from the clusterId list these clusters from avoid set: [1]
> 
> 2023-05-19 19:17:01,083 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) The clusterId list for the given offering tag: []
> 
> 2023-05-19 19:17:01,083 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) No clusters found after removing disabled clusters and clusters in avoid list, returning.
> 
> 2023-05-19 19:17:01,085 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) VM start attempt #2
> 
> 2023-05-19 19:17:01,103 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding pods to avoid lists for non-explicit VM deployment: []
> 
> 2023-05-19 19:17:01,104 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding clusters to avoid lists for non-explicit VM deployment: []
> 
> 2023-05-19 19:17:01,104 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Adding hosts to avoid lists for non-explicit VM deployment: []
> 
> 2023-05-19 19:17:01,104 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) DeploymentPlanner allocation algorithm: null
> 
> 2023-05-19 19:17:01,104 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Trying to allocate a host and storage pools from dc:2, pod:3,cluster:null, requested cpu: 4800, requested ram: (8.00 GB) 8589934592
> 
> 2023-05-19 19:17:01,104 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Is ROOT volume READY (pool already allocated)?: No
> 
> 2023-05-19 19:17:01,117 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Deploy avoids pods: [], clusters: [1], hosts: [2]
> 
> 2023-05-19 19:17:01,117 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Deploy hosts with priorities {} , hosts have NORMAL priority by default
> 
> 2023-05-19 19:17:01,119 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) This VM has last host_id specified, trying to choose the same host: 2
> 
> 2023-05-19 19:17:01,126 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) The last host of this VM is in avoid set
> 
> 2023-05-19 19:17:01,126 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Cannot choose the last host to deploy this VM
> 
> 2023-05-19 19:17:01,128 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Searching resources only under specified Pod: 3
> 
> 2023-05-19 19:17:01,133 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Listing clusters in order of aggregate capacity, that have (at least one host with) enough CPU and RAM capacity under this Pod: 3
> 
> 2023-05-19 19:17:01,141 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Removing from the clusterId list these clusters from avoid set: [1]
> 
> 2023-05-19 19:17:01,145 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) The clusterId list for the given offering tag: []
> 
> 2023-05-19 19:17:01,145 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) No clusters found after removing disabled clusters and clusters in avoid list, returning.
> 
> 2023-05-19 19:17:01,179 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) VM instance {"id":686,"instanceName":"i-31-686-VM","type":"User","uuid":"d7a8a80a-bef4-4778-9f3c-fc9c06b2cfe1"} state transited from [Starting] to [Stopped] with event [OperationFailed]. VM's original host: Host {"id":2,"name":"co-centro1a-phy-k1-hyp-01.kuasar.cloud","type":"Routing","uuid":"ba87d8ff-21e0-4f20-8bda-b8fa76453a1c"}, new host: null, host before state transition: null
> 
> 2023-05-19 19:17:01,198 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Invocation exception, caused by:com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":686,"instanceName":"i-31-686-VM","type":"User","uuid":"d7a8a80a-bef4-4778-9f3c-fc9c06b2cfe1"}Scope=interfacecom.cloud.dc.DataCenter; id=2
> 
> 2023-05-19 19:17:01,198 INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097 ctx-593b1611) (logid:6edb3ce9) Rethrow exceptioncom.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM instance {"id":686,"instanceName":"i-31-686-VM","type":"User","uuid":"d7a8a80a-bef4-4778-9f3c-fc9c06b2cfe1"}Scope=interfacecom.cloud.dc.DataCenter; id=2
> 
> 2023-05-19 19:17:01,198 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-28:ctx-f26a211c job-18095/job-18097) (logid:6edb3ce9) Done with run of VM work job:com.cloud.vm.VmWorkStart for VM 686, job origin: 18095
> 
> BR,
> 
> Ricardo Pertuz
>