You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Ricardo Pertuz <ri...@kuasar.co> on 2022/07/08 02:42:42 UTC
endless starting state
Hi team,
After a complete reboot, many vms are stuck in an endless starting state, no error in logs, what else can be checked?
ACS: 4.15.2
Hyp: KVM
Regards,
Re: endless starting state
Posted by Ricardo Pertuz <ri...@kuasar.co>.
Thanks for the hint, no rare logs in any hosts, all vms works well except for this VirtualRouter, any way to do force a VPC cleanup ?
Regards
On 8/07/22, 7:54 AM, "Vash_X@gmx.de" <Va...@gmx.de> wrote:
A look into the log-files of the hosts where the router shall be deployed
to might be helpfull as well. Even if it is not "running" on the host - it
might be that the host has some trouble to start the job.
Regards,
Chris
Am Fr., 8. Juli 2022 um 14:30 Uhr schrieb Ricardo Pertuz <
ricardo.pertuz@kuasar.co>:
> Hi,
>
> Thanks for the quick reply, these would be the answers:
>
> 0. yes, full reboot Management Server and Hosts
>
> 1. MS Servers are UP (we have 2)
> 2. The affected VM is a virtual router, still in starting state, no
> running on any host
> 3. Capacity is less than 10% at the moment
> 4. I am able to launch new instances without error but in other VPC, not
> in the affected one as the VR is in starting state, so not VPC cleanup is
> possible
>
> I have restarted the MS to check if I filter some logs from this VR, none
> so far.
>
> BR,
>
> Ricardo
>
> On 7/07/22, 10:23 PM, "Harikrishna Patnala" <
> Harikrishna.Patnala@shapeblue.com> wrote:
>
> Hi Ricardo,
>
> I assume the complete reboot means the CloudStack management server
> restart/reboot or is it a KVM host reboot ?
>
> Please check for the following if anything can be found.
>
> 1. Check for hosts status from CloudStack UI if it is UP
> 2. Check if these VMs are there and running on the host
> 3. Check if hosts have enough capacity
> 4. Check if new VMs can be deployed as well
>
> You can focus on one of the VM's id and look for the last log related
> to the VM wrt any operation, which may give us some hints.
>
> Thanks,
> Harikrishna
> ________________________________
> From: Ricardo Pertuz <ri...@kuasar.co>
> Sent: Friday, July 8, 2022 8:12 AM
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> Subject: endless starting state
>
> Hi team,
>
> After a complete reboot, many vms are stuck in an endless starting
> state, no error in logs, what else can be checked?
>
> ACS: 4.15.2
> Hyp: KVM
>
> Regards,
>
>
>
>
>
>
Re: endless starting state
Posted by "Vash_X@gmx.de" <Va...@gmx.de>.
A look into the log-files of the hosts where the router shall be deployed
to might be helpfull as well. Even if it is not "running" on the host - it
might be that the host has some trouble to start the job.
Regards,
Chris
Am Fr., 8. Juli 2022 um 14:30 Uhr schrieb Ricardo Pertuz <
ricardo.pertuz@kuasar.co>:
> Hi,
>
> Thanks for the quick reply, these would be the answers:
>
> 0. yes, full reboot Management Server and Hosts
>
> 1. MS Servers are UP (we have 2)
> 2. The affected VM is a virtual router, still in starting state, no
> running on any host
> 3. Capacity is less than 10% at the moment
> 4. I am able to launch new instances without error but in other VPC, not
> in the affected one as the VR is in starting state, so not VPC cleanup is
> possible
>
> I have restarted the MS to check if I filter some logs from this VR, none
> so far.
>
> BR,
>
> Ricardo
>
> On 7/07/22, 10:23 PM, "Harikrishna Patnala" <
> Harikrishna.Patnala@shapeblue.com> wrote:
>
> Hi Ricardo,
>
> I assume the complete reboot means the CloudStack management server
> restart/reboot or is it a KVM host reboot ?
>
> Please check for the following if anything can be found.
>
> 1. Check for hosts status from CloudStack UI if it is UP
> 2. Check if these VMs are there and running on the host
> 3. Check if hosts have enough capacity
> 4. Check if new VMs can be deployed as well
>
> You can focus on one of the VM's id and look for the last log related
> to the VM wrt any operation, which may give us some hints.
>
> Thanks,
> Harikrishna
> ________________________________
> From: Ricardo Pertuz <ri...@kuasar.co>
> Sent: Friday, July 8, 2022 8:12 AM
> To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
> Subject: endless starting state
>
> Hi team,
>
> After a complete reboot, many vms are stuck in an endless starting
> state, no error in logs, what else can be checked?
>
> ACS: 4.15.2
> Hyp: KVM
>
> Regards,
>
>
>
>
>
>
Re: endless starting state
Posted by Ricardo Pertuz <ri...@kuasar.co>.
Some updates on logs
2022-07-08 00:01:31,809 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-11:null) (logid:) vm id: 7532 - time since last state update(120809ms) has passed graceful period
2022-07-08 00:01:31,812 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-11:null) (logid:) VM state report is updated. host: 9, vm id: 7532, power state: PowerReportMissing
2022-07-08 00:01:31,813 INFO [c.c.v.VirtualMachineManagerImpl] (AgentManager-Handler-11:null) (logid:) There is pending job or HA tasks working on the VM. vm id: 7532, postpone power-change report by resetting power-change counters
2022-07-08 00:01:31,816 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (AgentManager-Handler-11:null) (logid:) Done with process of VM state report. host: 9
Nothing else
On 8/07/22, 7:30 AM, "Ricardo Pertuz" <ri...@kuasar.co> wrote:
Hi,
Thanks for the quick reply, these would be the answers:
0. yes, full reboot Management Server and Hosts
1. MS Servers are UP (we have 2)
2. The affected VM is a virtual router, still in starting state, no running on any host
3. Capacity is less than 10% at the moment
4. I am able to launch new instances without error but in other VPC, not in the affected one as the VR is in starting state, so not VPC cleanup is possible
I have restarted the MS to check if I filter some logs from this VR, none so far.
BR,
Ricardo
On 7/07/22, 10:23 PM, "Harikrishna Patnala" <Ha...@shapeblue.com> wrote:
Hi Ricardo,
I assume the complete reboot means the CloudStack management server restart/reboot or is it a KVM host reboot ?
Please check for the following if anything can be found.
1. Check for hosts status from CloudStack UI if it is UP
2. Check if these VMs are there and running on the host
3. Check if hosts have enough capacity
4. Check if new VMs can be deployed as well
You can focus on one of the VM's id and look for the last log related to the VM wrt any operation, which may give us some hints.
Thanks,
Harikrishna
________________________________
From: Ricardo Pertuz <ri...@kuasar.co>
Sent: Friday, July 8, 2022 8:12 AM
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: endless starting state
Hi team,
After a complete reboot, many vms are stuck in an endless starting state, no error in logs, what else can be checked?
ACS: 4.15.2
Hyp: KVM
Regards,
Re: endless starting state
Posted by Ricardo Pertuz <ri...@kuasar.co>.
Hi,
Thanks for the quick reply, these would be the answers:
0. yes, full reboot Management Server and Hosts
1. MS Servers are UP (we have 2)
2. The affected VM is a virtual router, still in starting state, no running on any host
3. Capacity is less than 10% at the moment
4. I am able to launch new instances without error but in other VPC, not in the affected one as the VR is in starting state, so not VPC cleanup is possible
I have restarted the MS to check if I filter some logs from this VR, none so far.
BR,
Ricardo
On 7/07/22, 10:23 PM, "Harikrishna Patnala" <Ha...@shapeblue.com> wrote:
Hi Ricardo,
I assume the complete reboot means the CloudStack management server restart/reboot or is it a KVM host reboot ?
Please check for the following if anything can be found.
1. Check for hosts status from CloudStack UI if it is UP
2. Check if these VMs are there and running on the host
3. Check if hosts have enough capacity
4. Check if new VMs can be deployed as well
You can focus on one of the VM's id and look for the last log related to the VM wrt any operation, which may give us some hints.
Thanks,
Harikrishna
________________________________
From: Ricardo Pertuz <ri...@kuasar.co>
Sent: Friday, July 8, 2022 8:12 AM
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: endless starting state
Hi team,
After a complete reboot, many vms are stuck in an endless starting state, no error in logs, what else can be checked?
ACS: 4.15.2
Hyp: KVM
Regards,
Re: endless starting state
Posted by Harikrishna Patnala <Ha...@shapeblue.com>.
Hi Ricardo,
I assume the complete reboot means the CloudStack management server restart/reboot or is it a KVM host reboot ?
Please check for the following if anything can be found.
1. Check for hosts status from CloudStack UI if it is UP
2. Check if these VMs are there and running on the host
3. Check if hosts have enough capacity
4. Check if new VMs can be deployed as well
You can focus on one of the VM's id and look for the last log related to the VM wrt any operation, which may give us some hints.
Thanks,
Harikrishna
________________________________
From: Ricardo Pertuz <ri...@kuasar.co>
Sent: Friday, July 8, 2022 8:12 AM
To: users@cloudstack.apache.org <us...@cloudstack.apache.org>
Subject: endless starting state
Hi team,
After a complete reboot, many vms are stuck in an endless starting state, no error in logs, what else can be checked?
ACS: 4.15.2
Hyp: KVM
Regards,