You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Andrei Mikhailovsky <an...@arhont.com.INVALID> on 2017/12/21 12:41:22 UTC

kvm/ceph volume snapshots cause other jobs to fail

Hello everyone, 

I have noticed after the recent upgrade to 4.9.3.0 I started having a problem. While the volume snapshots (kvm with ceph primary storage) take place, I am unable to do most things within ACS. For example, stopping / starting / migrating vms simply time out. I have done some testing and this seems to be related to the volume snapshots. If I wait for the snapshot to finish, or if I manually kill the qemu-img process on the host server, the operations resume to normal. VMs operations can work just as before. However, as soon as the snapshot schedule kicks in the next snapshot job, ACS becomes unfunctional again. 

Could you please let me know if there is a workaround for this bug? 

thanks 

Andrei 

Re: kvm/ceph volume snapshots cause other jobs to fail

Posted by Andrei Mikhailovsky <an...@arhont.com.INVALID>.
Hi Rohit,

the issue that I am facing is with every single volume. I have only noticed it on 4.9.3.0 and I don't think it was present in the previous releases. At least I've not seen it before.

It would be challanging to downgrade a live environment at the moment. Perhaps I can later upgrade to 4.10.x when the next point release is out. By the way, any ideal when the next point release of 4.10 is going out?

Thanks

Andrei

----- Original Message -----
> From: "Rohit Yadav" <ro...@shapeblue.com>
> To: "users" <us...@cloudstack.apache.org>
> Sent: Friday, 22 December, 2017 11:11:45
> Subject: Re: kvm/ceph volume snapshots cause other jobs to fail

> Hi Andrei,
> 
> 
> I think it's because snapshots jobs block the job-queue for other items for the
> KVM agent (host), other jobs don't get the opportunity to finish. Are you
> facing this with a particular VM/volume or in general with any VM/host?
> 
> 
> If you think the issue is related to the CloudStack version, you may downgrade
> to 4.9.2.0 and retry. Alternatively, compare against a test 4.9.2.0 and 4.9.3.0
> environment and help report a ticket/bug with more details. Thanks.
> 
> 
> Regards,
> 
> Rohit Yadav
> 
> Software Architect, ShapeBlue
> 
> http://rohityadav.cloud | @rhtyd
> 
> 
>  __?.o/  Apache CloudStack
> (    )#     The best IaaS cloud platform
> (___(_)   https://cloudstack.apache.org
> 
> 
> ________________________________
> From: Andrei Mikhailovsky <an...@arhont.com.INVALID>
> Sent: Thursday, December 21, 2017 6:11:22 PM
> To: users
> Subject: kvm/ceph volume snapshots cause other jobs to fail
> 
> Hello everyone,
> 
> I have noticed after the recent upgrade to 4.9.3.0 I started having a problem.
> While the volume snapshots (kvm with ceph primary storage) take place, I am
> unable to do most things within ACS. For example, stopping / starting /
> migrating vms simply time out. I have done some testing and this seems to be
> related to the volume snapshots. If I wait for the snapshot to finish, or if I
> manually kill the qemu-img process on the host server, the operations resume to
> normal. VMs operations can work just as before. However, as soon as the
> snapshot schedule kicks in the next snapshot job, ACS becomes unfunctional
> again.
> 
> Could you please let me know if there is a workaround for this bug?
> 
> thanks
> 
> Andrei
> 
> rohit.yadav@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue

Re: kvm/ceph volume snapshots cause other jobs to fail

Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Andrei,


I think it's because snapshots jobs block the job-queue for other items for the KVM agent (host), other jobs don't get the opportunity to finish. Are you facing this with a particular VM/volume or in general with any VM/host?


If you think the issue is related to the CloudStack version, you may downgrade to 4.9.2.0 and retry. Alternatively, compare against a test 4.9.2.0 and 4.9.3.0 environment and help report a ticket/bug with more details. Thanks.


Regards,

Rohit Yadav

Software Architect, ShapeBlue

http://rohityadav.cloud | @rhtyd


  __?.o/  Apache CloudStack
 (    )#     The best IaaS cloud platform
(___(_)   https://cloudstack.apache.org


________________________________
From: Andrei Mikhailovsky <an...@arhont.com.INVALID>
Sent: Thursday, December 21, 2017 6:11:22 PM
To: users
Subject: kvm/ceph volume snapshots cause other jobs to fail

Hello everyone,

I have noticed after the recent upgrade to 4.9.3.0 I started having a problem. While the volume snapshots (kvm with ceph primary storage) take place, I am unable to do most things within ACS. For example, stopping / starting / migrating vms simply time out. I have done some testing and this seems to be related to the volume snapshots. If I wait for the snapshot to finish, or if I manually kill the qemu-img process on the host server, the operations resume to normal. VMs operations can work just as before. However, as soon as the snapshot schedule kicks in the next snapshot job, ACS becomes unfunctional again.

Could you please let me know if there is a workaround for this bug?

thanks

Andrei

rohit.yadav@shapeblue.comĀ 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue