You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Garith Dugmore <ga...@saao.ac.za> on 2014/10/28 13:27:17 UTC
Live migration failed
Hello,
My installation is coming along slowly. I now have ACS 4.4.1 on centos6.4 and two xen 6.2 hosts connected and up and running.
I've installed an instance using an ubuntu 14.04 iso image and when attempting a live migration I get an error:
Job failed due to exception Unable to migrate due to Catch Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate VM(i-2-10-VM) from host(a225ecbc-192d-4e3c-bc86-ec9f7b8acfea) due to Task failed! Task record: uuid: c9d45535-fbeb-0d28-0cc0-1667dabb430f nameLabel: Async.VM.pool_migrate nameDescription: allowedOperations: [] currentOperations: {} created: Tue Oct 28 14:05:58 SAST 2014 finished: Tue Oct 28 14:05:58 SAST 2014 status: failure residentOn: com.xensource.xenapi.Host@a725d84b progress: 1.0 type: <none/> result: errorInfo: [VM_MISSING_PV_DRIVERS, OpaqueRef:59aba8f5-86c4-1c64-2887-5bf69d0a2e00] otherConfig: {} subtaskOf: com.xensource.xenapi.Task@aaf13f6f subtasks: []
After some reading up on VM_MISSING_PV_DRIVERS I installed "xen-tools" on the ubuntu guest and rebooted. After the reboot the migration still was unhappy and gave the same error above.
I also installed the whole xen hypervisor following this guide -> https://help.ubuntu.com/community/XenProposed#Installing_Xen but also no luck after a reboot.
I'm not sure what to try next? Any ideas?
All logs entries before, after and including the error are posted here -> http://pastebin.com/0L08nEFQ
Cheers,
--
Garith Dugmore
South African Astronomical Observatory
and Southern African Large Telescope
Re: Live migration failed
Posted by Garith Dugmore <ga...@saao.ac.za>.
Hi Andrei,
Ok, checked and no iso attached.
The search continues :_)
----- Original Message -----
> From: "Andrei Mikhailovsky" <an...@arhont.com>
> To: users@cloudstack.apache.org
> Sent: Tuesday, October 28, 2014 5:54:16 PM
> Subject: Re: Live migration failed
>
> Garith, I remember I was having issues with live migration while vms had ISO
> disks attached to them. Please check if your vms have any ISOs attached and
> if so, try detaching and doing the migration again.
>
> Andrei
> ----- Original Message -----
>
> > From: "Garith Dugmore" <ga...@saao.ac.za>
> > To: users@cloudstack.apache.org
> > Sent: Tuesday, 28 October, 2014 12:27:17 PM
> > Subject: Live migration failed
>
> > Hello,
>
> > My installation is coming along slowly. I now have ACS 4.4.1 on
> > centos6.4 and two xen 6.2 hosts connected and up and running.
>
> > I've installed an instance using an ubuntu 14.04 iso image and when
> > attempting a live migration I get an error:
>
> > Job failed due to exception Unable to migrate due to Catch Exception
> > com.cloud.utils.exception.CloudRuntimeException: Migration failed
> > due to com.cloud.utils.exception.CloudRuntimeException: Unable to
> > migrate VM(i-2-10-VM) from
> > host(a225ecbc-192d-4e3c-bc86-ec9f7b8acfea) due to Task failed! Task
> > record: uuid: c9d45535-fbeb-0d28-0cc0-1667dabb430f nameLabel:
> > Async.VM.pool_migrate nameDescription: allowedOperations: []
> > currentOperations: {} created: Tue Oct 28 14:05:58 SAST 2014
> > finished: Tue Oct 28 14:05:58 SAST 2014 status: failure residentOn:
> > com.xensource.xenapi.Host@a725d84b progress: 1.0 type: <none/>
> > result: errorInfo: [VM_MISSING_PV_DRIVERS,
> > OpaqueRef:59aba8f5-86c4-1c64-2887-5bf69d0a2e00] otherConfig: {}
> > subtaskOf: com.xensource.xenapi.Task@aaf13f6f subtasks: []
>
> > After some reading up on VM_MISSING_PV_DRIVERS I installed
> > "xen-tools" on the ubuntu guest and rebooted. After the reboot the
> > migration still was unhappy and gave the same error above.
>
> > I also installed the whole xen hypervisor following this guide ->
> > https://help.ubuntu.com/community/XenProposed#Installing_Xen but
> > also no luck after a reboot.
>
> > I'm not sure what to try next? Any ideas?
>
> > All logs entries before, after and including the error are posted
> > here -> http://pastebin.com/0L08nEFQ
>
> > Cheers,
> > --
>
> > Garith Dugmore
> > South African Astronomical Observatory
> > and Southern African Large Telescope
>
>
> !DSPAM:544fbc4f16029783696275!
>
--
Garith Dugmore
South African Astronomical Observatory
and Southern African Large Telescope
Re: Live migration failed
Posted by Andrei Mikhailovsky <an...@arhont.com>.
Garith, I remember I was having issues with live migration while vms had ISO disks attached to them. Please check if your vms have any ISOs attached and if so, try detaching and doing the migration again.
Andrei
----- Original Message -----
> From: "Garith Dugmore" <ga...@saao.ac.za>
> To: users@cloudstack.apache.org
> Sent: Tuesday, 28 October, 2014 12:27:17 PM
> Subject: Live migration failed
> Hello,
> My installation is coming along slowly. I now have ACS 4.4.1 on
> centos6.4 and two xen 6.2 hosts connected and up and running.
> I've installed an instance using an ubuntu 14.04 iso image and when
> attempting a live migration I get an error:
> Job failed due to exception Unable to migrate due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException: Migration failed
> due to com.cloud.utils.exception.CloudRuntimeException: Unable to
> migrate VM(i-2-10-VM) from
> host(a225ecbc-192d-4e3c-bc86-ec9f7b8acfea) due to Task failed! Task
> record: uuid: c9d45535-fbeb-0d28-0cc0-1667dabb430f nameLabel:
> Async.VM.pool_migrate nameDescription: allowedOperations: []
> currentOperations: {} created: Tue Oct 28 14:05:58 SAST 2014
> finished: Tue Oct 28 14:05:58 SAST 2014 status: failure residentOn:
> com.xensource.xenapi.Host@a725d84b progress: 1.0 type: <none/>
> result: errorInfo: [VM_MISSING_PV_DRIVERS,
> OpaqueRef:59aba8f5-86c4-1c64-2887-5bf69d0a2e00] otherConfig: {}
> subtaskOf: com.xensource.xenapi.Task@aaf13f6f subtasks: []
> After some reading up on VM_MISSING_PV_DRIVERS I installed
> "xen-tools" on the ubuntu guest and rebooted. After the reboot the
> migration still was unhappy and gave the same error above.
> I also installed the whole xen hypervisor following this guide ->
> https://help.ubuntu.com/community/XenProposed#Installing_Xen but
> also no luck after a reboot.
> I'm not sure what to try next? Any ideas?
> All logs entries before, after and including the error are posted
> here -> http://pastebin.com/0L08nEFQ
> Cheers,
> --
> Garith Dugmore
> South African Astronomical Observatory
> and Southern African Large Telescope
Re: Live migration failed
Posted by Garith Dugmore <ga...@saao.ac.za>.
Update:
Following this guide for ubuntu 14.04 -> http://softlit.blogspot.com/2014/05/xenserver-62-and-ubuntu-1404-lts.html I used the ubuntu server edition and then of course discovered cloudstack already has an iso called "xs-tools.iso" which has the guest additions. I installed those and I can now live migrate - woohoo!
Something worth noting though is that even though I'm able to live migrate the option "XenServer Tools Version 6.1+" under the instance's "Details" is still showing "No".
Thanks for all the input people.
----- Original Message -----
> From: "Garith Dugmore" <ga...@saao.ac.za>
> To: users@cloudstack.apache.org
> Sent: Wednesday, October 29, 2014 8:58:00 AM
> Subject: Re: Live migration failed
>
> Hi Tim,
>
> I did give this a whirl aswell by doing:
>
> # xe vm-migrate vm=i-2-38-VM remote-master=XXXXXX remote-username=root
> remote-password='XXXXXXXXXXXX'
> Performing a Storage XenMotion migration. Your VM's VDIs will be migrated
> with the VM.
> Selecting remote pool's default SR for migrating VDIs
> Will migrate to remote host: test-20.cape.saao.ac.za, using remote network:
> boom. Here is the VDI mapping:
> VDI c5c7825f-0282-46bc-97f6-d4965e51ae47 -> SR
> 657ea73c-8a0b-8fa4-2c8a-1ad56aa800a7
> You attempted an operation on a VM which requires PV drivers to be installed
> but the drivers were not detected.
> vm: db394ae7-c695-ed9e-8b1b-09b14e59c8cf (i-2-38-VM)
> #
>
> So the same error pretty much:
>
> "You attempted an operation on a VM which requires PV drivers to be installed
> but the drivers were not detected."
>
> At least I know Cloudstack is interpreting the error correctly.
>
> So it sounds like I am missing some PV drivers which is what I'm struggling
> with. Under "Details" of the instance it does actually state "XenServer
> Tools Version 6.1+ No" so I guess thats my problem right there. My search
> for these PV drivers/tools continues..
>
> Cheers,
>
> ----- Original Message -----
> > From: "Tim Mackey" <tm...@gmail.com>
> > To: users@cloudstack.apache.org
> > Sent: Tuesday, October 28, 2014 5:57:29 PM
> > Subject: Re: Live migration failed
> >
> > Before going too far down the CloudStack debugging path, I'd confirm that
> > you can migrate natively. If XenServer won't let you, CloudStack won't
> > force it.
> > On Oct 28, 2014 6:03 AM, "Garith Dugmore" <ga...@saao.ac.za> wrote:
> >
> --
>
> Garith Dugmore
> South African Astronomical Observatory
> and Southern African Large Telescope
>
> !DSPAM:5450903016024808313231!
>
>
>
--
Garith Dugmore
South African Astronomical Observatory
and Southern African Large Telescope
Re: Live migration failed
Posted by Garith Dugmore <ga...@saao.ac.za>.
Hi Tim,
I did give this a whirl aswell by doing:
# xe vm-migrate vm=i-2-38-VM remote-master=XXXXXX remote-username=root remote-password='XXXXXXXXXXXX'
Performing a Storage XenMotion migration. Your VM's VDIs will be migrated with the VM.
Selecting remote pool's default SR for migrating VDIs
Will migrate to remote host: test-20.cape.saao.ac.za, using remote network: boom. Here is the VDI mapping:
VDI c5c7825f-0282-46bc-97f6-d4965e51ae47 -> SR 657ea73c-8a0b-8fa4-2c8a-1ad56aa800a7
You attempted an operation on a VM which requires PV drivers to be installed but the drivers were not detected.
vm: db394ae7-c695-ed9e-8b1b-09b14e59c8cf (i-2-38-VM)
#
So the same error pretty much:
"You attempted an operation on a VM which requires PV drivers to be installed but the drivers were not detected."
At least I know Cloudstack is interpreting the error correctly.
So it sounds like I am missing some PV drivers which is what I'm struggling with. Under "Details" of the instance it does actually state "XenServer Tools Version 6.1+ No" so I guess thats my problem right there. My search for these PV drivers/tools continues..
Cheers,
----- Original Message -----
> From: "Tim Mackey" <tm...@gmail.com>
> To: users@cloudstack.apache.org
> Sent: Tuesday, October 28, 2014 5:57:29 PM
> Subject: Re: Live migration failed
>
> Before going too far down the CloudStack debugging path, I'd confirm that
> you can migrate natively. If XenServer won't let you, CloudStack won't
> force it.
> On Oct 28, 2014 6:03 AM, "Garith Dugmore" <ga...@saao.ac.za> wrote:
>
--
Garith Dugmore
South African Astronomical Observatory
and Southern African Large Telescope
Re: Live migration failed
Posted by Tim Mackey <tm...@gmail.com>.
Before going too far down the CloudStack debugging path, I'd confirm that
you can migrate natively. If XenServer won't let you, CloudStack won't
force it.
On Oct 28, 2014 6:03 AM, "Garith Dugmore" <ga...@saao.ac.za> wrote:
> Hi GopalaKrishnan.S,
>
> Thanks for the speedy reply. Interesting that you mention capacity as I
> had a problem before this error where there wasn't enough RAM on the
> destination host. I've installed more RAM and additionally downsized the
> instance and that specific error has disappeared. To answer your other
> questions - yes, the instance's status is "running" and the 2 xen hosts are
> in the same cluster with shared primary storage (NFS).
>
> I should also mention the ISO "OS Type" is set as "Other Ubuntu (64bit)".
> I'm going to try "Other PV (64bit)" now to see if that makes a difference.
>
> ----- Original Message -----
> > From: "GopalaKrishnan" <go...@assistanz.com>
> > To: users@cloudstack.apache.org
> > Sent: Tuesday, October 28, 2014 2:47:03 PM
> > Subject: Re: Live migration failed
> >
> > Hi Garith,
> >
> > Please make sure you have used the same cluster host to migrate your VM.
> > Also make sure VMs should be running status while migration process. VMs
> > should be used only shared primary storage.
> >
> > Also check your distination host have enough capacity.
> >
> > Thank You.
> > GopalaKrishnan.S
> > Cloud Consultant - Fogpanel
> >
> >
> >
> > ----- Original Message -----
> > From: "Garith Dugmore" <ga...@saao.ac.za>
> > To: <us...@cloudstack.apache.org>
> > Sent: Tuesday, October 28, 2014 5:57 PM
> > Subject: Live migration failed
> >
> >
> > > Hello,
> > >
> > > My installation is coming along slowly. I now have ACS 4.4.1 on
> centos6.4
> > > and two xen 6.2 hosts connected and up and running.
> > >
> > > I've installed an instance using an ubuntu 14.04 iso image and when
> > > attempting a live migration I get an error:
> > >
> > > Job failed due to exception Unable to migrate due to Catch Exception
> > > com.cloud.utils.exception.CloudRuntimeException: Migration failed due
> to
> > > com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
> > > VM(i-2-10-VM) from host(a225ecbc-192d-4e3c-bc86-ec9f7b8acfea) due to
> Task
> > > failed! Task record: uuid: c9d45535-fbeb-0d28-0cc0-1667dabb430f
> nameLabel:
> > > Async.VM.pool_migrate nameDescription: allowedOperations: []
> > > currentOperations: {} created: Tue Oct 28 14:05:58 SAST 2014 finished:
> Tue
> > > Oct 28 14:05:58 SAST 2014 status: failure residentOn:
> > > com.xensource.xenapi.Host@a725d84b progress: 1.0 type: <none/> result:
> > > errorInfo: [VM_MISSING_PV_DRIVERS,
> > > OpaqueRef:59aba8f5-86c4-1c64-2887-5bf69d0a2e00] otherConfig: {}
> subtaskOf:
> > > com.xensource.xenapi.Task@aaf13f6f subtasks: []
> > >
> > > After some reading up on VM_MISSING_PV_DRIVERS I installed "xen-tools"
> on
> > > the ubuntu guest and rebooted. After the reboot the migration still was
> > > unhappy and gave the same error above.
> > >
> > > I also installed the whole xen hypervisor following this guide ->
> > > https://help.ubuntu.com/community/XenProposed#Installing_Xen but also
> no
> > > luck after a reboot.
> > >
> > > I'm not sure what to try next? Any ideas?
> > >
> > > All logs entries before, after and including the error are posted here
> ->
> > > http://pastebin.com/0L08nEFQ
> > >
> > > Cheers,
> > > --
> > >
> > > Garith Dugmore
> > > South African Astronomical Observatory
> > > and Southern African Large Telescope
> >
> >
> > ---
> > This email is free from viruses and malware because avast! Antivirus
> > protection is active.
> > http://www.avast.com
> >
> >
> > !DSPAM:544f8ff116021951155018!
> >
> >
> >
>
> --
>
> Garith Dugmore
> Senior System Administrator
> South African Astronomical Observatory
> and Southern African Large Telescope
>
> SAAO Telephone: +27 21 447 0025
> Direct Line: +27 21 460 6296
> Ext: 4004
> SAAO Website: http://www.saao.ac.za
> SALT Website: http://www.salt.ac.za
> Jabber: gar_I_th@jabber.co.za
> MSN: garithd@hotmail.com
> Skype: garith5512
> Gtalk: garithd@gmail.com
>
>
Re: Live migration failed
Posted by Garith Dugmore <ga...@saao.ac.za>.
Hi GopalaKrishnan.S,
Thanks for the speedy reply. Interesting that you mention capacity as I had a problem before this error where there wasn't enough RAM on the destination host. I've installed more RAM and additionally downsized the instance and that specific error has disappeared. To answer your other questions - yes, the instance's status is "running" and the 2 xen hosts are in the same cluster with shared primary storage (NFS).
I should also mention the ISO "OS Type" is set as "Other Ubuntu (64bit)". I'm going to try "Other PV (64bit)" now to see if that makes a difference.
----- Original Message -----
> From: "GopalaKrishnan" <go...@assistanz.com>
> To: users@cloudstack.apache.org
> Sent: Tuesday, October 28, 2014 2:47:03 PM
> Subject: Re: Live migration failed
>
> Hi Garith,
>
> Please make sure you have used the same cluster host to migrate your VM.
> Also make sure VMs should be running status while migration process. VMs
> should be used only shared primary storage.
>
> Also check your distination host have enough capacity.
>
> Thank You.
> GopalaKrishnan.S
> Cloud Consultant - Fogpanel
>
>
>
> ----- Original Message -----
> From: "Garith Dugmore" <ga...@saao.ac.za>
> To: <us...@cloudstack.apache.org>
> Sent: Tuesday, October 28, 2014 5:57 PM
> Subject: Live migration failed
>
>
> > Hello,
> >
> > My installation is coming along slowly. I now have ACS 4.4.1 on centos6.4
> > and two xen 6.2 hosts connected and up and running.
> >
> > I've installed an instance using an ubuntu 14.04 iso image and when
> > attempting a live migration I get an error:
> >
> > Job failed due to exception Unable to migrate due to Catch Exception
> > com.cloud.utils.exception.CloudRuntimeException: Migration failed due to
> > com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
> > VM(i-2-10-VM) from host(a225ecbc-192d-4e3c-bc86-ec9f7b8acfea) due to Task
> > failed! Task record: uuid: c9d45535-fbeb-0d28-0cc0-1667dabb430f nameLabel:
> > Async.VM.pool_migrate nameDescription: allowedOperations: []
> > currentOperations: {} created: Tue Oct 28 14:05:58 SAST 2014 finished: Tue
> > Oct 28 14:05:58 SAST 2014 status: failure residentOn:
> > com.xensource.xenapi.Host@a725d84b progress: 1.0 type: <none/> result:
> > errorInfo: [VM_MISSING_PV_DRIVERS,
> > OpaqueRef:59aba8f5-86c4-1c64-2887-5bf69d0a2e00] otherConfig: {} subtaskOf:
> > com.xensource.xenapi.Task@aaf13f6f subtasks: []
> >
> > After some reading up on VM_MISSING_PV_DRIVERS I installed "xen-tools" on
> > the ubuntu guest and rebooted. After the reboot the migration still was
> > unhappy and gave the same error above.
> >
> > I also installed the whole xen hypervisor following this guide ->
> > https://help.ubuntu.com/community/XenProposed#Installing_Xen but also no
> > luck after a reboot.
> >
> > I'm not sure what to try next? Any ideas?
> >
> > All logs entries before, after and including the error are posted here ->
> > http://pastebin.com/0L08nEFQ
> >
> > Cheers,
> > --
> >
> > Garith Dugmore
> > South African Astronomical Observatory
> > and Southern African Large Telescope
>
>
> ---
> This email is free from viruses and malware because avast! Antivirus
> protection is active.
> http://www.avast.com
>
>
> !DSPAM:544f8ff116021951155018!
>
>
>
--
Garith Dugmore
Senior System Administrator
South African Astronomical Observatory
and Southern African Large Telescope
SAAO Telephone: +27 21 447 0025
Direct Line: +27 21 460 6296
Ext: 4004
SAAO Website: http://www.saao.ac.za
SALT Website: http://www.salt.ac.za
Jabber: gar_I_th@jabber.co.za
MSN: garithd@hotmail.com
Skype: garith5512
Gtalk: garithd@gmail.com
Re: Live migration failed
Posted by GopalaKrishnan <go...@assistanz.com>.
Hi Garith,
Please make sure you have used the same cluster host to migrate your VM.
Also make sure VMs should be running status while migration process. VMs
should be used only shared primary storage.
Also check your distination host have enough capacity.
Thank You.
GopalaKrishnan.S
Cloud Consultant - Fogpanel
----- Original Message -----
From: "Garith Dugmore" <ga...@saao.ac.za>
To: <us...@cloudstack.apache.org>
Sent: Tuesday, October 28, 2014 5:57 PM
Subject: Live migration failed
> Hello,
>
> My installation is coming along slowly. I now have ACS 4.4.1 on centos6.4
> and two xen 6.2 hosts connected and up and running.
>
> I've installed an instance using an ubuntu 14.04 iso image and when
> attempting a live migration I get an error:
>
> Job failed due to exception Unable to migrate due to Catch Exception
> com.cloud.utils.exception.CloudRuntimeException: Migration failed due to
> com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
> VM(i-2-10-VM) from host(a225ecbc-192d-4e3c-bc86-ec9f7b8acfea) due to Task
> failed! Task record: uuid: c9d45535-fbeb-0d28-0cc0-1667dabb430f nameLabel:
> Async.VM.pool_migrate nameDescription: allowedOperations: []
> currentOperations: {} created: Tue Oct 28 14:05:58 SAST 2014 finished: Tue
> Oct 28 14:05:58 SAST 2014 status: failure residentOn:
> com.xensource.xenapi.Host@a725d84b progress: 1.0 type: <none/> result:
> errorInfo: [VM_MISSING_PV_DRIVERS,
> OpaqueRef:59aba8f5-86c4-1c64-2887-5bf69d0a2e00] otherConfig: {} subtaskOf:
> com.xensource.xenapi.Task@aaf13f6f subtasks: []
>
> After some reading up on VM_MISSING_PV_DRIVERS I installed "xen-tools" on
> the ubuntu guest and rebooted. After the reboot the migration still was
> unhappy and gave the same error above.
>
> I also installed the whole xen hypervisor following this guide ->
> https://help.ubuntu.com/community/XenProposed#Installing_Xen but also no
> luck after a reboot.
>
> I'm not sure what to try next? Any ideas?
>
> All logs entries before, after and including the error are posted here ->
> http://pastebin.com/0L08nEFQ
>
> Cheers,
> --
>
> Garith Dugmore
> South African Astronomical Observatory
> and Southern African Large Telescope
---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com