You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Dave Cahill <dc...@midokura.com> on 2013/02/08 10:34:18 UTC
QEMU support in CloudStack
Hi,
Recently I encountered two "nested virtualization" use cases which made me
want QEMU hypervisor support in CloudStack. I'm interested to hear if
anyone else is interested in this feature, and any notes on how it should
be implemented.
Here is a good explanation from OpenStack docs [2] on why they support QEMU:
"From the perspective of the Compute service, the QEMU hypervisor is very
similar to the KVM hypervisor. Both are controlled through libvirt, both
support the same feature set, and all virtual machine images that are
compatible with KVM are also compatible with QEMU. The main difference is
that QEMU does not support native virtualization. Consequently, QEMU has
worse performance than KVM and is a poor choice for a production
deployment."
Here are the use cases I encountered:
[Use case: Dev environment]
Wanted to use Vagrant [1] to create a portable multi-node dev
environment; however Vagrant uses VirtualBox, which doesn't support KVM.
Also, devcloud uses VirtualBox and devcloud-kvm uses kvm-within-kvm. I
imagine maintenance of devcloud and devcloud-kvm would be easier if
devcloud-kvm could use VirtualBox too.
Note: Of course, I'm aware of devcloud-kvm as an alternative for this
use case, and I'll be looking into that next.
[Use case: Demo environment]
We may want to spin up a multi-node CloudStack install in Amazon AWS
for demo purposes.
Again, AWS instances don't support KVM, so this is not possible without
QEMU support.
[Implementation ideas]
The management server currently does a check for KVM support ("kvm-ok")
on the host, and refuses to add the host if that fails. I think this check
could be removed, as the agent setup scripts will fail anyway if the user
is trying to setup a certain hypervisor on a machine which doesn't support
it.
Create a new setting in agent.properties like "use_qemu", with a
default of "false". If the person deploying CloudStack agent sets this to
"true", cloud-setup-agent and other setup scripts would ignore lack of KVM
support as long as QEMU support was available.
Lastly, when creating the libvirt XML file for a VM, set hypervisor to
QEMU rather than KVM in the XML file depending on the config setting.
Thanks for reading,
Dave.
[1] http://www.vagrantup.com/
[2]
http://docs.openstack.org/trunk/openstack-compute/install/yum/content/qemu.html
RE: QEMU support in CloudStack
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> Sent: Friday, February 08, 2013 10:57 AM
> To: Edison Su
> Cc: Alex Huang; Sebastien Goasguen; Wido den Hollander; cloudstack-
> dev@incubator.apache.org
> Subject: Re: QEMU support in CloudStack
>
> Hmm, looks like /usr/sbin/virt-what in the system vms is returning 'qemu',
> rather than 'kvm', so cloud-early-config fails to do any setup. Forcing virt-
> what to return kvm in place of qemu gets things going, but perhaps we'd
> change cloud-early-config to do the same things for kvm|qemu.
Yes, system vm shouldn't care about it's qemu or qemu-kvm.
>
> So with those three changes, we have an environment that generally seems
> to work and can be QA'd.
>
> On Fri, Feb 8, 2013 at 11:42 AM, Marcus Sorensen <sh...@gmail.com>
> wrote:
> > Looks like it would work, but the centos 6.3 libvirt isn't new enough.
> > libvirt says 'unknown os type hvm' on centos, even though I've
> > verified that os type of 'hvm' works qemu and without kvm modules on
> > ubuntu 12.04.
> >
> > I upgraded libvirt to a fedora version, and it worked (at least the
> > system vms started coming up, need to wait and see if functions work).
> > Changes made:
> >
> > IsHVMEnabled returns true always
> > hardcoded _hypervisorType to 'qemu' rather than 'kvm'
> >
> > Obviously this was just for the test, we would make these changes some
> > other way.
> >
> > On Fri, Feb 8, 2013 at 11:15 AM, Edison Su <Ed...@citrix.com> wrote:
> >> Wondering, how do you get devcloud-kvm work?
> >
> > by modprobing kvm, kvm_intel in the guest it can run virtual machines.
> > Host system needs to have nested=1 set in it's kvm_intel or kvm_amd
> > kernel module parameters (default on many distributions now).
> >
> >>
> >>> -----Original Message-----
> >>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> >>> Sent: Friday, February 08, 2013 10:02 AM
> >>> To: Alex Huang
> >>> Cc: Sebastien Goasguen; Wido den Hollander; cloudstack-
> >>> dev@incubator.apache.org
> >>> Subject: Re: QEMU support in CloudStack
> >>>
> >>> I'm running a quick sanity test here... just seeing if switching out
> >>> kvm with qemu works, all else the same. Looks like there's a
> >>> setHvsType for the LibvirtVMDef as well, that's hardcoded to 'kvm'.
> >>> That should be easy to adjust as well, assuming everything just runs
> >>> with these changes.
> >>>
> >>> On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com>
> wrote:
> >>> > In that case, why not create two resources with the kvm resource
> >>> extending the qemu resource and do what Marcus suggests here in the
> >>> qemu resource?
> >>> >
> >>> > Effectively then we have an agent for qemu and one for kvm and
> >>> > they each
> >>> can carry their own capabilities.
> >>> >
> >>> > --Alex
> >>> >
> >>> >> -----Original Message-----
> >>> >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> >>> >> Sent: Friday, February 08, 2013 9:41 AM
> >>> >> To: Sebastien Goasguen
> >>> >> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
> >>> >> Subject: Re: QEMU support in CloudStack
> >>> >>
> >>> >> You would in theory have to disable the check in the agent
> >>> >> startup code that looks for the kvm kernel modules, and then
> >>> >> libvirt should just fall back to qemu for everything automatically.
> >>> >>
> >>> >> In LibvirtComputingResource.java, comment out the check for
> >>> >> IsHVMEnabled, and then rmmod any kvm modules, then try to do
> >>> >> stuff with that version of cloudstack.
> >>> >>
> >>> >> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen
> >>> >> <ru...@gmail.com>
> >>> >> wrote:
> >>> >> >
> >>> >> > On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl>
> >>> wrote:
> >>> >> >
> >>> >> >> Hi,
> >>> >> >>
> >>> >> >> On 02/08/2013 10:34 AM, Dave Cahill wrote:
> >>> >> >>> Hi,
> >>> >> >>>
> >>> >> >>> Recently I encountered two "nested virtualization" use cases
> >>> >> >>> which
> >>> >> made me
> >>> >> >>> want QEMU hypervisor support in CloudStack. I'm interested to
> >>> >> >>> hear if anyone else is interested in this feature, and any
> >>> >> >>> notes on how it should be implemented.
> >>> >> >>>
> >>> >> >>> Here is a good explanation from OpenStack docs [2] on why
> >>> >> >>> they
> >>> >> support QEMU:
> >>> >> >>> "From the perspective of the Compute service, the QEMU
> >>> >> >>> hypervisor is
> >>> >> very
> >>> >> >>> similar to the KVM hypervisor. Both are controlled through
> >>> >> >>> libvirt, both support the same feature set, and all virtual
> >>> >> >>> machine images that are compatible with KVM are also
> >>> >> >>> compatible with QEMU. The main
> >>> >> difference is
> >>> >> >>> that QEMU does not support native virtualization.
> >>> >> >>> Consequently, QEMU
> >>> >> has
> >>> >> >>> worse performance than KVM and is a poor choice for a
> >>> >> >>> production deployment."
> >>> >> >>>
> >>> >> >>
> >>> >> >> So, I've been reading into the code and found this on my
> >>> >> >> Ubuntu
> >>> systems.
> >>> >> >>
> >>> >> >> root@stack01:~# ls -l /usr/bin/kvm lrwxrwxrwx 1 root root 18
> >>> >> >> Oct
> >>> >> >> 4 02:44 /usr/bin/kvm -> qemu-system-
> >>> >> x86_64
> >>> >> >> root@stack01:~#
> >>> >> >>
> >>> >> >> Imho Qemu is Qemu and KVM only comes into play when the
> kernel
> >>> >> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
> >>> >> >>
> >>> >> >>> Here are the use cases I encountered:
> >>> >> >>>
> >>> >> >>> [Use case: Dev environment]
> >>> >> >>> Wanted to use Vagrant [1] to create a portable multi-node
> >>> >> >>> dev environment; however Vagrant uses VirtualBox, which
> >>> >> >>> doesn't support
> >>> >> KVM.
> >>> >> >>> Also, devcloud uses VirtualBox and devcloud-kvm uses
> >>> >> >>> kvm-within-
> >>> >> kvm. I
> >>> >> >>> imagine maintenance of devcloud and devcloud-kvm would be
> >>> >> >>> easier if devcloud-kvm could use VirtualBox too.
> >>> >> >>> Note: Of course, I'm aware of devcloud-kvm as an
> >>> >> >>> alternative for this use case, and I'll be looking into that next.
> >>> >> >>>
> >>> >> >>> [Use case: Demo environment]
> >>> >> >>> We may want to spin up a multi-node CloudStack install in
> >>> >> >>> Amazon
> >>> >> AWS
> >>> >> >>> for demo purposes.
> >>> >> >>> Again, AWS instances don't support KVM, so this is not
> >>> >> >>> possible
> >>> >> without
> >>> >> >>> QEMU support.
> >>> >> >>>
> >>> >> >>> [Implementation ideas]
> >>> >> >>> The management server currently does a check for KVM
> >>> >> >>> support
> >>> >> ("kvm-ok")
> >>> >> >>> on the host, and refuses to add the host if that fails. I
> >>> >> >>> think this check could be removed, as the agent setup scripts
> >>> >> >>> will fail anyway if the user is trying to setup a certain
> >>> >> >>> hypervisor on a machine which doesn't
> >>> >> support
> >>> >> >>> it.
> >>> >> >>
> >>> >> >> This way you could do nested virtualization indeed, but it
> >>> >> >> could also hurt
> >>> >> users who have their BIOS set to disabled and could lead to long
> >>> debugging.
> >>> >> >>
> >>> >> >>> Create a new setting in agent.properties like "use_qemu",
> >>> >> >>> with a default of "false". If the person deploying CloudStack
> >>> >> >>> agent sets this to "true", cloud-setup-agent and other setup
> >>> >> >>> scripts would ignore lack of
> >>> >> KVM
> >>> >> >>> support as long as QEMU support was available.
> >>> >> >>
> >>> >> >> cloud-setup-agent generates a agent.properties, so at that
> >>> >> >> point it
> >>> >> doesn't know that the user intents to use the system without KVM
> >>> support.
> >>> >> >>
> >>> >> >>> Lastly, when creating the libvirt XML file for a VM, set
> >>> >> >>> hypervisor to QEMU rather than KVM in the XML file depending
> >>> >> >>> on
> >>> the config setting.
> >>> >> >>>
> >>> >> >>
> >>> >> >> That's not hard coded. The Agent does a getCapabilities() call
> >>> >> >> to libvirt
> >>> >> which returns a list of possible emulators.
> >>> >> >>
> >>> >> >> /usr/bin/kvm is just one of them which is returned and matches
> >>> >> >> the
> >>> >> architecture.
> >>> >> >>
> >>> >> >
> >>> >> > Wido,
> >>> >> >
> >>> >> > So can I add a "KVM" host that would in fact just use qemu ?
> >>> >> > How would I do that ?
> >>> >> >
> >>> >> > -sebastien
> >>> >> >
> >>> >> >
> >>> >> >> Wido
> >>> >> >>
> >>> >> >>> Thanks for reading,
> >>> >> >>> Dave.
> >>> >> >>>
> >>> >> >>> [1] http://www.vagrantup.com/ [2]
> >>> >> >>> http://docs.openstack.org/trunk/openstack-
> >>> >> compute/install/yum/content/qemu.html
> >>> >> >>>
> >>> >> >>
> >>> >> >
Re: QEMU support in CloudStack
Posted by Marcus Sorensen <sh...@gmail.com>.
Yes. If you are using the provided devcloud-kvm image
(https://cwiki.apache.org/confluence/display/CLOUDSTACK/devcloud-kvm),
it is set up to use the kvm modules. The changes I described seem to
allow it to use the qemu emulation that doesn't require hardware
support/kvm modules. But since they're code-based changes (even though
the'yre simple), it seems rather a pain to develop against any given
branch + your local changes to make it run in qemu, and keep things
straight.
On Sun, Feb 10, 2013 at 1:26 PM, Sebastien Goasguen <ru...@gmail.com> wrote:
> Hi Marus, sorry If I am bit confused.
>
> What I am trying to do is run devcloud as a KVM VM (instead of a virtual box instance), but inside devlcoud-KVM used as a cloudstack host, I want to use Qemu.
> Is that what you are describing ?
>
> thanks,
>
>
> -Sebastien
>
> On Feb 8, 2013, at 8:03 PM, Marcus Sorensen <sh...@gmail.com> wrote:
>
>> Well, they're changes to the cloudstack code itself. It's sort of
>> tough to say "here's devcloud-kvm, which works fine if you run it
>> under Linux or VMware Fusion, but if you apply these patches to the
>> code you pull down, you can run it in VirtualBox". I don't mind
>> sharing how it's done, but I don't think it belongs in a how-to. We
>> should probably just decide if we want to make it work in the code.
>> It's probably not of much use for dev anyway unless we go through a
>> bit more testing and ensure that there aren't any edge cases that
>> would cause someone to pull their hair out.
>>
>> On Fri, Feb 8, 2013 at 11:59 AM, Sebastien Goasguen <ru...@gmail.com> wrote:
>>>
>>> On Feb 8, 2013, at 7:57 PM, Marcus Sorensen <sh...@gmail.com> wrote:
>>>
>>>> Hmm, looks like /usr/sbin/virt-what in the system vms is returning
>>>> 'qemu', rather than 'kvm', so cloud-early-config fails to do any
>>>> setup. Forcing virt-what to return kvm in place of qemu gets things
>>>> going, but perhaps we'd change cloud-early-config to do the same
>>>> things for kvm|qemu.
>>>>
>>>> So with those three changes, we have an environment that generally
>>>> seems to work and can be QA'd.
>>>
>>>
>>> Could you post those changes to the devcloud-kvm page ?
>>>
>>> I know someone who is trying to use devcloud-kvm using qemu within devcloud.
>>>
>>> thanks,
>>>
>>> -sebastien
>>>
>>>>
>>>> On Fri, Feb 8, 2013 at 11:42 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>>>>> Looks like it would work, but the centos 6.3 libvirt isn't new enough.
>>>>> libvirt says 'unknown os type hvm' on centos, even though I've
>>>>> verified that os type of 'hvm' works qemu and without kvm modules on
>>>>> ubuntu 12.04.
>>>>>
>>>>> I upgraded libvirt to a fedora version, and it worked (at least the
>>>>> system vms started coming up, need to wait and see if functions work).
>>>>> Changes made:
>>>>>
>>>>> IsHVMEnabled returns true always
>>>>> hardcoded _hypervisorType to 'qemu' rather than 'kvm'
>>>>>
>>>>> Obviously this was just for the test, we would make these changes some
>>>>> other way.
>>>>>
>>>>> On Fri, Feb 8, 2013 at 11:15 AM, Edison Su <Ed...@citrix.com> wrote:
>>>>>> Wondering, how do you get devcloud-kvm work?
>>>>>
>>>>> by modprobing kvm, kvm_intel in the guest it can run virtual machines.
>>>>> Host system needs to have nested=1 set in it's kvm_intel or kvm_amd
>>>>> kernel module parameters (default on many distributions now).
>>>>>
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>>> Sent: Friday, February 08, 2013 10:02 AM
>>>>>>> To: Alex Huang
>>>>>>> Cc: Sebastien Goasguen; Wido den Hollander; cloudstack-
>>>>>>> dev@incubator.apache.org
>>>>>>> Subject: Re: QEMU support in CloudStack
>>>>>>>
>>>>>>> I'm running a quick sanity test here... just seeing if switching out kvm with
>>>>>>> qemu works, all else the same. Looks like there's a setHvsType for the
>>>>>>> LibvirtVMDef as well, that's hardcoded to 'kvm'.
>>>>>>> That should be easy to adjust as well, assuming everything just runs with
>>>>>>> these changes.
>>>>>>>
>>>>>>> On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com> wrote:
>>>>>>>> In that case, why not create two resources with the kvm resource
>>>>>>> extending the qemu resource and do what Marcus suggests here in the
>>>>>>> qemu resource?
>>>>>>>>
>>>>>>>> Effectively then we have an agent for qemu and one for kvm and they each
>>>>>>> can carry their own capabilities.
>>>>>>>>
>>>>>>>> --Alex
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>>>>> Sent: Friday, February 08, 2013 9:41 AM
>>>>>>>>> To: Sebastien Goasguen
>>>>>>>>> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
>>>>>>>>> Subject: Re: QEMU support in CloudStack
>>>>>>>>>
>>>>>>>>> You would in theory have to disable the check in the agent startup
>>>>>>>>> code that looks for the kvm kernel modules, and then libvirt should
>>>>>>>>> just fall back to qemu for everything automatically.
>>>>>>>>>
>>>>>>>>> In LibvirtComputingResource.java, comment out the check for
>>>>>>>>> IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
>>>>>>>>> with that version of cloudstack.
>>>>>>>>>
>>>>>>>>> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen
>>>>>>>>> <ru...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Recently I encountered two "nested virtualization" use cases
>>>>>>>>>>>> which
>>>>>>>>> made me
>>>>>>>>>>>> want QEMU hypervisor support in CloudStack. I'm interested to
>>>>>>>>>>>> hear if anyone else is interested in this feature, and any notes
>>>>>>>>>>>> on how it should be implemented.
>>>>>>>>>>>>
>>>>>>>>>>>> Here is a good explanation from OpenStack docs [2] on why they
>>>>>>>>> support QEMU:
>>>>>>>>>>>> "From the perspective of the Compute service, the QEMU hypervisor
>>>>>>>>>>>> is
>>>>>>>>> very
>>>>>>>>>>>> similar to the KVM hypervisor. Both are controlled through
>>>>>>>>>>>> libvirt, both support the same feature set, and all virtual
>>>>>>>>>>>> machine images that are compatible with KVM are also compatible
>>>>>>>>>>>> with QEMU. The main
>>>>>>>>> difference is
>>>>>>>>>>>> that QEMU does not support native virtualization. Consequently,
>>>>>>>>>>>> QEMU
>>>>>>>>> has
>>>>>>>>>>>> worse performance than KVM and is a poor choice for a production
>>>>>>>>>>>> deployment."
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> So, I've been reading into the code and found this on my Ubuntu
>>>>>>> systems.
>>>>>>>>>>>
>>>>>>>>>>> root@stack01:~# ls -l /usr/bin/kvm lrwxrwxrwx 1 root root 18 Oct
>>>>>>>>>>> 4 02:44 /usr/bin/kvm -> qemu-system-
>>>>>>>>> x86_64
>>>>>>>>>>> root@stack01:~#
>>>>>>>>>>>
>>>>>>>>>>> Imho Qemu is Qemu and KVM only comes into play when the kernel
>>>>>>>>> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>>>>>>>>>>>
>>>>>>>>>>>> Here are the use cases I encountered:
>>>>>>>>>>>>
>>>>>>>>>>>> [Use case: Dev environment]
>>>>>>>>>>>> Wanted to use Vagrant [1] to create a portable multi-node dev
>>>>>>>>>>>> environment; however Vagrant uses VirtualBox, which doesn't
>>>>>>>>>>>> support
>>>>>>>>> KVM.
>>>>>>>>>>>> Also, devcloud uses VirtualBox and devcloud-kvm uses
>>>>>>>>>>>> kvm-within-
>>>>>>>>> kvm. I
>>>>>>>>>>>> imagine maintenance of devcloud and devcloud-kvm would be easier
>>>>>>>>>>>> if devcloud-kvm could use VirtualBox too.
>>>>>>>>>>>> Note: Of course, I'm aware of devcloud-kvm as an alternative
>>>>>>>>>>>> for this use case, and I'll be looking into that next.
>>>>>>>>>>>>
>>>>>>>>>>>> [Use case: Demo environment]
>>>>>>>>>>>> We may want to spin up a multi-node CloudStack install in
>>>>>>>>>>>> Amazon
>>>>>>>>> AWS
>>>>>>>>>>>> for demo purposes.
>>>>>>>>>>>> Again, AWS instances don't support KVM, so this is not
>>>>>>>>>>>> possible
>>>>>>>>> without
>>>>>>>>>>>> QEMU support.
>>>>>>>>>>>>
>>>>>>>>>>>> [Implementation ideas]
>>>>>>>>>>>> The management server currently does a check for KVM support
>>>>>>>>> ("kvm-ok")
>>>>>>>>>>>> on the host, and refuses to add the host if that fails. I think
>>>>>>>>>>>> this check could be removed, as the agent setup scripts will fail
>>>>>>>>>>>> anyway if the user is trying to setup a certain hypervisor on a
>>>>>>>>>>>> machine which doesn't
>>>>>>>>> support
>>>>>>>>>>>> it.
>>>>>>>>>>>
>>>>>>>>>>> This way you could do nested virtualization indeed, but it could
>>>>>>>>>>> also hurt
>>>>>>>>> users who have their BIOS set to disabled and could lead to long
>>>>>>> debugging.
>>>>>>>>>>>
>>>>>>>>>>>> Create a new setting in agent.properties like "use_qemu",
>>>>>>>>>>>> with a default of "false". If the person deploying CloudStack
>>>>>>>>>>>> agent sets this to "true", cloud-setup-agent and other setup
>>>>>>>>>>>> scripts would ignore lack of
>>>>>>>>> KVM
>>>>>>>>>>>> support as long as QEMU support was available.
>>>>>>>>>>>
>>>>>>>>>>> cloud-setup-agent generates a agent.properties, so at that point
>>>>>>>>>>> it
>>>>>>>>> doesn't know that the user intents to use the system without KVM
>>>>>>> support.
>>>>>>>>>>>
>>>>>>>>>>>> Lastly, when creating the libvirt XML file for a VM, set
>>>>>>>>>>>> hypervisor to QEMU rather than KVM in the XML file depending on
>>>>>>> the config setting.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> That's not hard coded. The Agent does a getCapabilities() call to
>>>>>>>>>>> libvirt
>>>>>>>>> which returns a list of possible emulators.
>>>>>>>>>>>
>>>>>>>>>>> /usr/bin/kvm is just one of them which is returned and matches the
>>>>>>>>> architecture.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Wido,
>>>>>>>>>>
>>>>>>>>>> So can I add a "KVM" host that would in fact just use qemu ?
>>>>>>>>>> How would I do that ?
>>>>>>>>>>
>>>>>>>>>> -sebastien
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Wido
>>>>>>>>>>>
>>>>>>>>>>>> Thanks for reading,
>>>>>>>>>>>> Dave.
>>>>>>>>>>>>
>>>>>>>>>>>> [1] http://www.vagrantup.com/
>>>>>>>>>>>> [2]
>>>>>>>>>>>> http://docs.openstack.org/trunk/openstack-
>>>>>>>>> compute/install/yum/content/qemu.html
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>
>
Re: QEMU support in CloudStack
Posted by Sebastien Goasguen <ru...@gmail.com>.
Hi Marus, sorry If I am bit confused.
What I am trying to do is run devcloud as a KVM VM (instead of a virtual box instance), but inside devlcoud-KVM used as a cloudstack host, I want to use Qemu.
Is that what you are describing ?
thanks,
-Sebastien
On Feb 8, 2013, at 8:03 PM, Marcus Sorensen <sh...@gmail.com> wrote:
> Well, they're changes to the cloudstack code itself. It's sort of
> tough to say "here's devcloud-kvm, which works fine if you run it
> under Linux or VMware Fusion, but if you apply these patches to the
> code you pull down, you can run it in VirtualBox". I don't mind
> sharing how it's done, but I don't think it belongs in a how-to. We
> should probably just decide if we want to make it work in the code.
> It's probably not of much use for dev anyway unless we go through a
> bit more testing and ensure that there aren't any edge cases that
> would cause someone to pull their hair out.
>
> On Fri, Feb 8, 2013 at 11:59 AM, Sebastien Goasguen <ru...@gmail.com> wrote:
>>
>> On Feb 8, 2013, at 7:57 PM, Marcus Sorensen <sh...@gmail.com> wrote:
>>
>>> Hmm, looks like /usr/sbin/virt-what in the system vms is returning
>>> 'qemu', rather than 'kvm', so cloud-early-config fails to do any
>>> setup. Forcing virt-what to return kvm in place of qemu gets things
>>> going, but perhaps we'd change cloud-early-config to do the same
>>> things for kvm|qemu.
>>>
>>> So with those three changes, we have an environment that generally
>>> seems to work and can be QA'd.
>>
>>
>> Could you post those changes to the devcloud-kvm page ?
>>
>> I know someone who is trying to use devcloud-kvm using qemu within devcloud.
>>
>> thanks,
>>
>> -sebastien
>>
>>>
>>> On Fri, Feb 8, 2013 at 11:42 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>>>> Looks like it would work, but the centos 6.3 libvirt isn't new enough.
>>>> libvirt says 'unknown os type hvm' on centos, even though I've
>>>> verified that os type of 'hvm' works qemu and without kvm modules on
>>>> ubuntu 12.04.
>>>>
>>>> I upgraded libvirt to a fedora version, and it worked (at least the
>>>> system vms started coming up, need to wait and see if functions work).
>>>> Changes made:
>>>>
>>>> IsHVMEnabled returns true always
>>>> hardcoded _hypervisorType to 'qemu' rather than 'kvm'
>>>>
>>>> Obviously this was just for the test, we would make these changes some
>>>> other way.
>>>>
>>>> On Fri, Feb 8, 2013 at 11:15 AM, Edison Su <Ed...@citrix.com> wrote:
>>>>> Wondering, how do you get devcloud-kvm work?
>>>>
>>>> by modprobing kvm, kvm_intel in the guest it can run virtual machines.
>>>> Host system needs to have nested=1 set in it's kvm_intel or kvm_amd
>>>> kernel module parameters (default on many distributions now).
>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>> Sent: Friday, February 08, 2013 10:02 AM
>>>>>> To: Alex Huang
>>>>>> Cc: Sebastien Goasguen; Wido den Hollander; cloudstack-
>>>>>> dev@incubator.apache.org
>>>>>> Subject: Re: QEMU support in CloudStack
>>>>>>
>>>>>> I'm running a quick sanity test here... just seeing if switching out kvm with
>>>>>> qemu works, all else the same. Looks like there's a setHvsType for the
>>>>>> LibvirtVMDef as well, that's hardcoded to 'kvm'.
>>>>>> That should be easy to adjust as well, assuming everything just runs with
>>>>>> these changes.
>>>>>>
>>>>>> On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com> wrote:
>>>>>>> In that case, why not create two resources with the kvm resource
>>>>>> extending the qemu resource and do what Marcus suggests here in the
>>>>>> qemu resource?
>>>>>>>
>>>>>>> Effectively then we have an agent for qemu and one for kvm and they each
>>>>>> can carry their own capabilities.
>>>>>>>
>>>>>>> --Alex
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>>>> Sent: Friday, February 08, 2013 9:41 AM
>>>>>>>> To: Sebastien Goasguen
>>>>>>>> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
>>>>>>>> Subject: Re: QEMU support in CloudStack
>>>>>>>>
>>>>>>>> You would in theory have to disable the check in the agent startup
>>>>>>>> code that looks for the kvm kernel modules, and then libvirt should
>>>>>>>> just fall back to qemu for everything automatically.
>>>>>>>>
>>>>>>>> In LibvirtComputingResource.java, comment out the check for
>>>>>>>> IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
>>>>>>>> with that version of cloudstack.
>>>>>>>>
>>>>>>>> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen
>>>>>>>> <ru...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Recently I encountered two "nested virtualization" use cases
>>>>>>>>>>> which
>>>>>>>> made me
>>>>>>>>>>> want QEMU hypervisor support in CloudStack. I'm interested to
>>>>>>>>>>> hear if anyone else is interested in this feature, and any notes
>>>>>>>>>>> on how it should be implemented.
>>>>>>>>>>>
>>>>>>>>>>> Here is a good explanation from OpenStack docs [2] on why they
>>>>>>>> support QEMU:
>>>>>>>>>>> "From the perspective of the Compute service, the QEMU hypervisor
>>>>>>>>>>> is
>>>>>>>> very
>>>>>>>>>>> similar to the KVM hypervisor. Both are controlled through
>>>>>>>>>>> libvirt, both support the same feature set, and all virtual
>>>>>>>>>>> machine images that are compatible with KVM are also compatible
>>>>>>>>>>> with QEMU. The main
>>>>>>>> difference is
>>>>>>>>>>> that QEMU does not support native virtualization. Consequently,
>>>>>>>>>>> QEMU
>>>>>>>> has
>>>>>>>>>>> worse performance than KVM and is a poor choice for a production
>>>>>>>>>>> deployment."
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So, I've been reading into the code and found this on my Ubuntu
>>>>>> systems.
>>>>>>>>>>
>>>>>>>>>> root@stack01:~# ls -l /usr/bin/kvm lrwxrwxrwx 1 root root 18 Oct
>>>>>>>>>> 4 02:44 /usr/bin/kvm -> qemu-system-
>>>>>>>> x86_64
>>>>>>>>>> root@stack01:~#
>>>>>>>>>>
>>>>>>>>>> Imho Qemu is Qemu and KVM only comes into play when the kernel
>>>>>>>> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>>>>>>>>>>
>>>>>>>>>>> Here are the use cases I encountered:
>>>>>>>>>>>
>>>>>>>>>>> [Use case: Dev environment]
>>>>>>>>>>> Wanted to use Vagrant [1] to create a portable multi-node dev
>>>>>>>>>>> environment; however Vagrant uses VirtualBox, which doesn't
>>>>>>>>>>> support
>>>>>>>> KVM.
>>>>>>>>>>> Also, devcloud uses VirtualBox and devcloud-kvm uses
>>>>>>>>>>> kvm-within-
>>>>>>>> kvm. I
>>>>>>>>>>> imagine maintenance of devcloud and devcloud-kvm would be easier
>>>>>>>>>>> if devcloud-kvm could use VirtualBox too.
>>>>>>>>>>> Note: Of course, I'm aware of devcloud-kvm as an alternative
>>>>>>>>>>> for this use case, and I'll be looking into that next.
>>>>>>>>>>>
>>>>>>>>>>> [Use case: Demo environment]
>>>>>>>>>>> We may want to spin up a multi-node CloudStack install in
>>>>>>>>>>> Amazon
>>>>>>>> AWS
>>>>>>>>>>> for demo purposes.
>>>>>>>>>>> Again, AWS instances don't support KVM, so this is not
>>>>>>>>>>> possible
>>>>>>>> without
>>>>>>>>>>> QEMU support.
>>>>>>>>>>>
>>>>>>>>>>> [Implementation ideas]
>>>>>>>>>>> The management server currently does a check for KVM support
>>>>>>>> ("kvm-ok")
>>>>>>>>>>> on the host, and refuses to add the host if that fails. I think
>>>>>>>>>>> this check could be removed, as the agent setup scripts will fail
>>>>>>>>>>> anyway if the user is trying to setup a certain hypervisor on a
>>>>>>>>>>> machine which doesn't
>>>>>>>> support
>>>>>>>>>>> it.
>>>>>>>>>>
>>>>>>>>>> This way you could do nested virtualization indeed, but it could
>>>>>>>>>> also hurt
>>>>>>>> users who have their BIOS set to disabled and could lead to long
>>>>>> debugging.
>>>>>>>>>>
>>>>>>>>>>> Create a new setting in agent.properties like "use_qemu",
>>>>>>>>>>> with a default of "false". If the person deploying CloudStack
>>>>>>>>>>> agent sets this to "true", cloud-setup-agent and other setup
>>>>>>>>>>> scripts would ignore lack of
>>>>>>>> KVM
>>>>>>>>>>> support as long as QEMU support was available.
>>>>>>>>>>
>>>>>>>>>> cloud-setup-agent generates a agent.properties, so at that point
>>>>>>>>>> it
>>>>>>>> doesn't know that the user intents to use the system without KVM
>>>>>> support.
>>>>>>>>>>
>>>>>>>>>>> Lastly, when creating the libvirt XML file for a VM, set
>>>>>>>>>>> hypervisor to QEMU rather than KVM in the XML file depending on
>>>>>> the config setting.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> That's not hard coded. The Agent does a getCapabilities() call to
>>>>>>>>>> libvirt
>>>>>>>> which returns a list of possible emulators.
>>>>>>>>>>
>>>>>>>>>> /usr/bin/kvm is just one of them which is returned and matches the
>>>>>>>> architecture.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Wido,
>>>>>>>>>
>>>>>>>>> So can I add a "KVM" host that would in fact just use qemu ?
>>>>>>>>> How would I do that ?
>>>>>>>>>
>>>>>>>>> -sebastien
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Wido
>>>>>>>>>>
>>>>>>>>>>> Thanks for reading,
>>>>>>>>>>> Dave.
>>>>>>>>>>>
>>>>>>>>>>> [1] http://www.vagrantup.com/
>>>>>>>>>>> [2]
>>>>>>>>>>> http://docs.openstack.org/trunk/openstack-
>>>>>>>> compute/install/yum/content/qemu.html
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>
Re: QEMU support in CloudStack
Posted by Marcus Sorensen <sh...@gmail.com>.
I should also mention that in such a dev environment I would be
concerned that the code changes required to run might accidentally get
merged! Or at least screw you up if you switch branches.
On Fri, Feb 8, 2013 at 12:03 PM, Marcus Sorensen <sh...@gmail.com> wrote:
> Well, they're changes to the cloudstack code itself. It's sort of
> tough to say "here's devcloud-kvm, which works fine if you run it
> under Linux or VMware Fusion, but if you apply these patches to the
> code you pull down, you can run it in VirtualBox". I don't mind
> sharing how it's done, but I don't think it belongs in a how-to. We
> should probably just decide if we want to make it work in the code.
> It's probably not of much use for dev anyway unless we go through a
> bit more testing and ensure that there aren't any edge cases that
> would cause someone to pull their hair out.
>
> On Fri, Feb 8, 2013 at 11:59 AM, Sebastien Goasguen <ru...@gmail.com> wrote:
>>
>> On Feb 8, 2013, at 7:57 PM, Marcus Sorensen <sh...@gmail.com> wrote:
>>
>>> Hmm, looks like /usr/sbin/virt-what in the system vms is returning
>>> 'qemu', rather than 'kvm', so cloud-early-config fails to do any
>>> setup. Forcing virt-what to return kvm in place of qemu gets things
>>> going, but perhaps we'd change cloud-early-config to do the same
>>> things for kvm|qemu.
>>>
>>> So with those three changes, we have an environment that generally
>>> seems to work and can be QA'd.
>>
>>
>> Could you post those changes to the devcloud-kvm page ?
>>
>> I know someone who is trying to use devcloud-kvm using qemu within devcloud.
>>
>> thanks,
>>
>> -sebastien
>>
>>>
>>> On Fri, Feb 8, 2013 at 11:42 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>>>> Looks like it would work, but the centos 6.3 libvirt isn't new enough.
>>>> libvirt says 'unknown os type hvm' on centos, even though I've
>>>> verified that os type of 'hvm' works qemu and without kvm modules on
>>>> ubuntu 12.04.
>>>>
>>>> I upgraded libvirt to a fedora version, and it worked (at least the
>>>> system vms started coming up, need to wait and see if functions work).
>>>> Changes made:
>>>>
>>>> IsHVMEnabled returns true always
>>>> hardcoded _hypervisorType to 'qemu' rather than 'kvm'
>>>>
>>>> Obviously this was just for the test, we would make these changes some
>>>> other way.
>>>>
>>>> On Fri, Feb 8, 2013 at 11:15 AM, Edison Su <Ed...@citrix.com> wrote:
>>>>> Wondering, how do you get devcloud-kvm work?
>>>>
>>>> by modprobing kvm, kvm_intel in the guest it can run virtual machines.
>>>> Host system needs to have nested=1 set in it's kvm_intel or kvm_amd
>>>> kernel module parameters (default on many distributions now).
>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>> Sent: Friday, February 08, 2013 10:02 AM
>>>>>> To: Alex Huang
>>>>>> Cc: Sebastien Goasguen; Wido den Hollander; cloudstack-
>>>>>> dev@incubator.apache.org
>>>>>> Subject: Re: QEMU support in CloudStack
>>>>>>
>>>>>> I'm running a quick sanity test here... just seeing if switching out kvm with
>>>>>> qemu works, all else the same. Looks like there's a setHvsType for the
>>>>>> LibvirtVMDef as well, that's hardcoded to 'kvm'.
>>>>>> That should be easy to adjust as well, assuming everything just runs with
>>>>>> these changes.
>>>>>>
>>>>>> On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com> wrote:
>>>>>>> In that case, why not create two resources with the kvm resource
>>>>>> extending the qemu resource and do what Marcus suggests here in the
>>>>>> qemu resource?
>>>>>>>
>>>>>>> Effectively then we have an agent for qemu and one for kvm and they each
>>>>>> can carry their own capabilities.
>>>>>>>
>>>>>>> --Alex
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>>>> Sent: Friday, February 08, 2013 9:41 AM
>>>>>>>> To: Sebastien Goasguen
>>>>>>>> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
>>>>>>>> Subject: Re: QEMU support in CloudStack
>>>>>>>>
>>>>>>>> You would in theory have to disable the check in the agent startup
>>>>>>>> code that looks for the kvm kernel modules, and then libvirt should
>>>>>>>> just fall back to qemu for everything automatically.
>>>>>>>>
>>>>>>>> In LibvirtComputingResource.java, comment out the check for
>>>>>>>> IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
>>>>>>>> with that version of cloudstack.
>>>>>>>>
>>>>>>>> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen
>>>>>>>> <ru...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Recently I encountered two "nested virtualization" use cases
>>>>>>>>>>> which
>>>>>>>> made me
>>>>>>>>>>> want QEMU hypervisor support in CloudStack. I'm interested to
>>>>>>>>>>> hear if anyone else is interested in this feature, and any notes
>>>>>>>>>>> on how it should be implemented.
>>>>>>>>>>>
>>>>>>>>>>> Here is a good explanation from OpenStack docs [2] on why they
>>>>>>>> support QEMU:
>>>>>>>>>>> "From the perspective of the Compute service, the QEMU hypervisor
>>>>>>>>>>> is
>>>>>>>> very
>>>>>>>>>>> similar to the KVM hypervisor. Both are controlled through
>>>>>>>>>>> libvirt, both support the same feature set, and all virtual
>>>>>>>>>>> machine images that are compatible with KVM are also compatible
>>>>>>>>>>> with QEMU. The main
>>>>>>>> difference is
>>>>>>>>>>> that QEMU does not support native virtualization. Consequently,
>>>>>>>>>>> QEMU
>>>>>>>> has
>>>>>>>>>>> worse performance than KVM and is a poor choice for a production
>>>>>>>>>>> deployment."
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So, I've been reading into the code and found this on my Ubuntu
>>>>>> systems.
>>>>>>>>>>
>>>>>>>>>> root@stack01:~# ls -l /usr/bin/kvm lrwxrwxrwx 1 root root 18 Oct
>>>>>>>>>> 4 02:44 /usr/bin/kvm -> qemu-system-
>>>>>>>> x86_64
>>>>>>>>>> root@stack01:~#
>>>>>>>>>>
>>>>>>>>>> Imho Qemu is Qemu and KVM only comes into play when the kernel
>>>>>>>> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>>>>>>>>>>
>>>>>>>>>>> Here are the use cases I encountered:
>>>>>>>>>>>
>>>>>>>>>>> [Use case: Dev environment]
>>>>>>>>>>> Wanted to use Vagrant [1] to create a portable multi-node dev
>>>>>>>>>>> environment; however Vagrant uses VirtualBox, which doesn't
>>>>>>>>>>> support
>>>>>>>> KVM.
>>>>>>>>>>> Also, devcloud uses VirtualBox and devcloud-kvm uses
>>>>>>>>>>> kvm-within-
>>>>>>>> kvm. I
>>>>>>>>>>> imagine maintenance of devcloud and devcloud-kvm would be easier
>>>>>>>>>>> if devcloud-kvm could use VirtualBox too.
>>>>>>>>>>> Note: Of course, I'm aware of devcloud-kvm as an alternative
>>>>>>>>>>> for this use case, and I'll be looking into that next.
>>>>>>>>>>>
>>>>>>>>>>> [Use case: Demo environment]
>>>>>>>>>>> We may want to spin up a multi-node CloudStack install in
>>>>>>>>>>> Amazon
>>>>>>>> AWS
>>>>>>>>>>> for demo purposes.
>>>>>>>>>>> Again, AWS instances don't support KVM, so this is not
>>>>>>>>>>> possible
>>>>>>>> without
>>>>>>>>>>> QEMU support.
>>>>>>>>>>>
>>>>>>>>>>> [Implementation ideas]
>>>>>>>>>>> The management server currently does a check for KVM support
>>>>>>>> ("kvm-ok")
>>>>>>>>>>> on the host, and refuses to add the host if that fails. I think
>>>>>>>>>>> this check could be removed, as the agent setup scripts will fail
>>>>>>>>>>> anyway if the user is trying to setup a certain hypervisor on a
>>>>>>>>>>> machine which doesn't
>>>>>>>> support
>>>>>>>>>>> it.
>>>>>>>>>>
>>>>>>>>>> This way you could do nested virtualization indeed, but it could
>>>>>>>>>> also hurt
>>>>>>>> users who have their BIOS set to disabled and could lead to long
>>>>>> debugging.
>>>>>>>>>>
>>>>>>>>>>> Create a new setting in agent.properties like "use_qemu",
>>>>>>>>>>> with a default of "false". If the person deploying CloudStack
>>>>>>>>>>> agent sets this to "true", cloud-setup-agent and other setup
>>>>>>>>>>> scripts would ignore lack of
>>>>>>>> KVM
>>>>>>>>>>> support as long as QEMU support was available.
>>>>>>>>>>
>>>>>>>>>> cloud-setup-agent generates a agent.properties, so at that point
>>>>>>>>>> it
>>>>>>>> doesn't know that the user intents to use the system without KVM
>>>>>> support.
>>>>>>>>>>
>>>>>>>>>>> Lastly, when creating the libvirt XML file for a VM, set
>>>>>>>>>>> hypervisor to QEMU rather than KVM in the XML file depending on
>>>>>> the config setting.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> That's not hard coded. The Agent does a getCapabilities() call to
>>>>>>>>>> libvirt
>>>>>>>> which returns a list of possible emulators.
>>>>>>>>>>
>>>>>>>>>> /usr/bin/kvm is just one of them which is returned and matches the
>>>>>>>> architecture.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Wido,
>>>>>>>>>
>>>>>>>>> So can I add a "KVM" host that would in fact just use qemu ?
>>>>>>>>> How would I do that ?
>>>>>>>>>
>>>>>>>>> -sebastien
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Wido
>>>>>>>>>>
>>>>>>>>>>> Thanks for reading,
>>>>>>>>>>> Dave.
>>>>>>>>>>>
>>>>>>>>>>> [1] http://www.vagrantup.com/
>>>>>>>>>>> [2]
>>>>>>>>>>> http://docs.openstack.org/trunk/openstack-
>>>>>>>> compute/install/yum/content/qemu.html
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>
Re: QEMU support in CloudStack
Posted by Marcus Sorensen <sh...@gmail.com>.
Well, they're changes to the cloudstack code itself. It's sort of
tough to say "here's devcloud-kvm, which works fine if you run it
under Linux or VMware Fusion, but if you apply these patches to the
code you pull down, you can run it in VirtualBox". I don't mind
sharing how it's done, but I don't think it belongs in a how-to. We
should probably just decide if we want to make it work in the code.
It's probably not of much use for dev anyway unless we go through a
bit more testing and ensure that there aren't any edge cases that
would cause someone to pull their hair out.
On Fri, Feb 8, 2013 at 11:59 AM, Sebastien Goasguen <ru...@gmail.com> wrote:
>
> On Feb 8, 2013, at 7:57 PM, Marcus Sorensen <sh...@gmail.com> wrote:
>
>> Hmm, looks like /usr/sbin/virt-what in the system vms is returning
>> 'qemu', rather than 'kvm', so cloud-early-config fails to do any
>> setup. Forcing virt-what to return kvm in place of qemu gets things
>> going, but perhaps we'd change cloud-early-config to do the same
>> things for kvm|qemu.
>>
>> So with those three changes, we have an environment that generally
>> seems to work and can be QA'd.
>
>
> Could you post those changes to the devcloud-kvm page ?
>
> I know someone who is trying to use devcloud-kvm using qemu within devcloud.
>
> thanks,
>
> -sebastien
>
>>
>> On Fri, Feb 8, 2013 at 11:42 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>>> Looks like it would work, but the centos 6.3 libvirt isn't new enough.
>>> libvirt says 'unknown os type hvm' on centos, even though I've
>>> verified that os type of 'hvm' works qemu and without kvm modules on
>>> ubuntu 12.04.
>>>
>>> I upgraded libvirt to a fedora version, and it worked (at least the
>>> system vms started coming up, need to wait and see if functions work).
>>> Changes made:
>>>
>>> IsHVMEnabled returns true always
>>> hardcoded _hypervisorType to 'qemu' rather than 'kvm'
>>>
>>> Obviously this was just for the test, we would make these changes some
>>> other way.
>>>
>>> On Fri, Feb 8, 2013 at 11:15 AM, Edison Su <Ed...@citrix.com> wrote:
>>>> Wondering, how do you get devcloud-kvm work?
>>>
>>> by modprobing kvm, kvm_intel in the guest it can run virtual machines.
>>> Host system needs to have nested=1 set in it's kvm_intel or kvm_amd
>>> kernel module parameters (default on many distributions now).
>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>> Sent: Friday, February 08, 2013 10:02 AM
>>>>> To: Alex Huang
>>>>> Cc: Sebastien Goasguen; Wido den Hollander; cloudstack-
>>>>> dev@incubator.apache.org
>>>>> Subject: Re: QEMU support in CloudStack
>>>>>
>>>>> I'm running a quick sanity test here... just seeing if switching out kvm with
>>>>> qemu works, all else the same. Looks like there's a setHvsType for the
>>>>> LibvirtVMDef as well, that's hardcoded to 'kvm'.
>>>>> That should be easy to adjust as well, assuming everything just runs with
>>>>> these changes.
>>>>>
>>>>> On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com> wrote:
>>>>>> In that case, why not create two resources with the kvm resource
>>>>> extending the qemu resource and do what Marcus suggests here in the
>>>>> qemu resource?
>>>>>>
>>>>>> Effectively then we have an agent for qemu and one for kvm and they each
>>>>> can carry their own capabilities.
>>>>>>
>>>>>> --Alex
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>>> Sent: Friday, February 08, 2013 9:41 AM
>>>>>>> To: Sebastien Goasguen
>>>>>>> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
>>>>>>> Subject: Re: QEMU support in CloudStack
>>>>>>>
>>>>>>> You would in theory have to disable the check in the agent startup
>>>>>>> code that looks for the kvm kernel modules, and then libvirt should
>>>>>>> just fall back to qemu for everything automatically.
>>>>>>>
>>>>>>> In LibvirtComputingResource.java, comment out the check for
>>>>>>> IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
>>>>>>> with that version of cloudstack.
>>>>>>>
>>>>>>> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen
>>>>>>> <ru...@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl>
>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Recently I encountered two "nested virtualization" use cases
>>>>>>>>>> which
>>>>>>> made me
>>>>>>>>>> want QEMU hypervisor support in CloudStack. I'm interested to
>>>>>>>>>> hear if anyone else is interested in this feature, and any notes
>>>>>>>>>> on how it should be implemented.
>>>>>>>>>>
>>>>>>>>>> Here is a good explanation from OpenStack docs [2] on why they
>>>>>>> support QEMU:
>>>>>>>>>> "From the perspective of the Compute service, the QEMU hypervisor
>>>>>>>>>> is
>>>>>>> very
>>>>>>>>>> similar to the KVM hypervisor. Both are controlled through
>>>>>>>>>> libvirt, both support the same feature set, and all virtual
>>>>>>>>>> machine images that are compatible with KVM are also compatible
>>>>>>>>>> with QEMU. The main
>>>>>>> difference is
>>>>>>>>>> that QEMU does not support native virtualization. Consequently,
>>>>>>>>>> QEMU
>>>>>>> has
>>>>>>>>>> worse performance than KVM and is a poor choice for a production
>>>>>>>>>> deployment."
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So, I've been reading into the code and found this on my Ubuntu
>>>>> systems.
>>>>>>>>>
>>>>>>>>> root@stack01:~# ls -l /usr/bin/kvm lrwxrwxrwx 1 root root 18 Oct
>>>>>>>>> 4 02:44 /usr/bin/kvm -> qemu-system-
>>>>>>> x86_64
>>>>>>>>> root@stack01:~#
>>>>>>>>>
>>>>>>>>> Imho Qemu is Qemu and KVM only comes into play when the kernel
>>>>>>> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>>>>>>>>>
>>>>>>>>>> Here are the use cases I encountered:
>>>>>>>>>>
>>>>>>>>>> [Use case: Dev environment]
>>>>>>>>>> Wanted to use Vagrant [1] to create a portable multi-node dev
>>>>>>>>>> environment; however Vagrant uses VirtualBox, which doesn't
>>>>>>>>>> support
>>>>>>> KVM.
>>>>>>>>>> Also, devcloud uses VirtualBox and devcloud-kvm uses
>>>>>>>>>> kvm-within-
>>>>>>> kvm. I
>>>>>>>>>> imagine maintenance of devcloud and devcloud-kvm would be easier
>>>>>>>>>> if devcloud-kvm could use VirtualBox too.
>>>>>>>>>> Note: Of course, I'm aware of devcloud-kvm as an alternative
>>>>>>>>>> for this use case, and I'll be looking into that next.
>>>>>>>>>>
>>>>>>>>>> [Use case: Demo environment]
>>>>>>>>>> We may want to spin up a multi-node CloudStack install in
>>>>>>>>>> Amazon
>>>>>>> AWS
>>>>>>>>>> for demo purposes.
>>>>>>>>>> Again, AWS instances don't support KVM, so this is not
>>>>>>>>>> possible
>>>>>>> without
>>>>>>>>>> QEMU support.
>>>>>>>>>>
>>>>>>>>>> [Implementation ideas]
>>>>>>>>>> The management server currently does a check for KVM support
>>>>>>> ("kvm-ok")
>>>>>>>>>> on the host, and refuses to add the host if that fails. I think
>>>>>>>>>> this check could be removed, as the agent setup scripts will fail
>>>>>>>>>> anyway if the user is trying to setup a certain hypervisor on a
>>>>>>>>>> machine which doesn't
>>>>>>> support
>>>>>>>>>> it.
>>>>>>>>>
>>>>>>>>> This way you could do nested virtualization indeed, but it could
>>>>>>>>> also hurt
>>>>>>> users who have their BIOS set to disabled and could lead to long
>>>>> debugging.
>>>>>>>>>
>>>>>>>>>> Create a new setting in agent.properties like "use_qemu",
>>>>>>>>>> with a default of "false". If the person deploying CloudStack
>>>>>>>>>> agent sets this to "true", cloud-setup-agent and other setup
>>>>>>>>>> scripts would ignore lack of
>>>>>>> KVM
>>>>>>>>>> support as long as QEMU support was available.
>>>>>>>>>
>>>>>>>>> cloud-setup-agent generates a agent.properties, so at that point
>>>>>>>>> it
>>>>>>> doesn't know that the user intents to use the system without KVM
>>>>> support.
>>>>>>>>>
>>>>>>>>>> Lastly, when creating the libvirt XML file for a VM, set
>>>>>>>>>> hypervisor to QEMU rather than KVM in the XML file depending on
>>>>> the config setting.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> That's not hard coded. The Agent does a getCapabilities() call to
>>>>>>>>> libvirt
>>>>>>> which returns a list of possible emulators.
>>>>>>>>>
>>>>>>>>> /usr/bin/kvm is just one of them which is returned and matches the
>>>>>>> architecture.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Wido,
>>>>>>>>
>>>>>>>> So can I add a "KVM" host that would in fact just use qemu ?
>>>>>>>> How would I do that ?
>>>>>>>>
>>>>>>>> -sebastien
>>>>>>>>
>>>>>>>>
>>>>>>>>> Wido
>>>>>>>>>
>>>>>>>>>> Thanks for reading,
>>>>>>>>>> Dave.
>>>>>>>>>>
>>>>>>>>>> [1] http://www.vagrantup.com/
>>>>>>>>>> [2]
>>>>>>>>>> http://docs.openstack.org/trunk/openstack-
>>>>>>> compute/install/yum/content/qemu.html
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>
Re: QEMU support in CloudStack
Posted by Sebastien Goasguen <ru...@gmail.com>.
On Feb 8, 2013, at 7:57 PM, Marcus Sorensen <sh...@gmail.com> wrote:
> Hmm, looks like /usr/sbin/virt-what in the system vms is returning
> 'qemu', rather than 'kvm', so cloud-early-config fails to do any
> setup. Forcing virt-what to return kvm in place of qemu gets things
> going, but perhaps we'd change cloud-early-config to do the same
> things for kvm|qemu.
>
> So with those three changes, we have an environment that generally
> seems to work and can be QA'd.
Could you post those changes to the devcloud-kvm page ?
I know someone who is trying to use devcloud-kvm using qemu within devcloud.
thanks,
-sebastien
>
> On Fri, Feb 8, 2013 at 11:42 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>> Looks like it would work, but the centos 6.3 libvirt isn't new enough.
>> libvirt says 'unknown os type hvm' on centos, even though I've
>> verified that os type of 'hvm' works qemu and without kvm modules on
>> ubuntu 12.04.
>>
>> I upgraded libvirt to a fedora version, and it worked (at least the
>> system vms started coming up, need to wait and see if functions work).
>> Changes made:
>>
>> IsHVMEnabled returns true always
>> hardcoded _hypervisorType to 'qemu' rather than 'kvm'
>>
>> Obviously this was just for the test, we would make these changes some
>> other way.
>>
>> On Fri, Feb 8, 2013 at 11:15 AM, Edison Su <Ed...@citrix.com> wrote:
>>> Wondering, how do you get devcloud-kvm work?
>>
>> by modprobing kvm, kvm_intel in the guest it can run virtual machines.
>> Host system needs to have nested=1 set in it's kvm_intel or kvm_amd
>> kernel module parameters (default on many distributions now).
>>
>>>
>>>> -----Original Message-----
>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>> Sent: Friday, February 08, 2013 10:02 AM
>>>> To: Alex Huang
>>>> Cc: Sebastien Goasguen; Wido den Hollander; cloudstack-
>>>> dev@incubator.apache.org
>>>> Subject: Re: QEMU support in CloudStack
>>>>
>>>> I'm running a quick sanity test here... just seeing if switching out kvm with
>>>> qemu works, all else the same. Looks like there's a setHvsType for the
>>>> LibvirtVMDef as well, that's hardcoded to 'kvm'.
>>>> That should be easy to adjust as well, assuming everything just runs with
>>>> these changes.
>>>>
>>>> On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com> wrote:
>>>>> In that case, why not create two resources with the kvm resource
>>>> extending the qemu resource and do what Marcus suggests here in the
>>>> qemu resource?
>>>>>
>>>>> Effectively then we have an agent for qemu and one for kvm and they each
>>>> can carry their own capabilities.
>>>>>
>>>>> --Alex
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>> Sent: Friday, February 08, 2013 9:41 AM
>>>>>> To: Sebastien Goasguen
>>>>>> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
>>>>>> Subject: Re: QEMU support in CloudStack
>>>>>>
>>>>>> You would in theory have to disable the check in the agent startup
>>>>>> code that looks for the kvm kernel modules, and then libvirt should
>>>>>> just fall back to qemu for everything automatically.
>>>>>>
>>>>>> In LibvirtComputingResource.java, comment out the check for
>>>>>> IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
>>>>>> with that version of cloudstack.
>>>>>>
>>>>>> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen
>>>>>> <ru...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl>
>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Recently I encountered two "nested virtualization" use cases
>>>>>>>>> which
>>>>>> made me
>>>>>>>>> want QEMU hypervisor support in CloudStack. I'm interested to
>>>>>>>>> hear if anyone else is interested in this feature, and any notes
>>>>>>>>> on how it should be implemented.
>>>>>>>>>
>>>>>>>>> Here is a good explanation from OpenStack docs [2] on why they
>>>>>> support QEMU:
>>>>>>>>> "From the perspective of the Compute service, the QEMU hypervisor
>>>>>>>>> is
>>>>>> very
>>>>>>>>> similar to the KVM hypervisor. Both are controlled through
>>>>>>>>> libvirt, both support the same feature set, and all virtual
>>>>>>>>> machine images that are compatible with KVM are also compatible
>>>>>>>>> with QEMU. The main
>>>>>> difference is
>>>>>>>>> that QEMU does not support native virtualization. Consequently,
>>>>>>>>> QEMU
>>>>>> has
>>>>>>>>> worse performance than KVM and is a poor choice for a production
>>>>>>>>> deployment."
>>>>>>>>>
>>>>>>>>
>>>>>>>> So, I've been reading into the code and found this on my Ubuntu
>>>> systems.
>>>>>>>>
>>>>>>>> root@stack01:~# ls -l /usr/bin/kvm lrwxrwxrwx 1 root root 18 Oct
>>>>>>>> 4 02:44 /usr/bin/kvm -> qemu-system-
>>>>>> x86_64
>>>>>>>> root@stack01:~#
>>>>>>>>
>>>>>>>> Imho Qemu is Qemu and KVM only comes into play when the kernel
>>>>>> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>>>>>>>>
>>>>>>>>> Here are the use cases I encountered:
>>>>>>>>>
>>>>>>>>> [Use case: Dev environment]
>>>>>>>>> Wanted to use Vagrant [1] to create a portable multi-node dev
>>>>>>>>> environment; however Vagrant uses VirtualBox, which doesn't
>>>>>>>>> support
>>>>>> KVM.
>>>>>>>>> Also, devcloud uses VirtualBox and devcloud-kvm uses
>>>>>>>>> kvm-within-
>>>>>> kvm. I
>>>>>>>>> imagine maintenance of devcloud and devcloud-kvm would be easier
>>>>>>>>> if devcloud-kvm could use VirtualBox too.
>>>>>>>>> Note: Of course, I'm aware of devcloud-kvm as an alternative
>>>>>>>>> for this use case, and I'll be looking into that next.
>>>>>>>>>
>>>>>>>>> [Use case: Demo environment]
>>>>>>>>> We may want to spin up a multi-node CloudStack install in
>>>>>>>>> Amazon
>>>>>> AWS
>>>>>>>>> for demo purposes.
>>>>>>>>> Again, AWS instances don't support KVM, so this is not
>>>>>>>>> possible
>>>>>> without
>>>>>>>>> QEMU support.
>>>>>>>>>
>>>>>>>>> [Implementation ideas]
>>>>>>>>> The management server currently does a check for KVM support
>>>>>> ("kvm-ok")
>>>>>>>>> on the host, and refuses to add the host if that fails. I think
>>>>>>>>> this check could be removed, as the agent setup scripts will fail
>>>>>>>>> anyway if the user is trying to setup a certain hypervisor on a
>>>>>>>>> machine which doesn't
>>>>>> support
>>>>>>>>> it.
>>>>>>>>
>>>>>>>> This way you could do nested virtualization indeed, but it could
>>>>>>>> also hurt
>>>>>> users who have their BIOS set to disabled and could lead to long
>>>> debugging.
>>>>>>>>
>>>>>>>>> Create a new setting in agent.properties like "use_qemu",
>>>>>>>>> with a default of "false". If the person deploying CloudStack
>>>>>>>>> agent sets this to "true", cloud-setup-agent and other setup
>>>>>>>>> scripts would ignore lack of
>>>>>> KVM
>>>>>>>>> support as long as QEMU support was available.
>>>>>>>>
>>>>>>>> cloud-setup-agent generates a agent.properties, so at that point
>>>>>>>> it
>>>>>> doesn't know that the user intents to use the system without KVM
>>>> support.
>>>>>>>>
>>>>>>>>> Lastly, when creating the libvirt XML file for a VM, set
>>>>>>>>> hypervisor to QEMU rather than KVM in the XML file depending on
>>>> the config setting.
>>>>>>>>>
>>>>>>>>
>>>>>>>> That's not hard coded. The Agent does a getCapabilities() call to
>>>>>>>> libvirt
>>>>>> which returns a list of possible emulators.
>>>>>>>>
>>>>>>>> /usr/bin/kvm is just one of them which is returned and matches the
>>>>>> architecture.
>>>>>>>>
>>>>>>>
>>>>>>> Wido,
>>>>>>>
>>>>>>> So can I add a "KVM" host that would in fact just use qemu ?
>>>>>>> How would I do that ?
>>>>>>>
>>>>>>> -sebastien
>>>>>>>
>>>>>>>
>>>>>>>> Wido
>>>>>>>>
>>>>>>>>> Thanks for reading,
>>>>>>>>> Dave.
>>>>>>>>>
>>>>>>>>> [1] http://www.vagrantup.com/
>>>>>>>>> [2]
>>>>>>>>> http://docs.openstack.org/trunk/openstack-
>>>>>> compute/install/yum/content/qemu.html
>>>>>>>>>
>>>>>>>>
>>>>>>>
Re: QEMU support in CloudStack
Posted by Marcus Sorensen <sh...@gmail.com>.
Hmm, looks like /usr/sbin/virt-what in the system vms is returning
'qemu', rather than 'kvm', so cloud-early-config fails to do any
setup. Forcing virt-what to return kvm in place of qemu gets things
going, but perhaps we'd change cloud-early-config to do the same
things for kvm|qemu.
So with those three changes, we have an environment that generally
seems to work and can be QA'd.
On Fri, Feb 8, 2013 at 11:42 AM, Marcus Sorensen <sh...@gmail.com> wrote:
> Looks like it would work, but the centos 6.3 libvirt isn't new enough.
> libvirt says 'unknown os type hvm' on centos, even though I've
> verified that os type of 'hvm' works qemu and without kvm modules on
> ubuntu 12.04.
>
> I upgraded libvirt to a fedora version, and it worked (at least the
> system vms started coming up, need to wait and see if functions work).
> Changes made:
>
> IsHVMEnabled returns true always
> hardcoded _hypervisorType to 'qemu' rather than 'kvm'
>
> Obviously this was just for the test, we would make these changes some
> other way.
>
> On Fri, Feb 8, 2013 at 11:15 AM, Edison Su <Ed...@citrix.com> wrote:
>> Wondering, how do you get devcloud-kvm work?
>
> by modprobing kvm, kvm_intel in the guest it can run virtual machines.
> Host system needs to have nested=1 set in it's kvm_intel or kvm_amd
> kernel module parameters (default on many distributions now).
>
>>
>>> -----Original Message-----
>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>> Sent: Friday, February 08, 2013 10:02 AM
>>> To: Alex Huang
>>> Cc: Sebastien Goasguen; Wido den Hollander; cloudstack-
>>> dev@incubator.apache.org
>>> Subject: Re: QEMU support in CloudStack
>>>
>>> I'm running a quick sanity test here... just seeing if switching out kvm with
>>> qemu works, all else the same. Looks like there's a setHvsType for the
>>> LibvirtVMDef as well, that's hardcoded to 'kvm'.
>>> That should be easy to adjust as well, assuming everything just runs with
>>> these changes.
>>>
>>> On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com> wrote:
>>> > In that case, why not create two resources with the kvm resource
>>> extending the qemu resource and do what Marcus suggests here in the
>>> qemu resource?
>>> >
>>> > Effectively then we have an agent for qemu and one for kvm and they each
>>> can carry their own capabilities.
>>> >
>>> > --Alex
>>> >
>>> >> -----Original Message-----
>>> >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>> >> Sent: Friday, February 08, 2013 9:41 AM
>>> >> To: Sebastien Goasguen
>>> >> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
>>> >> Subject: Re: QEMU support in CloudStack
>>> >>
>>> >> You would in theory have to disable the check in the agent startup
>>> >> code that looks for the kvm kernel modules, and then libvirt should
>>> >> just fall back to qemu for everything automatically.
>>> >>
>>> >> In LibvirtComputingResource.java, comment out the check for
>>> >> IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
>>> >> with that version of cloudstack.
>>> >>
>>> >> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen
>>> >> <ru...@gmail.com>
>>> >> wrote:
>>> >> >
>>> >> > On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl>
>>> wrote:
>>> >> >
>>> >> >> Hi,
>>> >> >>
>>> >> >> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>>> >> >>> Hi,
>>> >> >>>
>>> >> >>> Recently I encountered two "nested virtualization" use cases
>>> >> >>> which
>>> >> made me
>>> >> >>> want QEMU hypervisor support in CloudStack. I'm interested to
>>> >> >>> hear if anyone else is interested in this feature, and any notes
>>> >> >>> on how it should be implemented.
>>> >> >>>
>>> >> >>> Here is a good explanation from OpenStack docs [2] on why they
>>> >> support QEMU:
>>> >> >>> "From the perspective of the Compute service, the QEMU hypervisor
>>> >> >>> is
>>> >> very
>>> >> >>> similar to the KVM hypervisor. Both are controlled through
>>> >> >>> libvirt, both support the same feature set, and all virtual
>>> >> >>> machine images that are compatible with KVM are also compatible
>>> >> >>> with QEMU. The main
>>> >> difference is
>>> >> >>> that QEMU does not support native virtualization. Consequently,
>>> >> >>> QEMU
>>> >> has
>>> >> >>> worse performance than KVM and is a poor choice for a production
>>> >> >>> deployment."
>>> >> >>>
>>> >> >>
>>> >> >> So, I've been reading into the code and found this on my Ubuntu
>>> systems.
>>> >> >>
>>> >> >> root@stack01:~# ls -l /usr/bin/kvm lrwxrwxrwx 1 root root 18 Oct
>>> >> >> 4 02:44 /usr/bin/kvm -> qemu-system-
>>> >> x86_64
>>> >> >> root@stack01:~#
>>> >> >>
>>> >> >> Imho Qemu is Qemu and KVM only comes into play when the kernel
>>> >> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>>> >> >>
>>> >> >>> Here are the use cases I encountered:
>>> >> >>>
>>> >> >>> [Use case: Dev environment]
>>> >> >>> Wanted to use Vagrant [1] to create a portable multi-node dev
>>> >> >>> environment; however Vagrant uses VirtualBox, which doesn't
>>> >> >>> support
>>> >> KVM.
>>> >> >>> Also, devcloud uses VirtualBox and devcloud-kvm uses
>>> >> >>> kvm-within-
>>> >> kvm. I
>>> >> >>> imagine maintenance of devcloud and devcloud-kvm would be easier
>>> >> >>> if devcloud-kvm could use VirtualBox too.
>>> >> >>> Note: Of course, I'm aware of devcloud-kvm as an alternative
>>> >> >>> for this use case, and I'll be looking into that next.
>>> >> >>>
>>> >> >>> [Use case: Demo environment]
>>> >> >>> We may want to spin up a multi-node CloudStack install in
>>> >> >>> Amazon
>>> >> AWS
>>> >> >>> for demo purposes.
>>> >> >>> Again, AWS instances don't support KVM, so this is not
>>> >> >>> possible
>>> >> without
>>> >> >>> QEMU support.
>>> >> >>>
>>> >> >>> [Implementation ideas]
>>> >> >>> The management server currently does a check for KVM support
>>> >> ("kvm-ok")
>>> >> >>> on the host, and refuses to add the host if that fails. I think
>>> >> >>> this check could be removed, as the agent setup scripts will fail
>>> >> >>> anyway if the user is trying to setup a certain hypervisor on a
>>> >> >>> machine which doesn't
>>> >> support
>>> >> >>> it.
>>> >> >>
>>> >> >> This way you could do nested virtualization indeed, but it could
>>> >> >> also hurt
>>> >> users who have their BIOS set to disabled and could lead to long
>>> debugging.
>>> >> >>
>>> >> >>> Create a new setting in agent.properties like "use_qemu",
>>> >> >>> with a default of "false". If the person deploying CloudStack
>>> >> >>> agent sets this to "true", cloud-setup-agent and other setup
>>> >> >>> scripts would ignore lack of
>>> >> KVM
>>> >> >>> support as long as QEMU support was available.
>>> >> >>
>>> >> >> cloud-setup-agent generates a agent.properties, so at that point
>>> >> >> it
>>> >> doesn't know that the user intents to use the system without KVM
>>> support.
>>> >> >>
>>> >> >>> Lastly, when creating the libvirt XML file for a VM, set
>>> >> >>> hypervisor to QEMU rather than KVM in the XML file depending on
>>> the config setting.
>>> >> >>>
>>> >> >>
>>> >> >> That's not hard coded. The Agent does a getCapabilities() call to
>>> >> >> libvirt
>>> >> which returns a list of possible emulators.
>>> >> >>
>>> >> >> /usr/bin/kvm is just one of them which is returned and matches the
>>> >> architecture.
>>> >> >>
>>> >> >
>>> >> > Wido,
>>> >> >
>>> >> > So can I add a "KVM" host that would in fact just use qemu ?
>>> >> > How would I do that ?
>>> >> >
>>> >> > -sebastien
>>> >> >
>>> >> >
>>> >> >> Wido
>>> >> >>
>>> >> >>> Thanks for reading,
>>> >> >>> Dave.
>>> >> >>>
>>> >> >>> [1] http://www.vagrantup.com/
>>> >> >>> [2]
>>> >> >>> http://docs.openstack.org/trunk/openstack-
>>> >> compute/install/yum/content/qemu.html
>>> >> >>>
>>> >> >>
>>> >> >
Re: QEMU support in CloudStack
Posted by Marcus Sorensen <sh...@gmail.com>.
Looks like it would work, but the centos 6.3 libvirt isn't new enough.
libvirt says 'unknown os type hvm' on centos, even though I've
verified that os type of 'hvm' works qemu and without kvm modules on
ubuntu 12.04.
I upgraded libvirt to a fedora version, and it worked (at least the
system vms started coming up, need to wait and see if functions work).
Changes made:
IsHVMEnabled returns true always
hardcoded _hypervisorType to 'qemu' rather than 'kvm'
Obviously this was just for the test, we would make these changes some
other way.
On Fri, Feb 8, 2013 at 11:15 AM, Edison Su <Ed...@citrix.com> wrote:
> Wondering, how do you get devcloud-kvm work?
by modprobing kvm, kvm_intel in the guest it can run virtual machines.
Host system needs to have nested=1 set in it's kvm_intel or kvm_amd
kernel module parameters (default on many distributions now).
>
>> -----Original Message-----
>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> Sent: Friday, February 08, 2013 10:02 AM
>> To: Alex Huang
>> Cc: Sebastien Goasguen; Wido den Hollander; cloudstack-
>> dev@incubator.apache.org
>> Subject: Re: QEMU support in CloudStack
>>
>> I'm running a quick sanity test here... just seeing if switching out kvm with
>> qemu works, all else the same. Looks like there's a setHvsType for the
>> LibvirtVMDef as well, that's hardcoded to 'kvm'.
>> That should be easy to adjust as well, assuming everything just runs with
>> these changes.
>>
>> On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com> wrote:
>> > In that case, why not create two resources with the kvm resource
>> extending the qemu resource and do what Marcus suggests here in the
>> qemu resource?
>> >
>> > Effectively then we have an agent for qemu and one for kvm and they each
>> can carry their own capabilities.
>> >
>> > --Alex
>> >
>> >> -----Original Message-----
>> >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> >> Sent: Friday, February 08, 2013 9:41 AM
>> >> To: Sebastien Goasguen
>> >> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
>> >> Subject: Re: QEMU support in CloudStack
>> >>
>> >> You would in theory have to disable the check in the agent startup
>> >> code that looks for the kvm kernel modules, and then libvirt should
>> >> just fall back to qemu for everything automatically.
>> >>
>> >> In LibvirtComputingResource.java, comment out the check for
>> >> IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
>> >> with that version of cloudstack.
>> >>
>> >> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen
>> >> <ru...@gmail.com>
>> >> wrote:
>> >> >
>> >> > On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl>
>> wrote:
>> >> >
>> >> >> Hi,
>> >> >>
>> >> >> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>> >> >>> Hi,
>> >> >>>
>> >> >>> Recently I encountered two "nested virtualization" use cases
>> >> >>> which
>> >> made me
>> >> >>> want QEMU hypervisor support in CloudStack. I'm interested to
>> >> >>> hear if anyone else is interested in this feature, and any notes
>> >> >>> on how it should be implemented.
>> >> >>>
>> >> >>> Here is a good explanation from OpenStack docs [2] on why they
>> >> support QEMU:
>> >> >>> "From the perspective of the Compute service, the QEMU hypervisor
>> >> >>> is
>> >> very
>> >> >>> similar to the KVM hypervisor. Both are controlled through
>> >> >>> libvirt, both support the same feature set, and all virtual
>> >> >>> machine images that are compatible with KVM are also compatible
>> >> >>> with QEMU. The main
>> >> difference is
>> >> >>> that QEMU does not support native virtualization. Consequently,
>> >> >>> QEMU
>> >> has
>> >> >>> worse performance than KVM and is a poor choice for a production
>> >> >>> deployment."
>> >> >>>
>> >> >>
>> >> >> So, I've been reading into the code and found this on my Ubuntu
>> systems.
>> >> >>
>> >> >> root@stack01:~# ls -l /usr/bin/kvm lrwxrwxrwx 1 root root 18 Oct
>> >> >> 4 02:44 /usr/bin/kvm -> qemu-system-
>> >> x86_64
>> >> >> root@stack01:~#
>> >> >>
>> >> >> Imho Qemu is Qemu and KVM only comes into play when the kernel
>> >> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>> >> >>
>> >> >>> Here are the use cases I encountered:
>> >> >>>
>> >> >>> [Use case: Dev environment]
>> >> >>> Wanted to use Vagrant [1] to create a portable multi-node dev
>> >> >>> environment; however Vagrant uses VirtualBox, which doesn't
>> >> >>> support
>> >> KVM.
>> >> >>> Also, devcloud uses VirtualBox and devcloud-kvm uses
>> >> >>> kvm-within-
>> >> kvm. I
>> >> >>> imagine maintenance of devcloud and devcloud-kvm would be easier
>> >> >>> if devcloud-kvm could use VirtualBox too.
>> >> >>> Note: Of course, I'm aware of devcloud-kvm as an alternative
>> >> >>> for this use case, and I'll be looking into that next.
>> >> >>>
>> >> >>> [Use case: Demo environment]
>> >> >>> We may want to spin up a multi-node CloudStack install in
>> >> >>> Amazon
>> >> AWS
>> >> >>> for demo purposes.
>> >> >>> Again, AWS instances don't support KVM, so this is not
>> >> >>> possible
>> >> without
>> >> >>> QEMU support.
>> >> >>>
>> >> >>> [Implementation ideas]
>> >> >>> The management server currently does a check for KVM support
>> >> ("kvm-ok")
>> >> >>> on the host, and refuses to add the host if that fails. I think
>> >> >>> this check could be removed, as the agent setup scripts will fail
>> >> >>> anyway if the user is trying to setup a certain hypervisor on a
>> >> >>> machine which doesn't
>> >> support
>> >> >>> it.
>> >> >>
>> >> >> This way you could do nested virtualization indeed, but it could
>> >> >> also hurt
>> >> users who have their BIOS set to disabled and could lead to long
>> debugging.
>> >> >>
>> >> >>> Create a new setting in agent.properties like "use_qemu",
>> >> >>> with a default of "false". If the person deploying CloudStack
>> >> >>> agent sets this to "true", cloud-setup-agent and other setup
>> >> >>> scripts would ignore lack of
>> >> KVM
>> >> >>> support as long as QEMU support was available.
>> >> >>
>> >> >> cloud-setup-agent generates a agent.properties, so at that point
>> >> >> it
>> >> doesn't know that the user intents to use the system without KVM
>> support.
>> >> >>
>> >> >>> Lastly, when creating the libvirt XML file for a VM, set
>> >> >>> hypervisor to QEMU rather than KVM in the XML file depending on
>> the config setting.
>> >> >>>
>> >> >>
>> >> >> That's not hard coded. The Agent does a getCapabilities() call to
>> >> >> libvirt
>> >> which returns a list of possible emulators.
>> >> >>
>> >> >> /usr/bin/kvm is just one of them which is returned and matches the
>> >> architecture.
>> >> >>
>> >> >
>> >> > Wido,
>> >> >
>> >> > So can I add a "KVM" host that would in fact just use qemu ?
>> >> > How would I do that ?
>> >> >
>> >> > -sebastien
>> >> >
>> >> >
>> >> >> Wido
>> >> >>
>> >> >>> Thanks for reading,
>> >> >>> Dave.
>> >> >>>
>> >> >>> [1] http://www.vagrantup.com/
>> >> >>> [2]
>> >> >>> http://docs.openstack.org/trunk/openstack-
>> >> compute/install/yum/content/qemu.html
>> >> >>>
>> >> >>
>> >> >
RE: QEMU support in CloudStack
Posted by Edison Su <Ed...@citrix.com>.
Wondering, how do you get devcloud-kvm work?
> -----Original Message-----
> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> Sent: Friday, February 08, 2013 10:02 AM
> To: Alex Huang
> Cc: Sebastien Goasguen; Wido den Hollander; cloudstack-
> dev@incubator.apache.org
> Subject: Re: QEMU support in CloudStack
>
> I'm running a quick sanity test here... just seeing if switching out kvm with
> qemu works, all else the same. Looks like there's a setHvsType for the
> LibvirtVMDef as well, that's hardcoded to 'kvm'.
> That should be easy to adjust as well, assuming everything just runs with
> these changes.
>
> On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com> wrote:
> > In that case, why not create two resources with the kvm resource
> extending the qemu resource and do what Marcus suggests here in the
> qemu resource?
> >
> > Effectively then we have an agent for qemu and one for kvm and they each
> can carry their own capabilities.
> >
> > --Alex
> >
> >> -----Original Message-----
> >> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> >> Sent: Friday, February 08, 2013 9:41 AM
> >> To: Sebastien Goasguen
> >> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
> >> Subject: Re: QEMU support in CloudStack
> >>
> >> You would in theory have to disable the check in the agent startup
> >> code that looks for the kvm kernel modules, and then libvirt should
> >> just fall back to qemu for everything automatically.
> >>
> >> In LibvirtComputingResource.java, comment out the check for
> >> IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
> >> with that version of cloudstack.
> >>
> >> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen
> >> <ru...@gmail.com>
> >> wrote:
> >> >
> >> > On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl>
> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> On 02/08/2013 10:34 AM, Dave Cahill wrote:
> >> >>> Hi,
> >> >>>
> >> >>> Recently I encountered two "nested virtualization" use cases
> >> >>> which
> >> made me
> >> >>> want QEMU hypervisor support in CloudStack. I'm interested to
> >> >>> hear if anyone else is interested in this feature, and any notes
> >> >>> on how it should be implemented.
> >> >>>
> >> >>> Here is a good explanation from OpenStack docs [2] on why they
> >> support QEMU:
> >> >>> "From the perspective of the Compute service, the QEMU hypervisor
> >> >>> is
> >> very
> >> >>> similar to the KVM hypervisor. Both are controlled through
> >> >>> libvirt, both support the same feature set, and all virtual
> >> >>> machine images that are compatible with KVM are also compatible
> >> >>> with QEMU. The main
> >> difference is
> >> >>> that QEMU does not support native virtualization. Consequently,
> >> >>> QEMU
> >> has
> >> >>> worse performance than KVM and is a poor choice for a production
> >> >>> deployment."
> >> >>>
> >> >>
> >> >> So, I've been reading into the code and found this on my Ubuntu
> systems.
> >> >>
> >> >> root@stack01:~# ls -l /usr/bin/kvm lrwxrwxrwx 1 root root 18 Oct
> >> >> 4 02:44 /usr/bin/kvm -> qemu-system-
> >> x86_64
> >> >> root@stack01:~#
> >> >>
> >> >> Imho Qemu is Qemu and KVM only comes into play when the kernel
> >> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
> >> >>
> >> >>> Here are the use cases I encountered:
> >> >>>
> >> >>> [Use case: Dev environment]
> >> >>> Wanted to use Vagrant [1] to create a portable multi-node dev
> >> >>> environment; however Vagrant uses VirtualBox, which doesn't
> >> >>> support
> >> KVM.
> >> >>> Also, devcloud uses VirtualBox and devcloud-kvm uses
> >> >>> kvm-within-
> >> kvm. I
> >> >>> imagine maintenance of devcloud and devcloud-kvm would be easier
> >> >>> if devcloud-kvm could use VirtualBox too.
> >> >>> Note: Of course, I'm aware of devcloud-kvm as an alternative
> >> >>> for this use case, and I'll be looking into that next.
> >> >>>
> >> >>> [Use case: Demo environment]
> >> >>> We may want to spin up a multi-node CloudStack install in
> >> >>> Amazon
> >> AWS
> >> >>> for demo purposes.
> >> >>> Again, AWS instances don't support KVM, so this is not
> >> >>> possible
> >> without
> >> >>> QEMU support.
> >> >>>
> >> >>> [Implementation ideas]
> >> >>> The management server currently does a check for KVM support
> >> ("kvm-ok")
> >> >>> on the host, and refuses to add the host if that fails. I think
> >> >>> this check could be removed, as the agent setup scripts will fail
> >> >>> anyway if the user is trying to setup a certain hypervisor on a
> >> >>> machine which doesn't
> >> support
> >> >>> it.
> >> >>
> >> >> This way you could do nested virtualization indeed, but it could
> >> >> also hurt
> >> users who have their BIOS set to disabled and could lead to long
> debugging.
> >> >>
> >> >>> Create a new setting in agent.properties like "use_qemu",
> >> >>> with a default of "false". If the person deploying CloudStack
> >> >>> agent sets this to "true", cloud-setup-agent and other setup
> >> >>> scripts would ignore lack of
> >> KVM
> >> >>> support as long as QEMU support was available.
> >> >>
> >> >> cloud-setup-agent generates a agent.properties, so at that point
> >> >> it
> >> doesn't know that the user intents to use the system without KVM
> support.
> >> >>
> >> >>> Lastly, when creating the libvirt XML file for a VM, set
> >> >>> hypervisor to QEMU rather than KVM in the XML file depending on
> the config setting.
> >> >>>
> >> >>
> >> >> That's not hard coded. The Agent does a getCapabilities() call to
> >> >> libvirt
> >> which returns a list of possible emulators.
> >> >>
> >> >> /usr/bin/kvm is just one of them which is returned and matches the
> >> architecture.
> >> >>
> >> >
> >> > Wido,
> >> >
> >> > So can I add a "KVM" host that would in fact just use qemu ?
> >> > How would I do that ?
> >> >
> >> > -sebastien
> >> >
> >> >
> >> >> Wido
> >> >>
> >> >>> Thanks for reading,
> >> >>> Dave.
> >> >>>
> >> >>> [1] http://www.vagrantup.com/
> >> >>> [2]
> >> >>> http://docs.openstack.org/trunk/openstack-
> >> compute/install/yum/content/qemu.html
> >> >>>
> >> >>
> >> >
Re: QEMU support in CloudStack
Posted by Marcus Sorensen <sh...@gmail.com>.
I'm running a quick sanity test here... just seeing if switching out
kvm with qemu works, all else the same. Looks like there's a
setHvsType for the LibvirtVMDef as well, that's hardcoded to 'kvm'.
That should be easy to adjust as well, assuming everything just runs
with these changes.
On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Al...@citrix.com> wrote:
> In that case, why not create two resources with the kvm resource extending the qemu resource and do what Marcus suggests here in the qemu resource?
>
> Effectively then we have an agent for qemu and one for kvm and they each can carry their own capabilities.
>
> --Alex
>
>> -----Original Message-----
>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> Sent: Friday, February 08, 2013 9:41 AM
>> To: Sebastien Goasguen
>> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
>> Subject: Re: QEMU support in CloudStack
>>
>> You would in theory have to disable the check in the agent startup
>> code that looks for the kvm kernel modules, and then libvirt should
>> just fall back to qemu for everything automatically.
>>
>> In LibvirtComputingResource.java, comment out the check for
>> IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
>> with that version of cloudstack.
>>
>> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen <ru...@gmail.com>
>> wrote:
>> >
>> > On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl> wrote:
>> >
>> >> Hi,
>> >>
>> >> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>> >>> Hi,
>> >>>
>> >>> Recently I encountered two "nested virtualization" use cases which
>> made me
>> >>> want QEMU hypervisor support in CloudStack. I'm interested to hear if
>> >>> anyone else is interested in this feature, and any notes on how it should
>> >>> be implemented.
>> >>>
>> >>> Here is a good explanation from OpenStack docs [2] on why they
>> support QEMU:
>> >>> "From the perspective of the Compute service, the QEMU hypervisor is
>> very
>> >>> similar to the KVM hypervisor. Both are controlled through libvirt, both
>> >>> support the same feature set, and all virtual machine images that are
>> >>> compatible with KVM are also compatible with QEMU. The main
>> difference is
>> >>> that QEMU does not support native virtualization. Consequently, QEMU
>> has
>> >>> worse performance than KVM and is a poor choice for a production
>> >>> deployment."
>> >>>
>> >>
>> >> So, I've been reading into the code and found this on my Ubuntu systems.
>> >>
>> >> root@stack01:~# ls -l /usr/bin/kvm
>> >> lrwxrwxrwx 1 root root 18 Oct 4 02:44 /usr/bin/kvm -> qemu-system-
>> x86_64
>> >> root@stack01:~#
>> >>
>> >> Imho Qemu is Qemu and KVM only comes into play when the kernel
>> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>> >>
>> >>> Here are the use cases I encountered:
>> >>>
>> >>> [Use case: Dev environment]
>> >>> Wanted to use Vagrant [1] to create a portable multi-node dev
>> >>> environment; however Vagrant uses VirtualBox, which doesn't support
>> KVM.
>> >>> Also, devcloud uses VirtualBox and devcloud-kvm uses kvm-within-
>> kvm. I
>> >>> imagine maintenance of devcloud and devcloud-kvm would be easier if
>> >>> devcloud-kvm could use VirtualBox too.
>> >>> Note: Of course, I'm aware of devcloud-kvm as an alternative for this
>> >>> use case, and I'll be looking into that next.
>> >>>
>> >>> [Use case: Demo environment]
>> >>> We may want to spin up a multi-node CloudStack install in Amazon
>> AWS
>> >>> for demo purposes.
>> >>> Again, AWS instances don't support KVM, so this is not possible
>> without
>> >>> QEMU support.
>> >>>
>> >>> [Implementation ideas]
>> >>> The management server currently does a check for KVM support
>> ("kvm-ok")
>> >>> on the host, and refuses to add the host if that fails. I think this check
>> >>> could be removed, as the agent setup scripts will fail anyway if the user
>> >>> is trying to setup a certain hypervisor on a machine which doesn't
>> support
>> >>> it.
>> >>
>> >> This way you could do nested virtualization indeed, but it could also hurt
>> users who have their BIOS set to disabled and could lead to long debugging.
>> >>
>> >>> Create a new setting in agent.properties like "use_qemu", with a
>> >>> default of "false". If the person deploying CloudStack agent sets this to
>> >>> "true", cloud-setup-agent and other setup scripts would ignore lack of
>> KVM
>> >>> support as long as QEMU support was available.
>> >>
>> >> cloud-setup-agent generates a agent.properties, so at that point it
>> doesn't know that the user intents to use the system without KVM support.
>> >>
>> >>> Lastly, when creating the libvirt XML file for a VM, set hypervisor to
>> >>> QEMU rather than KVM in the XML file depending on the config setting.
>> >>>
>> >>
>> >> That's not hard coded. The Agent does a getCapabilities() call to libvirt
>> which returns a list of possible emulators.
>> >>
>> >> /usr/bin/kvm is just one of them which is returned and matches the
>> architecture.
>> >>
>> >
>> > Wido,
>> >
>> > So can I add a "KVM" host that would in fact just use qemu ?
>> > How would I do that ?
>> >
>> > -sebastien
>> >
>> >
>> >> Wido
>> >>
>> >>> Thanks for reading,
>> >>> Dave.
>> >>>
>> >>> [1] http://www.vagrantup.com/
>> >>> [2]
>> >>> http://docs.openstack.org/trunk/openstack-
>> compute/install/yum/content/qemu.html
>> >>>
>> >>
>> >
RE: QEMU support in CloudStack
Posted by Alex Huang <Al...@citrix.com>.
In that case, why not create two resources with the kvm resource extending the qemu resource and do what Marcus suggests here in the qemu resource?
Effectively then we have an agent for qemu and one for kvm and they each can carry their own capabilities.
--Alex
> -----Original Message-----
> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> Sent: Friday, February 08, 2013 9:41 AM
> To: Sebastien Goasguen
> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
> Subject: Re: QEMU support in CloudStack
>
> You would in theory have to disable the check in the agent startup
> code that looks for the kvm kernel modules, and then libvirt should
> just fall back to qemu for everything automatically.
>
> In LibvirtComputingResource.java, comment out the check for
> IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
> with that version of cloudstack.
>
> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen <ru...@gmail.com>
> wrote:
> >
> > On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl> wrote:
> >
> >> Hi,
> >>
> >> On 02/08/2013 10:34 AM, Dave Cahill wrote:
> >>> Hi,
> >>>
> >>> Recently I encountered two "nested virtualization" use cases which
> made me
> >>> want QEMU hypervisor support in CloudStack. I'm interested to hear if
> >>> anyone else is interested in this feature, and any notes on how it should
> >>> be implemented.
> >>>
> >>> Here is a good explanation from OpenStack docs [2] on why they
> support QEMU:
> >>> "From the perspective of the Compute service, the QEMU hypervisor is
> very
> >>> similar to the KVM hypervisor. Both are controlled through libvirt, both
> >>> support the same feature set, and all virtual machine images that are
> >>> compatible with KVM are also compatible with QEMU. The main
> difference is
> >>> that QEMU does not support native virtualization. Consequently, QEMU
> has
> >>> worse performance than KVM and is a poor choice for a production
> >>> deployment."
> >>>
> >>
> >> So, I've been reading into the code and found this on my Ubuntu systems.
> >>
> >> root@stack01:~# ls -l /usr/bin/kvm
> >> lrwxrwxrwx 1 root root 18 Oct 4 02:44 /usr/bin/kvm -> qemu-system-
> x86_64
> >> root@stack01:~#
> >>
> >> Imho Qemu is Qemu and KVM only comes into play when the kernel
> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
> >>
> >>> Here are the use cases I encountered:
> >>>
> >>> [Use case: Dev environment]
> >>> Wanted to use Vagrant [1] to create a portable multi-node dev
> >>> environment; however Vagrant uses VirtualBox, which doesn't support
> KVM.
> >>> Also, devcloud uses VirtualBox and devcloud-kvm uses kvm-within-
> kvm. I
> >>> imagine maintenance of devcloud and devcloud-kvm would be easier if
> >>> devcloud-kvm could use VirtualBox too.
> >>> Note: Of course, I'm aware of devcloud-kvm as an alternative for this
> >>> use case, and I'll be looking into that next.
> >>>
> >>> [Use case: Demo environment]
> >>> We may want to spin up a multi-node CloudStack install in Amazon
> AWS
> >>> for demo purposes.
> >>> Again, AWS instances don't support KVM, so this is not possible
> without
> >>> QEMU support.
> >>>
> >>> [Implementation ideas]
> >>> The management server currently does a check for KVM support
> ("kvm-ok")
> >>> on the host, and refuses to add the host if that fails. I think this check
> >>> could be removed, as the agent setup scripts will fail anyway if the user
> >>> is trying to setup a certain hypervisor on a machine which doesn't
> support
> >>> it.
> >>
> >> This way you could do nested virtualization indeed, but it could also hurt
> users who have their BIOS set to disabled and could lead to long debugging.
> >>
> >>> Create a new setting in agent.properties like "use_qemu", with a
> >>> default of "false". If the person deploying CloudStack agent sets this to
> >>> "true", cloud-setup-agent and other setup scripts would ignore lack of
> KVM
> >>> support as long as QEMU support was available.
> >>
> >> cloud-setup-agent generates a agent.properties, so at that point it
> doesn't know that the user intents to use the system without KVM support.
> >>
> >>> Lastly, when creating the libvirt XML file for a VM, set hypervisor to
> >>> QEMU rather than KVM in the XML file depending on the config setting.
> >>>
> >>
> >> That's not hard coded. The Agent does a getCapabilities() call to libvirt
> which returns a list of possible emulators.
> >>
> >> /usr/bin/kvm is just one of them which is returned and matches the
> architecture.
> >>
> >
> > Wido,
> >
> > So can I add a "KVM" host that would in fact just use qemu ?
> > How would I do that ?
> >
> > -sebastien
> >
> >
> >> Wido
> >>
> >>> Thanks for reading,
> >>> Dave.
> >>>
> >>> [1] http://www.vagrantup.com/
> >>> [2]
> >>> http://docs.openstack.org/trunk/openstack-
> compute/install/yum/content/qemu.html
> >>>
> >>
> >
Re: QEMU support in CloudStack
Posted by Marcus Sorensen <sh...@gmail.com>.
You would in theory have to disable the check in the agent startup
code that looks for the kvm kernel modules, and then libvirt should
just fall back to qemu for everything automatically.
In LibvirtComputingResource.java, comment out the check for
IsHVMEnabled, and then rmmod any kvm modules, then try to do stuff
with that version of cloudstack.
On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen <ru...@gmail.com> wrote:
>
> On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl> wrote:
>
>> Hi,
>>
>> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>>> Hi,
>>>
>>> Recently I encountered two "nested virtualization" use cases which made me
>>> want QEMU hypervisor support in CloudStack. I'm interested to hear if
>>> anyone else is interested in this feature, and any notes on how it should
>>> be implemented.
>>>
>>> Here is a good explanation from OpenStack docs [2] on why they support QEMU:
>>> "From the perspective of the Compute service, the QEMU hypervisor is very
>>> similar to the KVM hypervisor. Both are controlled through libvirt, both
>>> support the same feature set, and all virtual machine images that are
>>> compatible with KVM are also compatible with QEMU. The main difference is
>>> that QEMU does not support native virtualization. Consequently, QEMU has
>>> worse performance than KVM and is a poor choice for a production
>>> deployment."
>>>
>>
>> So, I've been reading into the code and found this on my Ubuntu systems.
>>
>> root@stack01:~# ls -l /usr/bin/kvm
>> lrwxrwxrwx 1 root root 18 Oct 4 02:44 /usr/bin/kvm -> qemu-system-x86_64
>> root@stack01:~#
>>
>> Imho Qemu is Qemu and KVM only comes into play when the kernel module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>>
>>> Here are the use cases I encountered:
>>>
>>> [Use case: Dev environment]
>>> Wanted to use Vagrant [1] to create a portable multi-node dev
>>> environment; however Vagrant uses VirtualBox, which doesn't support KVM.
>>> Also, devcloud uses VirtualBox and devcloud-kvm uses kvm-within-kvm. I
>>> imagine maintenance of devcloud and devcloud-kvm would be easier if
>>> devcloud-kvm could use VirtualBox too.
>>> Note: Of course, I'm aware of devcloud-kvm as an alternative for this
>>> use case, and I'll be looking into that next.
>>>
>>> [Use case: Demo environment]
>>> We may want to spin up a multi-node CloudStack install in Amazon AWS
>>> for demo purposes.
>>> Again, AWS instances don't support KVM, so this is not possible without
>>> QEMU support.
>>>
>>> [Implementation ideas]
>>> The management server currently does a check for KVM support ("kvm-ok")
>>> on the host, and refuses to add the host if that fails. I think this check
>>> could be removed, as the agent setup scripts will fail anyway if the user
>>> is trying to setup a certain hypervisor on a machine which doesn't support
>>> it.
>>
>> This way you could do nested virtualization indeed, but it could also hurt users who have their BIOS set to disabled and could lead to long debugging.
>>
>>> Create a new setting in agent.properties like "use_qemu", with a
>>> default of "false". If the person deploying CloudStack agent sets this to
>>> "true", cloud-setup-agent and other setup scripts would ignore lack of KVM
>>> support as long as QEMU support was available.
>>
>> cloud-setup-agent generates a agent.properties, so at that point it doesn't know that the user intents to use the system without KVM support.
>>
>>> Lastly, when creating the libvirt XML file for a VM, set hypervisor to
>>> QEMU rather than KVM in the XML file depending on the config setting.
>>>
>>
>> That's not hard coded. The Agent does a getCapabilities() call to libvirt which returns a list of possible emulators.
>>
>> /usr/bin/kvm is just one of them which is returned and matches the architecture.
>>
>
> Wido,
>
> So can I add a "KVM" host that would in fact just use qemu ?
> How would I do that ?
>
> -sebastien
>
>
>> Wido
>>
>>> Thanks for reading,
>>> Dave.
>>>
>>> [1] http://www.vagrantup.com/
>>> [2]
>>> http://docs.openstack.org/trunk/openstack-compute/install/yum/content/qemu.html
>>>
>>
>
Re: QEMU support in CloudStack
Posted by Sebastien Goasguen <ru...@gmail.com>.
On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wi...@widodh.nl> wrote:
> Hi,
>
> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>> Hi,
>>
>> Recently I encountered two "nested virtualization" use cases which made me
>> want QEMU hypervisor support in CloudStack. I'm interested to hear if
>> anyone else is interested in this feature, and any notes on how it should
>> be implemented.
>>
>> Here is a good explanation from OpenStack docs [2] on why they support QEMU:
>> "From the perspective of the Compute service, the QEMU hypervisor is very
>> similar to the KVM hypervisor. Both are controlled through libvirt, both
>> support the same feature set, and all virtual machine images that are
>> compatible with KVM are also compatible with QEMU. The main difference is
>> that QEMU does not support native virtualization. Consequently, QEMU has
>> worse performance than KVM and is a poor choice for a production
>> deployment."
>>
>
> So, I've been reading into the code and found this on my Ubuntu systems.
>
> root@stack01:~# ls -l /usr/bin/kvm
> lrwxrwxrwx 1 root root 18 Oct 4 02:44 /usr/bin/kvm -> qemu-system-x86_64
> root@stack01:~#
>
> Imho Qemu is Qemu and KVM only comes into play when the kernel module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>
>> Here are the use cases I encountered:
>>
>> [Use case: Dev environment]
>> Wanted to use Vagrant [1] to create a portable multi-node dev
>> environment; however Vagrant uses VirtualBox, which doesn't support KVM.
>> Also, devcloud uses VirtualBox and devcloud-kvm uses kvm-within-kvm. I
>> imagine maintenance of devcloud and devcloud-kvm would be easier if
>> devcloud-kvm could use VirtualBox too.
>> Note: Of course, I'm aware of devcloud-kvm as an alternative for this
>> use case, and I'll be looking into that next.
>>
>> [Use case: Demo environment]
>> We may want to spin up a multi-node CloudStack install in Amazon AWS
>> for demo purposes.
>> Again, AWS instances don't support KVM, so this is not possible without
>> QEMU support.
>>
>> [Implementation ideas]
>> The management server currently does a check for KVM support ("kvm-ok")
>> on the host, and refuses to add the host if that fails. I think this check
>> could be removed, as the agent setup scripts will fail anyway if the user
>> is trying to setup a certain hypervisor on a machine which doesn't support
>> it.
>
> This way you could do nested virtualization indeed, but it could also hurt users who have their BIOS set to disabled and could lead to long debugging.
>
>> Create a new setting in agent.properties like "use_qemu", with a
>> default of "false". If the person deploying CloudStack agent sets this to
>> "true", cloud-setup-agent and other setup scripts would ignore lack of KVM
>> support as long as QEMU support was available.
>
> cloud-setup-agent generates a agent.properties, so at that point it doesn't know that the user intents to use the system without KVM support.
>
>> Lastly, when creating the libvirt XML file for a VM, set hypervisor to
>> QEMU rather than KVM in the XML file depending on the config setting.
>>
>
> That's not hard coded. The Agent does a getCapabilities() call to libvirt which returns a list of possible emulators.
>
> /usr/bin/kvm is just one of them which is returned and matches the architecture.
>
Wido,
So can I add a "KVM" host that would in fact just use qemu ?
How would I do that ?
-sebastien
> Wido
>
>> Thanks for reading,
>> Dave.
>>
>> [1] http://www.vagrantup.com/
>> [2]
>> http://docs.openstack.org/trunk/openstack-compute/install/yum/content/qemu.html
>>
>
Re: QEMU support in CloudStack
Posted by Wido den Hollander <wi...@widodh.nl>.
Hi,
On 02/08/2013 10:34 AM, Dave Cahill wrote:
> Hi,
>
> Recently I encountered two "nested virtualization" use cases which made me
> want QEMU hypervisor support in CloudStack. I'm interested to hear if
> anyone else is interested in this feature, and any notes on how it should
> be implemented.
>
> Here is a good explanation from OpenStack docs [2] on why they support QEMU:
> "From the perspective of the Compute service, the QEMU hypervisor is very
> similar to the KVM hypervisor. Both are controlled through libvirt, both
> support the same feature set, and all virtual machine images that are
> compatible with KVM are also compatible with QEMU. The main difference is
> that QEMU does not support native virtualization. Consequently, QEMU has
> worse performance than KVM and is a poor choice for a production
> deployment."
>
So, I've been reading into the code and found this on my Ubuntu systems.
root@stack01:~# ls -l /usr/bin/kvm
lrwxrwxrwx 1 root root 18 Oct 4 02:44 /usr/bin/kvm -> qemu-system-x86_64
root@stack01:~#
Imho Qemu is Qemu and KVM only comes into play when the kernel module
'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
> Here are the use cases I encountered:
>
> [Use case: Dev environment]
> Wanted to use Vagrant [1] to create a portable multi-node dev
> environment; however Vagrant uses VirtualBox, which doesn't support KVM.
> Also, devcloud uses VirtualBox and devcloud-kvm uses kvm-within-kvm. I
> imagine maintenance of devcloud and devcloud-kvm would be easier if
> devcloud-kvm could use VirtualBox too.
> Note: Of course, I'm aware of devcloud-kvm as an alternative for this
> use case, and I'll be looking into that next.
>
> [Use case: Demo environment]
> We may want to spin up a multi-node CloudStack install in Amazon AWS
> for demo purposes.
> Again, AWS instances don't support KVM, so this is not possible without
> QEMU support.
>
> [Implementation ideas]
> The management server currently does a check for KVM support ("kvm-ok")
> on the host, and refuses to add the host if that fails. I think this check
> could be removed, as the agent setup scripts will fail anyway if the user
> is trying to setup a certain hypervisor on a machine which doesn't support
> it.
This way you could do nested virtualization indeed, but it could also
hurt users who have their BIOS set to disabled and could lead to long
debugging.
> Create a new setting in agent.properties like "use_qemu", with a
> default of "false". If the person deploying CloudStack agent sets this to
> "true", cloud-setup-agent and other setup scripts would ignore lack of KVM
> support as long as QEMU support was available.
cloud-setup-agent generates a agent.properties, so at that point it
doesn't know that the user intents to use the system without KVM support.
> Lastly, when creating the libvirt XML file for a VM, set hypervisor to
> QEMU rather than KVM in the XML file depending on the config setting.
>
That's not hard coded. The Agent does a getCapabilities() call to
libvirt which returns a list of possible emulators.
/usr/bin/kvm is just one of them which is returned and matches the
architecture.
Wido
> Thanks for reading,
> Dave.
>
> [1] http://www.vagrantup.com/
> [2]
> http://docs.openstack.org/trunk/openstack-compute/install/yum/content/qemu.html
>
Re: QEMU support in CloudStack
Posted by Marcus Sorensen <sh...@gmail.com>.
On Feb 8, 2013 2:34 AM, "Dave Cahill" <dc...@midokura.com> wrote:
>
> Hi,
>
> Recently I encountered two "nested virtualization" use cases which made me
> want QEMU hypervisor support in CloudStack. I'm interested to hear if
> anyone else is interested in this feature, and any notes on how it should
> be implemented.
>
> Here is a good explanation from OpenStack docs [2] on why they support
QEMU:
> "From the perspective of the Compute service, the QEMU hypervisor is very
> similar to the KVM hypervisor. Both are controlled through libvirt, both
> support the same feature set, and all virtual machine images that are
> compatible with KVM are also compatible with QEMU. The main difference is
> that QEMU does not support native virtualization. Consequently, QEMU has
> worse performance than KVM and is a poor choice for a production
> deployment."
>
> Here are the use cases I encountered:
>
> [Use case: Dev environment]
> Wanted to use Vagrant [1] to create a portable multi-node dev
> environment; however Vagrant uses VirtualBox, which doesn't support KVM.
> Also, devcloud uses VirtualBox and devcloud-kvm uses kvm-within-kvm. I
> imagine maintenance of devcloud and devcloud-kvm would be easier if
> devcloud-kvm could use VirtualBox too.
As a side note, I actually went the other way. If you look at the devcloud
docs, you'll see that you can now run devcloud and devcloud-kvm together,
not in virtual box, but under qemu-kvm.
> Note: Of course, I'm aware of devcloud-kvm as an alternative for this
> use case, and I'll be looking into that next.
>
> [Use case: Demo environment]
> We may want to spin up a multi-node CloudStack install in Amazon AWS
> for demo purposes.
> Again, AWS instances don't support KVM, so this is not possible
without
> QEMU support.
>
> [Implementation ideas]
> The management server currently does a check for KVM support
("kvm-ok")
> on the host, and refuses to add the host if that fails. I think this check
> could be removed, as the agent setup scripts will fail anyway if the user
> is trying to setup a certain hypervisor on a machine which doesn't support
> it.
> Create a new setting in agent.properties like "use_qemu", with a
> default of "false". If the person deploying CloudStack agent sets this to
> "true", cloud-setup-agent and other setup scripts would ignore lack of KVM
> support as long as QEMU support was available.
> Lastly, when creating the libvirt XML file for a VM, set hypervisor to
> QEMU rather than KVM in the XML file depending on the config setting.
>
> Thanks for reading,
> Dave.
>
> [1] http://www.vagrantup.com/
> [2]
>
http://docs.openstack.org/trunk/openstack-compute/install/yum/content/qemu.html
Re: QEMU support in CloudStack
Posted by Marcus Sorensen <sh...@gmail.com>.
http://wiki.libvirt.org/page/Virtio#Requirements
It doesn't look like KVM is required for virtio (require for
hotplugging/systemvm functions), although I wonder what is meant by
"recent qemu" there, if there's no other blocker then it may be
possible to skip the check for kvm modules in some specific scenario,
maybe a global config that propagates to the agent.properties when a
host joins or something. Then as Wido says, the host should just fall
back to qemu. It should be possible. I wouldn't want any chance of a
user accidentally running in that mode though.
On Fri, Feb 8, 2013 at 9:05 AM, Chip Childers <ch...@sungard.com> wrote:
> On Fri, Feb 08, 2013 at 10:47:29AM +0000, Paul Angus wrote:
>> Hi Dave,
>>
>> Your suggestion ties in nicely which a proposal that I have been mulling over; and that is further abstraction of the hypervisor, by CloudStack becoming more ‘toolset agonistic’ to use Xen parlance, rather than hypervisor agnostic?
>>
>> It seems quite an elegant solution to expanding hypervisor support (which easy for me to say as I don't know how to do it)…
>>
>> The current toolsets / interfaces become something like:
>>
>> OVM Manager (OVM)
>> vCenter (ESXi)
>> XAPI (XenServer, XCP, Xen+XAPI)
>> Libvirt (KVM, Xen+Libvirt, QEMU)
>> WMI/PowerShell (Hyper-V)
>>
>> + a bit of logic when copying scripts across to put them in the correct location for any given hypervisor/distribution.
>
> Paul,
>
> I'm not sure I quite understand what you are suggesting. Can you
> elaborate?
>
> -chip
RE: QEMU support in CloudStack
Posted by Paul Angus <pa...@shapeblue.com>.
Sure. This came about while experimenting with various configurations of the Xen hypervisor.
I'll start my making the grand statement, then try to expain my thinking:
'By supporting the Libvirt and XAPI APIs rather than 'KVM' and 'XenServer', CloudStack would be able to (potentially) orchestrate many more hypervisors.'
I'll summarise my limited knowledge on the subject...
'Xen' is in-fact the Xen hypervisor with a 'toolset' on top (xl, xe, libvirt or xapi)
Xen + the XAPI toolset being the basis for XenServer and XCP
Libvirt (like XAPI) is an API layer that controls the underlying hypervisor. (so far so good)....
I'd created a Ubuntu Xen box with the Libvirt API on top thinking that it would act like a 'KVM' host as CloudStack talks to KVM via the Libvirt API - and failed miserably.
But this got me thinking and I looked closer into KVM, Libvirt, Xen and XAPI
Libvirt is (like XAPI) an API layer that controls the underlying hypervisor which could be any of:
Xen
QEMU / KVM
Linux Container
OpenVZ
UML
VirtualBox
VMware ESX
VMware Workstation / Player
Microsoft Hyper-V
Libvirt also supports the following storage:
Directory backend
Local filesystem backend
Network filesystem backend
Logical backend
Disk backend
iSCSI backend
SCSI backend
Multipath backend
RBD (RADOS Block Device) backend
Sheepdog backend
So....
If CloudStack were designed to communicate with XAPI and Libvirt - it would open quite a few doors.
(maybe??)
Regards,
Paul Angus
S: +44 20 3603 0540 | M: +447711418784
paul.angus@shapeblue.com
-----Original Message-----
From: Chip Childers [mailto:chip.childers@sungard.com]
Sent: 08 February 2013 16:06
To: Paul Angus
Cc: Dave Cahill; cloudstack-dev@incubator.apache.org
Subject: Re: QEMU support in CloudStack
On Fri, Feb 08, 2013 at 10:47:29AM +0000, Paul Angus wrote:
> Hi Dave,
>
> Your suggestion ties in nicely which a proposal that I have been mulling over; and that is further abstraction of the hypervisor, by CloudStack becoming more ‘toolset agonistic’ to use Xen parlance, rather than hypervisor agnostic?
>
> It seems quite an elegant solution to expanding hypervisor support
> (which easy for me to say as I don't know how to do it)…
>
> The current toolsets / interfaces become something like:
>
> OVM Manager (OVM)
> vCenter (ESXi)
> XAPI (XenServer, XCP, Xen+XAPI)
> Libvirt (KVM, Xen+Libvirt, QEMU)
> WMI/PowerShell (Hyper-V)
>
> + a bit of logic when copying scripts across to put them in the correct location for any given hypervisor/distribution.
Paul,
I'm not sure I quite understand what you are suggesting. Can you elaborate?
-chip
ShapeBlue provides a range of strategic and technical consulting and implementation services to help IT Service Providers and Enterprises to build a true IaaS compute cloud. ShapeBlue’s expertise, combined with CloudStack technology, allows IT Service Providers and Enterprises to deliver true, utility based, IaaS to the customer or end-user.
________________________________
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales.
Re: QEMU support in CloudStack
Posted by Chip Childers <ch...@sungard.com>.
On Fri, Feb 08, 2013 at 10:47:29AM +0000, Paul Angus wrote:
> Hi Dave,
>
> Your suggestion ties in nicely which a proposal that I have been mulling over; and that is further abstraction of the hypervisor, by CloudStack becoming more ‘toolset agonistic’ to use Xen parlance, rather than hypervisor agnostic?
>
> It seems quite an elegant solution to expanding hypervisor support (which easy for me to say as I don't know how to do it)…
>
> The current toolsets / interfaces become something like:
>
> OVM Manager (OVM)
> vCenter (ESXi)
> XAPI (XenServer, XCP, Xen+XAPI)
> Libvirt (KVM, Xen+Libvirt, QEMU)
> WMI/PowerShell (Hyper-V)
>
> + a bit of logic when copying scripts across to put them in the correct location for any given hypervisor/distribution.
Paul,
I'm not sure I quite understand what you are suggesting. Can you
elaborate?
-chip
RE: QEMU support in CloudStack
Posted by Paul Angus <pa...@shapeblue.com>.
Hi Dave,
Your suggestion ties in nicely which a proposal that I have been mulling over; and that is further abstraction of the hypervisor, by CloudStack becoming more ‘toolset agonistic’ to use Xen parlance, rather than hypervisor agnostic?
It seems quite an elegant solution to expanding hypervisor support (which easy for me to say as I don't know how to do it)…
The current toolsets / interfaces become something like:
OVM Manager (OVM)
vCenter (ESXi)
XAPI (XenServer, XCP, Xen+XAPI)
Libvirt (KVM, Xen+Libvirt, QEMU)
WMI/PowerShell (Hyper-V)
+ a bit of logic when copying scripts across to put them in the correct location for any given hypervisor/distribution.
Regards,
Paul Angus
S: +44 20 3603 0540 | M: +447711418784
paul.angus@shapeblue.com
-----Original Message-----
From: dcahill@midokura.jp [mailto:dcahill@midokura.jp] On Behalf Of Dave Cahill
Sent: 08 February 2013 09:34
To: cloudstack-dev@incubator.apache.org
Subject: QEMU support in CloudStack
Hi,
Recently I encountered two "nested virtualization" use cases which made me want QEMU hypervisor support in CloudStack. I'm interested to hear if anyone else is interested in this feature, and any notes on how it should be implemented.
Here is a good explanation from OpenStack docs [2] on why they support QEMU:
"From the perspective of the Compute service, the QEMU hypervisor is very similar to the KVM hypervisor. Both are controlled through libvirt, both support the same feature set, and all virtual machine images that are compatible with KVM are also compatible with QEMU. The main difference is that QEMU does not support native virtualization. Consequently, QEMU has worse performance than KVM and is a poor choice for a production deployment."
Here are the use cases I encountered:
[Use case: Dev environment]
Wanted to use Vagrant [1] to create a portable multi-node dev environment; however Vagrant uses VirtualBox, which doesn't support KVM.
Also, devcloud uses VirtualBox and devcloud-kvm uses kvm-within-kvm. I imagine maintenance of devcloud and devcloud-kvm would be easier if devcloud-kvm could use VirtualBox too.
Note: Of course, I'm aware of devcloud-kvm as an alternative for this use case, and I'll be looking into that next.
[Use case: Demo environment]
We may want to spin up a multi-node CloudStack install in Amazon AWS for demo purposes.
Again, AWS instances don't support KVM, so this is not possible without QEMU support.
[Implementation ideas]
The management server currently does a check for KVM support ("kvm-ok") on the host, and refuses to add the host if that fails. I think this check could be removed, as the agent setup scripts will fail anyway if the user is trying to setup a certain hypervisor on a machine which doesn't support it.
Create a new setting in agent.properties like "use_qemu", with a default of "false". If the person deploying CloudStack agent sets this to "true", cloud-setup-agent and other setup scripts would ignore lack of KVM support as long as QEMU support was available.
Lastly, when creating the libvirt XML file for a VM, set hypervisor to QEMU rather than KVM in the XML file depending on the config setting.
Thanks for reading,
Dave.
[1] http://www.vagrantup.com/
[2]
http://docs.openstack.org/trunk/openstack-compute/install/yum/content/qemu.html
ShapeBlue provides a range of strategic and technical consulting and implementation services to help IT Service Providers and Enterprises to build a true IaaS compute cloud. ShapeBlue’s expertise, combined with CloudStack technology, allows IT Service Providers and Enterprises to deliver true, utility based, IaaS to the customer or end-user.
________________________________
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales.