You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Marcus Sorensen <sh...@gmail.com> on 2013/12/14 06:39:16 UTC
kvm - why just qcow2
Is there any reason why we only support qcow2 format on KVM? It seems
like it would be fairly simple to support other formats, qemu-img can
handle going from VMDK to RAW for example, and qemu-kvm can even use
VMDK, QED, and other formats. It even seems like QemuImg.java was
written with other formats in mind. It seems like it wouldn't be a lot
of work to simply let other formats come through, we might have to
change LibvirtVMDef a bit so it can make the proper XML.
Re: kvm - why just qcow2
Posted by Marcus Sorensen <sh...@gmail.com>.
Yeah. I don't think it would be so bad, since we already do qemu-img
convert on any non-file based storage, or at least a copy. I think the
impact would be negligible or none going from a vmdk template to raw
compared to the existing qcow2 to raw, but definitely be felt on file based
primary storage that normally just copies the template file.
On Dec 14, 2013 4:35 AM, "Wido den Hollander" <wi...@widodh.nl> wrote:
>
>
> On 12/14/2013 09:05 AM, Marcus Sorensen wrote:
>
>> I suppose it would be best, and probably easiest, to accept templates in
>> vmdk, vdi, etc, but qemu-img convert to qcow2 during the copy to primary
>> storage, to keep the agent code from dealing with many formats. There's a
>> lot of code that would need rework to deal with non-qcow2 file based
>> disks.
>>
>
> I ran into this when implementing RBD. Since the code made all kinds of
> assumptions when it came to the format. RBD uses RAW (how KVM sees it).
>
> That's why I wrote QemuImg.java to abstract most of that work.
>
> But I agree with you, we shouldn't force ourselfs into QCOW2, but we
> should be aware that the hypervisors could be doing a lot of converting
> work.
>
> Wido
>
> On Dec 13, 2013 10:39 PM, "Marcus Sorensen" <sh...@gmail.com> wrote:
>>
>> Is there any reason why we only support qcow2 format on KVM? It seems
>>> like it would be fairly simple to support other formats, qemu-img can
>>> handle going from VMDK to RAW for example, and qemu-kvm can even use
>>> VMDK, QED, and other formats. It even seems like QemuImg.java was
>>> written with other formats in mind. It seems like it wouldn't be a lot
>>> of work to simply let other formats come through, we might have to
>>> change LibvirtVMDef a bit so it can make the proper XML.
>>>
>>>
>>
Re: kvm - why just qcow2
Posted by Wido den Hollander <wi...@widodh.nl>.
On 12/14/2013 09:05 AM, Marcus Sorensen wrote:
> I suppose it would be best, and probably easiest, to accept templates in
> vmdk, vdi, etc, but qemu-img convert to qcow2 during the copy to primary
> storage, to keep the agent code from dealing with many formats. There's a
> lot of code that would need rework to deal with non-qcow2 file based disks.
I ran into this when implementing RBD. Since the code made all kinds of
assumptions when it came to the format. RBD uses RAW (how KVM sees it).
That's why I wrote QemuImg.java to abstract most of that work.
But I agree with you, we shouldn't force ourselfs into QCOW2, but we
should be aware that the hypervisors could be doing a lot of converting
work.
Wido
> On Dec 13, 2013 10:39 PM, "Marcus Sorensen" <sh...@gmail.com> wrote:
>
>> Is there any reason why we only support qcow2 format on KVM? It seems
>> like it would be fairly simple to support other formats, qemu-img can
>> handle going from VMDK to RAW for example, and qemu-kvm can even use
>> VMDK, QED, and other formats. It even seems like QemuImg.java was
>> written with other formats in mind. It seems like it wouldn't be a lot
>> of work to simply let other formats come through, we might have to
>> change LibvirtVMDef a bit so it can make the proper XML.
>>
>
Re: kvm - why just qcow2
Posted by Marcus Sorensen <sh...@gmail.com>.
I've been using the systemvm VHD to test that format on my KVM
template patch, launching it as a user vm. I guess this means we may
be able to get away with not generating the qcow2 template anymore if
we wish. I haven't tested the system vm install script against it yet,
though.
On Tue, Dec 17, 2013 at 10:31 AM, Alex Huang <Al...@citrix.com> wrote:
> It makes sense to. I think we started with vmdk and is now using ova. If so, whoever did the change just didn't rename the file. Would have been better to just add a new processor when that change was done...
>
> --Alex
>
>> -----Original Message-----
>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> Sent: Tuesday, December 17, 2013 8:06 AM
>> To: dev@cloudstack.apache.org
>> Subject: Re: kvm - why just qcow2
>>
>> 1) Would it be OK to move VmdkProcessor to OVAProcessor, so we can use
>> VmdkProcessor for native .vmdk files?
>>
>> 2) I also noticed in my testing that .ova is the one format so far that doesn't
>> work out of the box with minor code changes for KVM, and it seems to be
>> due to the fact that VmdkProcessor untars the .ova to get the .vmdk and .ovf
>> files, but then registers the .ova file as the template file, not the .vmdk. It
>> would be nice to be able to use the same template for both vmware and kvm.
>>
>> I don't have a vmware setup to test any of this with. I can verify that the
>> OVAProcessor rename has the same results when registering a template, but
>> I wouldn't be able to do anything with #2.
>>
>> On Mon, Dec 16, 2013 at 6:55 PM, Marcus Sorensen <sh...@gmail.com>
>> wrote:
>> > No. You have to do qcow2 for KVM and VHD for XenServer as it is now,
>> > by the way. But if we allowes multiple types I think we'd want to
>> > convert to a single 'preferred' format for each primary storage,
>> > rather than messing with all different support between primary
>> > storage, image formats, and their respective support for snapshots,
>> > etc. So a file-based primary storage would always copy the template as
>> > qcow2 (from vmdk, vdi, raw, whaterver), and a raw based primary
>> > storage (like LVM or RBD) would always convert to raw (as it already
>> > does), when copying template to primary storage.
>> >
>> > On Mon, Dec 16, 2013 at 5:44 PM, Nux! <nu...@li.nux.ro> wrote:
>> >> On 16.12.2013 05:05, Marcus Sorensen wrote:
>> >>>
>> >>> Actually, I'm wrong. It used to qemu-img convert qcow2 to qcow2, but
>> >>> not any more. Qemu is actually just using vmdk natively.
>> >>
>> >>
>> >> Disappointment. I was going to build some "hv agnostic" templates,
>> >> but I really don't want my KVM to run vmdk files etc.
>> >> To clarify, if I add a raw template to CS, will it do any conversion
>> >> to VHD if it's a Xenserver or to Qcow2 if it's KVM?
>> >>
>> >> Lucian
>> >>
>> >> --
>> >> Sent from the Delta quadrant using Borg technology!
>> >>
>> >> Nux!
>> >> www.nux.ro
RE: kvm - why just qcow2
Posted by Alex Huang <Al...@citrix.com>.
It makes sense to. I think we started with vmdk and is now using ova. If so, whoever did the change just didn't rename the file. Would have been better to just add a new processor when that change was done...
--Alex
> -----Original Message-----
> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> Sent: Tuesday, December 17, 2013 8:06 AM
> To: dev@cloudstack.apache.org
> Subject: Re: kvm - why just qcow2
>
> 1) Would it be OK to move VmdkProcessor to OVAProcessor, so we can use
> VmdkProcessor for native .vmdk files?
>
> 2) I also noticed in my testing that .ova is the one format so far that doesn't
> work out of the box with minor code changes for KVM, and it seems to be
> due to the fact that VmdkProcessor untars the .ova to get the .vmdk and .ovf
> files, but then registers the .ova file as the template file, not the .vmdk. It
> would be nice to be able to use the same template for both vmware and kvm.
>
> I don't have a vmware setup to test any of this with. I can verify that the
> OVAProcessor rename has the same results when registering a template, but
> I wouldn't be able to do anything with #2.
>
> On Mon, Dec 16, 2013 at 6:55 PM, Marcus Sorensen <sh...@gmail.com>
> wrote:
> > No. You have to do qcow2 for KVM and VHD for XenServer as it is now,
> > by the way. But if we allowes multiple types I think we'd want to
> > convert to a single 'preferred' format for each primary storage,
> > rather than messing with all different support between primary
> > storage, image formats, and their respective support for snapshots,
> > etc. So a file-based primary storage would always copy the template as
> > qcow2 (from vmdk, vdi, raw, whaterver), and a raw based primary
> > storage (like LVM or RBD) would always convert to raw (as it already
> > does), when copying template to primary storage.
> >
> > On Mon, Dec 16, 2013 at 5:44 PM, Nux! <nu...@li.nux.ro> wrote:
> >> On 16.12.2013 05:05, Marcus Sorensen wrote:
> >>>
> >>> Actually, I'm wrong. It used to qemu-img convert qcow2 to qcow2, but
> >>> not any more. Qemu is actually just using vmdk natively.
> >>
> >>
> >> Disappointment. I was going to build some "hv agnostic" templates,
> >> but I really don't want my KVM to run vmdk files etc.
> >> To clarify, if I add a raw template to CS, will it do any conversion
> >> to VHD if it's a Xenserver or to Qcow2 if it's KVM?
> >>
> >> Lucian
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
Re: kvm - why just qcow2
Posted by Marcus Sorensen <sh...@gmail.com>.
1) Would it be OK to move VmdkProcessor to OVAProcessor, so we can use
VmdkProcessor for native .vmdk files?
2) I also noticed in my testing that .ova is the one format so far
that doesn't work out of the box with minor code changes for KVM, and
it seems to be due to the fact that VmdkProcessor untars the .ova to
get the .vmdk and .ovf files, but then registers the .ova file as the
template file, not the .vmdk. It would be nice to be able to use the
same template for both vmware and kvm.
I don't have a vmware setup to test any of this with. I can verify
that the OVAProcessor rename has the same results when registering a
template, but I wouldn't be able to do anything with #2.
On Mon, Dec 16, 2013 at 6:55 PM, Marcus Sorensen <sh...@gmail.com> wrote:
> No. You have to do qcow2 for KVM and VHD for XenServer as it is now,
> by the way. But if we allowes multiple types I think we'd want to
> convert to a single 'preferred' format for each primary storage,
> rather than messing with all different support between primary
> storage, image formats, and their respective support for snapshots,
> etc. So a file-based primary storage would always copy the template as
> qcow2 (from vmdk, vdi, raw, whaterver), and a raw based primary
> storage (like LVM or RBD) would always convert to raw (as it already
> does), when copying template to primary storage.
>
> On Mon, Dec 16, 2013 at 5:44 PM, Nux! <nu...@li.nux.ro> wrote:
>> On 16.12.2013 05:05, Marcus Sorensen wrote:
>>>
>>> Actually, I'm wrong. It used to qemu-img convert qcow2 to qcow2, but
>>> not any more. Qemu is actually just using vmdk natively.
>>
>>
>> Disappointment. I was going to build some "hv agnostic" templates, but I
>> really don't want my KVM to run vmdk files etc.
>> To clarify, if I add a raw template to CS, will it do any conversion to VHD
>> if it's a Xenserver or to Qcow2 if it's KVM?
>>
>> Lucian
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
Re: kvm - why just qcow2
Posted by Marcus Sorensen <sh...@gmail.com>.
No. You have to do qcow2 for KVM and VHD for XenServer as it is now,
by the way. But if we allowes multiple types I think we'd want to
convert to a single 'preferred' format for each primary storage,
rather than messing with all different support between primary
storage, image formats, and their respective support for snapshots,
etc. So a file-based primary storage would always copy the template as
qcow2 (from vmdk, vdi, raw, whaterver), and a raw based primary
storage (like LVM or RBD) would always convert to raw (as it already
does), when copying template to primary storage.
On Mon, Dec 16, 2013 at 5:44 PM, Nux! <nu...@li.nux.ro> wrote:
> On 16.12.2013 05:05, Marcus Sorensen wrote:
>>
>> Actually, I'm wrong. It used to qemu-img convert qcow2 to qcow2, but
>> not any more. Qemu is actually just using vmdk natively.
>
>
> Disappointment. I was going to build some "hv agnostic" templates, but I
> really don't want my KVM to run vmdk files etc.
> To clarify, if I add a raw template to CS, will it do any conversion to VHD
> if it's a Xenserver or to Qcow2 if it's KVM?
>
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
Re: kvm - why just qcow2
Posted by Nux! <nu...@li.nux.ro>.
On 16.12.2013 05:05, Marcus Sorensen wrote:
> Actually, I'm wrong. It used to qemu-img convert qcow2 to qcow2, but
> not any more. Qemu is actually just using vmdk natively.
Disappointment. I was going to build some "hv agnostic" templates, but
I really don't want my KVM to run vmdk files etc.
To clarify, if I add a raw template to CS, will it do any conversion to
VHD if it's a Xenserver or to Qcow2 if it's KVM?
Lucian
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
Re: kvm - why just qcow2
Posted by Chiradeep Vittal <Ch...@citrix.com>.
I thought the limitation was that qemu cannot take snapshots of vmdk
On 12/16/13 3:04 PM, "Marcus Sorensen" <sh...@gmail.com> wrote:
>In looking at the processors, I'm noticing that there's a
>VmdkProcessor, but it seems to want ova format, not vmdk (it expects
>.ova extension, for example). Sort of confusing if we want to support
>both vmdk and ova.
>
>On Sun, Dec 15, 2013 at 10:05 PM, Marcus Sorensen <sh...@gmail.com>
>wrote:
>> Actually, I'm wrong. It used to qemu-img convert qcow2 to qcow2, but
>> not any more. Qemu is actually just using vmdk natively.
>>
>> On Sun, Dec 15, 2013 at 9:33 PM, Marcus Sorensen <sh...@gmail.com>
>>wrote:
>>> I've been playing with this a little, and it seems that, interestingly
>>> enough, if I simply add 'vmdk' to the enum in
>>> api/src/com/cloud/storage/Storage.java, and tell CS about the file
>>> extension in
>>>server/src/com/cloud/template/HypervisorTemplateAdapter.java,
>>> KVM just seems to work with it. qemu-img autodetects the input format
>>> and converts it where it would normally would convert qcow2. This is
>>> partially enabled by the fact that we seem to qemu-img convert even
>>> qcow2 to qcow2 when copying templates to primary storage, so it just
>>> converted the vmdk as necessary. So as long as qemu-img has support
>>> for it, it seems you can upload any format and cloudstack will
>>> normalize to qcow2 (or raw, depending on primary storage format) when
>>> copying from secondary to primary.
>>>
>>> The preprocessor idea is interesting, but I do like the idea of
>>> eventually working toward having a single .qcow2 or .vmdk or .vhd that
>>> can be used by any hypervisor that's capable of converting it to
>>> native format upon copying to primary storage.
>>>
>>> On Sat, Dec 14, 2013 at 9:05 AM, Marcus Sorensen <sh...@gmail.com>
>>>wrote:
>>>> Sure, I'll take a look, it might just do the trick. I imagine this
>>>>would be
>>>> run in the SSVM? On the other hand, if we pass off format
>>>>conversation to
>>>> the hypervisors template copy as discussed, we could move toward multi
>>>> hypervisor templates, where you can upload and store a single vmdk or
>>>>vdi
>>>> and the hypervisors just convert as necessary when copying to primary
>>>> storage.
>>>>
>>>> On Dec 14, 2013 8:04 AM, "Alex Huang" <Al...@citrix.com> wrote:
>>>>>
>>>>> Marcus,
>>>>>
>>>>> When I refactored the template upload process ages ago, I left in an
>>>>> interface called Processor.java. The whole purpose of which was to
>>>>>allow
>>>>> others to add post processing of the template once its been
>>>>>uploaded. Such
>>>>> conversions can be done there. Take a look and see if it suits your
>>>>> purposes.
>>>>>
>>>>> --Alex
>>>>>
>>>>> > -----Original Message-----
>>>>> > From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>> > Sent: Saturday, December 14, 2013 12:06 AM
>>>>> > To: dev@cloudstack.apache.org
>>>>> > Subject: Re: kvm - why just qcow2
>>>>> >
>>>>> > I suppose it would be best, and probably easiest, to accept
>>>>>templates in
>>>>> > vmdk, vdi, etc, but qemu-img convert to qcow2 during the copy to
>>>>>primary
>>>>> > storage, to keep the agent code from dealing with many formats.
>>>>>There's
>>>>> > a
>>>>> > lot of code that would need rework to deal with non-qcow2 file
>>>>>based
>>>>> > disks.
>>>>> > On Dec 13, 2013 10:39 PM, "Marcus Sorensen" <sh...@gmail.com>
>>>>> > wrote:
>>>>> >
>>>>> > > Is there any reason why we only support qcow2 format on KVM? It
>>>>>seems
>>>>> > > like it would be fairly simple to support other formats,
>>>>>qemu-img can
>>>>> > > handle going from VMDK to RAW for example, and qemu-kvm can even
>>>>> > use
>>>>> > > VMDK, QED, and other formats. It even seems like QemuImg.java was
>>>>> > > written with other formats in mind. It seems like it wouldn't be
>>>>>a lot
>>>>> > > of work to simply let other formats come through, we might have
>>>>>to
>>>>> > > change LibvirtVMDef a bit so it can make the proper XML.
>>>>> > >
Re: kvm - why just qcow2
Posted by Marcus Sorensen <sh...@gmail.com>.
In looking at the processors, I'm noticing that there's a
VmdkProcessor, but it seems to want ova format, not vmdk (it expects
.ova extension, for example). Sort of confusing if we want to support
both vmdk and ova.
On Sun, Dec 15, 2013 at 10:05 PM, Marcus Sorensen <sh...@gmail.com> wrote:
> Actually, I'm wrong. It used to qemu-img convert qcow2 to qcow2, but
> not any more. Qemu is actually just using vmdk natively.
>
> On Sun, Dec 15, 2013 at 9:33 PM, Marcus Sorensen <sh...@gmail.com> wrote:
>> I've been playing with this a little, and it seems that, interestingly
>> enough, if I simply add 'vmdk' to the enum in
>> api/src/com/cloud/storage/Storage.java, and tell CS about the file
>> extension in server/src/com/cloud/template/HypervisorTemplateAdapter.java,
>> KVM just seems to work with it. qemu-img autodetects the input format
>> and converts it where it would normally would convert qcow2. This is
>> partially enabled by the fact that we seem to qemu-img convert even
>> qcow2 to qcow2 when copying templates to primary storage, so it just
>> converted the vmdk as necessary. So as long as qemu-img has support
>> for it, it seems you can upload any format and cloudstack will
>> normalize to qcow2 (or raw, depending on primary storage format) when
>> copying from secondary to primary.
>>
>> The preprocessor idea is interesting, but I do like the idea of
>> eventually working toward having a single .qcow2 or .vmdk or .vhd that
>> can be used by any hypervisor that's capable of converting it to
>> native format upon copying to primary storage.
>>
>> On Sat, Dec 14, 2013 at 9:05 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>>> Sure, I'll take a look, it might just do the trick. I imagine this would be
>>> run in the SSVM? On the other hand, if we pass off format conversation to
>>> the hypervisors template copy as discussed, we could move toward multi
>>> hypervisor templates, where you can upload and store a single vmdk or vdi
>>> and the hypervisors just convert as necessary when copying to primary
>>> storage.
>>>
>>> On Dec 14, 2013 8:04 AM, "Alex Huang" <Al...@citrix.com> wrote:
>>>>
>>>> Marcus,
>>>>
>>>> When I refactored the template upload process ages ago, I left in an
>>>> interface called Processor.java. The whole purpose of which was to allow
>>>> others to add post processing of the template once its been uploaded. Such
>>>> conversions can be done there. Take a look and see if it suits your
>>>> purposes.
>>>>
>>>> --Alex
>>>>
>>>> > -----Original Message-----
>>>> > From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>> > Sent: Saturday, December 14, 2013 12:06 AM
>>>> > To: dev@cloudstack.apache.org
>>>> > Subject: Re: kvm - why just qcow2
>>>> >
>>>> > I suppose it would be best, and probably easiest, to accept templates in
>>>> > vmdk, vdi, etc, but qemu-img convert to qcow2 during the copy to primary
>>>> > storage, to keep the agent code from dealing with many formats. There's
>>>> > a
>>>> > lot of code that would need rework to deal with non-qcow2 file based
>>>> > disks.
>>>> > On Dec 13, 2013 10:39 PM, "Marcus Sorensen" <sh...@gmail.com>
>>>> > wrote:
>>>> >
>>>> > > Is there any reason why we only support qcow2 format on KVM? It seems
>>>> > > like it would be fairly simple to support other formats, qemu-img can
>>>> > > handle going from VMDK to RAW for example, and qemu-kvm can even
>>>> > use
>>>> > > VMDK, QED, and other formats. It even seems like QemuImg.java was
>>>> > > written with other formats in mind. It seems like it wouldn't be a lot
>>>> > > of work to simply let other formats come through, we might have to
>>>> > > change LibvirtVMDef a bit so it can make the proper XML.
>>>> > >
Re: kvm - why just qcow2
Posted by Marcus Sorensen <sh...@gmail.com>.
Actually, I'm wrong. It used to qemu-img convert qcow2 to qcow2, but
not any more. Qemu is actually just using vmdk natively.
On Sun, Dec 15, 2013 at 9:33 PM, Marcus Sorensen <sh...@gmail.com> wrote:
> I've been playing with this a little, and it seems that, interestingly
> enough, if I simply add 'vmdk' to the enum in
> api/src/com/cloud/storage/Storage.java, and tell CS about the file
> extension in server/src/com/cloud/template/HypervisorTemplateAdapter.java,
> KVM just seems to work with it. qemu-img autodetects the input format
> and converts it where it would normally would convert qcow2. This is
> partially enabled by the fact that we seem to qemu-img convert even
> qcow2 to qcow2 when copying templates to primary storage, so it just
> converted the vmdk as necessary. So as long as qemu-img has support
> for it, it seems you can upload any format and cloudstack will
> normalize to qcow2 (or raw, depending on primary storage format) when
> copying from secondary to primary.
>
> The preprocessor idea is interesting, but I do like the idea of
> eventually working toward having a single .qcow2 or .vmdk or .vhd that
> can be used by any hypervisor that's capable of converting it to
> native format upon copying to primary storage.
>
> On Sat, Dec 14, 2013 at 9:05 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>> Sure, I'll take a look, it might just do the trick. I imagine this would be
>> run in the SSVM? On the other hand, if we pass off format conversation to
>> the hypervisors template copy as discussed, we could move toward multi
>> hypervisor templates, where you can upload and store a single vmdk or vdi
>> and the hypervisors just convert as necessary when copying to primary
>> storage.
>>
>> On Dec 14, 2013 8:04 AM, "Alex Huang" <Al...@citrix.com> wrote:
>>>
>>> Marcus,
>>>
>>> When I refactored the template upload process ages ago, I left in an
>>> interface called Processor.java. The whole purpose of which was to allow
>>> others to add post processing of the template once its been uploaded. Such
>>> conversions can be done there. Take a look and see if it suits your
>>> purposes.
>>>
>>> --Alex
>>>
>>> > -----Original Message-----
>>> > From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>> > Sent: Saturday, December 14, 2013 12:06 AM
>>> > To: dev@cloudstack.apache.org
>>> > Subject: Re: kvm - why just qcow2
>>> >
>>> > I suppose it would be best, and probably easiest, to accept templates in
>>> > vmdk, vdi, etc, but qemu-img convert to qcow2 during the copy to primary
>>> > storage, to keep the agent code from dealing with many formats. There's
>>> > a
>>> > lot of code that would need rework to deal with non-qcow2 file based
>>> > disks.
>>> > On Dec 13, 2013 10:39 PM, "Marcus Sorensen" <sh...@gmail.com>
>>> > wrote:
>>> >
>>> > > Is there any reason why we only support qcow2 format on KVM? It seems
>>> > > like it would be fairly simple to support other formats, qemu-img can
>>> > > handle going from VMDK to RAW for example, and qemu-kvm can even
>>> > use
>>> > > VMDK, QED, and other formats. It even seems like QemuImg.java was
>>> > > written with other formats in mind. It seems like it wouldn't be a lot
>>> > > of work to simply let other formats come through, we might have to
>>> > > change LibvirtVMDef a bit so it can make the proper XML.
>>> > >
Re: kvm - why just qcow2
Posted by Marcus Sorensen <sh...@gmail.com>.
I've been playing with this a little, and it seems that, interestingly
enough, if I simply add 'vmdk' to the enum in
api/src/com/cloud/storage/Storage.java, and tell CS about the file
extension in server/src/com/cloud/template/HypervisorTemplateAdapter.java,
KVM just seems to work with it. qemu-img autodetects the input format
and converts it where it would normally would convert qcow2. This is
partially enabled by the fact that we seem to qemu-img convert even
qcow2 to qcow2 when copying templates to primary storage, so it just
converted the vmdk as necessary. So as long as qemu-img has support
for it, it seems you can upload any format and cloudstack will
normalize to qcow2 (or raw, depending on primary storage format) when
copying from secondary to primary.
The preprocessor idea is interesting, but I do like the idea of
eventually working toward having a single .qcow2 or .vmdk or .vhd that
can be used by any hypervisor that's capable of converting it to
native format upon copying to primary storage.
On Sat, Dec 14, 2013 at 9:05 AM, Marcus Sorensen <sh...@gmail.com> wrote:
> Sure, I'll take a look, it might just do the trick. I imagine this would be
> run in the SSVM? On the other hand, if we pass off format conversation to
> the hypervisors template copy as discussed, we could move toward multi
> hypervisor templates, where you can upload and store a single vmdk or vdi
> and the hypervisors just convert as necessary when copying to primary
> storage.
>
> On Dec 14, 2013 8:04 AM, "Alex Huang" <Al...@citrix.com> wrote:
>>
>> Marcus,
>>
>> When I refactored the template upload process ages ago, I left in an
>> interface called Processor.java. The whole purpose of which was to allow
>> others to add post processing of the template once its been uploaded. Such
>> conversions can be done there. Take a look and see if it suits your
>> purposes.
>>
>> --Alex
>>
>> > -----Original Message-----
>> > From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>> > Sent: Saturday, December 14, 2013 12:06 AM
>> > To: dev@cloudstack.apache.org
>> > Subject: Re: kvm - why just qcow2
>> >
>> > I suppose it would be best, and probably easiest, to accept templates in
>> > vmdk, vdi, etc, but qemu-img convert to qcow2 during the copy to primary
>> > storage, to keep the agent code from dealing with many formats. There's
>> > a
>> > lot of code that would need rework to deal with non-qcow2 file based
>> > disks.
>> > On Dec 13, 2013 10:39 PM, "Marcus Sorensen" <sh...@gmail.com>
>> > wrote:
>> >
>> > > Is there any reason why we only support qcow2 format on KVM? It seems
>> > > like it would be fairly simple to support other formats, qemu-img can
>> > > handle going from VMDK to RAW for example, and qemu-kvm can even
>> > use
>> > > VMDK, QED, and other formats. It even seems like QemuImg.java was
>> > > written with other formats in mind. It seems like it wouldn't be a lot
>> > > of work to simply let other formats come through, we might have to
>> > > change LibvirtVMDef a bit so it can make the proper XML.
>> > >
RE: kvm - why just qcow2
Posted by Marcus Sorensen <sh...@gmail.com>.
Sure, I'll take a look, it might just do the trick. I imagine this would be
run in the SSVM? On the other hand, if we pass off format conversation to
the hypervisors template copy as discussed, we could move toward multi
hypervisor templates, where you can upload and store a single vmdk or vdi
and the hypervisors just convert as necessary when copying to primary
storage.
On Dec 14, 2013 8:04 AM, "Alex Huang" <Al...@citrix.com> wrote:
> Marcus,
>
> When I refactored the template upload process ages ago, I left in an
> interface called Processor.java. The whole purpose of which was to allow
> others to add post processing of the template once its been uploaded. Such
> conversions can be done there. Take a look and see if it suits your
> purposes.
>
> --Alex
>
> > -----Original Message-----
> > From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> > Sent: Saturday, December 14, 2013 12:06 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: kvm - why just qcow2
> >
> > I suppose it would be best, and probably easiest, to accept templates in
> > vmdk, vdi, etc, but qemu-img convert to qcow2 during the copy to primary
> > storage, to keep the agent code from dealing with many formats. There's a
> > lot of code that would need rework to deal with non-qcow2 file based
> disks.
> > On Dec 13, 2013 10:39 PM, "Marcus Sorensen" <sh...@gmail.com>
> > wrote:
> >
> > > Is there any reason why we only support qcow2 format on KVM? It seems
> > > like it would be fairly simple to support other formats, qemu-img can
> > > handle going from VMDK to RAW for example, and qemu-kvm can even
> > use
> > > VMDK, QED, and other formats. It even seems like QemuImg.java was
> > > written with other formats in mind. It seems like it wouldn't be a lot
> > > of work to simply let other formats come through, we might have to
> > > change LibvirtVMDef a bit so it can make the proper XML.
> > >
>
RE: kvm - why just qcow2
Posted by Alex Huang <Al...@citrix.com>.
Marcus,
When I refactored the template upload process ages ago, I left in an interface called Processor.java. The whole purpose of which was to allow others to add post processing of the template once its been uploaded. Such conversions can be done there. Take a look and see if it suits your purposes.
--Alex
> -----Original Message-----
> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
> Sent: Saturday, December 14, 2013 12:06 AM
> To: dev@cloudstack.apache.org
> Subject: Re: kvm - why just qcow2
>
> I suppose it would be best, and probably easiest, to accept templates in
> vmdk, vdi, etc, but qemu-img convert to qcow2 during the copy to primary
> storage, to keep the agent code from dealing with many formats. There's a
> lot of code that would need rework to deal with non-qcow2 file based disks.
> On Dec 13, 2013 10:39 PM, "Marcus Sorensen" <sh...@gmail.com>
> wrote:
>
> > Is there any reason why we only support qcow2 format on KVM? It seems
> > like it would be fairly simple to support other formats, qemu-img can
> > handle going from VMDK to RAW for example, and qemu-kvm can even
> use
> > VMDK, QED, and other formats. It even seems like QemuImg.java was
> > written with other formats in mind. It seems like it wouldn't be a lot
> > of work to simply let other formats come through, we might have to
> > change LibvirtVMDef a bit so it can make the proper XML.
> >
Re: kvm - why just qcow2
Posted by Marcus Sorensen <sh...@gmail.com>.
I suppose it would be best, and probably easiest, to accept templates in
vmdk, vdi, etc, but qemu-img convert to qcow2 during the copy to primary
storage, to keep the agent code from dealing with many formats. There's a
lot of code that would need rework to deal with non-qcow2 file based disks.
On Dec 13, 2013 10:39 PM, "Marcus Sorensen" <sh...@gmail.com> wrote:
> Is there any reason why we only support qcow2 format on KVM? It seems
> like it would be fairly simple to support other formats, qemu-img can
> handle going from VMDK to RAW for example, and qemu-kvm can even use
> VMDK, QED, and other formats. It even seems like QemuImg.java was
> written with other formats in mind. It seems like it wouldn't be a lot
> of work to simply let other formats come through, we might have to
> change LibvirtVMDef a bit so it can make the proper XML.
>