You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Will Stevens <ws...@cloudops.com> on 2014/12/01 22:06:27 UTC
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Edison,
According to this page the default compat option is 0.10:
http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
Did you find that to not be the case and is that why you had to add the
compat option?
BTW, in an attempt to get the master system vms building again, I committed
a change to master to remove the compat option. We now have master system
vms building correctly, but Rohit rightly pointed out that I had basically
reverted your change.
I created a Jira ticket for this issue:
https://issues.apache.org/jira/browse/CLOUDSTACK-7959
I have not reverted my change in master at this point, so it is building
right now, but I also did not make the change to 4.5, so that branch is
currently failing to build system vms.
I will see if I can track down in which version of qemu the compat option
was added so we can add some intelligent logic around this command.
Will
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com> wrote:
> Hi Leo,
> Our new internal build system are using ubuntu 14.04 or something like
> that, which has qemu 1.x installed by default, that's why I added the
> "compat" option in the build script, otherwise, the image build by
> qemu 1.x, won't work on RHEL 6.x.
> The fix would be, in the build script, check the version of qemu-img,
> if it's 0.x, then don't add "compat" option. There are a lot of people
> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
> we build can still work on these machines.
>
> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <LS...@schubergphilis.com>
> wrote:
> > Hey Edison, all,
> >
> >
> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
> >
> > breaks our internal build with
> >
> > + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
> > systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
> > Unknown option 'compat'
> > qemu-img: Invalid options for file format 'qcow2'.
> >
> > This is on CentOS release 6.6, which has
> qemu-img-0.15.0-1.el6.rfx.x86_64.
> >
> > Based on
> > http://wiki.qemu.org/OlderNews
> > that seems like it is a really old qemu (august 2011).
> >
> > I'm guessing you have a newer OS / newer qemu? Can you please let me
> know what OS, OS version and qemu(-img) version you are using?
> >
> > Also, does anyone know if there some minimum version of qemu-img that
> should be used / cloudstack assumes? Is 0.15 still ok to do an acceptable
> image conversion with? (we currently don't have any kvm use ourselves, but,
> I'd like for our build infra to produce useful kvm images nonetheless).
> >
> > According to
> > http://wiki.qemu.org/ChangeLog/1.1
> > the -o compat switch was introduced in 1.1.
> >
> > According to
> >
> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
> > the default format was changed from 0.10 to 1.1 in qemu 1.7 and onwards.
> >
> > The libvirt people
> > https://bugzilla.redhat.com/show_bug.cgi?id=997977
> > say they pass -o compat when qemu supports it (so when v >= 1.1 I
> imagine).
> >
> > I think we should do the same in the build script and I'll make a patch.
> >
> > But, should we publish newer-format images too? According to
> > http://wiki.qemu.org/Features/Qcow3
> > the new format is much better so I imagine qcow/kvm users will really
> appreciate the newer formats.
> >
> > Thoughts?
> >
> >
> > cheers,
> >
> >
> > Leo
> >
>
>
>
> --
> Best Regards,
> Edison
>
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Will Stevens <ws...@cloudops.com>.
I have gotten a bit side tracked with other work today. I have pushed this
change to master and I have verified that everything is working correctly.
I will be pushing a MERGE REQUEST for the same (ish) change to the 4.5
branch. Once it is committed I will validate that everything works as
expected since I have to kick off a build of the system vms to be positive
that everything works as expected.
Cheers,
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Dec 2, 2014 at 11:34 AM, Edison Su <Ed...@citrix.com> wrote:
>
>
> > -----Original Message-----
> > From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> > Behalf Of Will Stevens
> > Sent: Tuesday, December 02, 2014 8:06 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
> >
> > I will do some testing and see what I can come up with.
> >
> > BTW, I would be doing it the reverse of what you said. I would try to
> specify
> > the 'compat=0.10' option initially and if that is OK, then we are good.
> If it fails
> > (it is currently failing on the 4.5 branch) this is most likely because
> the version
> > of qemu-img does not support the 'compat' option being passed to the
> > convert call (which is what is happening right now on our build
> systems). In
> > that case, we will want to run the command without the compat option
> > because if it is too old to support the 'compat' option then it will
> have the
> > compat=0.10 default.
> Seems the qemu version is a little bit of mess, I think above method is
> good enough.
> BTW, thanks to look into the issue...
>
> >
> > Basically, I will try it the new way that Edison introduced. If that
> fails, I will fall
> > back to the way it was being done before Edison's change...
> >
> > Will
> >
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> > @CloudOps_
> >
> > On Tue, Dec 2, 2014 at 10:53 AM, Rohit Yadav <ro...@shapeblue.com>
> > wrote:
> >
> > > Hi Will,
> > >
> > > Reply in-line;
> > >
> > > On Tuesday 02 December 2014 08:41 PM, Will Stevens wrote:
> > >
> > >> @Rohit: If the script is run with 'set -e' it should not be a
> > >> problem because I was planning to do an explicit 'set +e' before the
> > >> call and then a 'set -e' after the call. This should handle this
> case, no?
> > >>
> > >
> > > Yes, I think that will work as well. If we are not sure about qemu
> > > version, it should be alright to try once without compat and check $?
> > > and retry again with compat option. The only corner case I can think
> > > of is when qemu-img genuinely exits with a non-zero exit code during
> > > image conversion.
> > >
> > >
> > >> @Nux: When you say "EL7 comes with qemu-img 1.5.3 and it requires the
> > >> compat option", what do you mean by that? Are you saying that if you
> > >> run the build system vms job on EL7, that you have to specify the
> > >> compat option even though it comes with qemu-img 1.5.3? That should
> > >> not be the case because the default compat flag should not have been
> > >> changed till 1.7. If you are talking about having a KVM host which
> > >> runs EL7 and has a qemu-img version of 1.5.3, then that is not really
> > >> relevant for this bash script.
> > >> The version of qemu-img which this script checking is the version on
> > >> the build system, not on the KVM host. It is validating if the
> > >> compat option needs to be passed to the 'qemu-img convert' command in
> > >> order for the image to have the 'compat=0.10' option set. We are not
> > >> debating that the 'compat=0.10' option needs to be set, that is for
> > >> sure. We are determining when it needs to be explicitly set on the
> > >> build systems.
> > >>
> > >> In short...
> > >> Jenkins kicks off a job to build the system vms. This build system
> > >> could have one of many different qemu-img versions. When this script
> > >> runs on the build system, we need to determine which command needs
> > to
> > >> be run in order to output a 'compat=0.10' compatible image.
> > >>
> > >> I will do some testing with Edison's suggestion and see if I can come
> > >> up with a solution that way as well and then we can pick which we
> > >> want to implement.
> > >>
> > >> Cheers,
> > >>
> > >> Will
> > >>
> > >>
> > >> *Will STEVENS*
> > >> Lead Developer
> > >>
> > >> *CloudOps* *| *Cloud Solutions Experts
> > >>
> > >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > tw
> > >> @CloudOps_
> > >>
> > >> On Tue, Dec 2, 2014 at 3:32 AM, Nux! <nu...@li.nux.ro> wrote:
> > >>
> > >> As I said above EL7 comes with qemu-img 1.5.3 and it requires the
> > >> compat
> > >>> option, so it's not only 1.7+.
> > >>> Personally I'm in favour of Will's approach.
> > >>>
> > >>> Lucian
> > >>>
> > >>> --
> > >>> Sent from the Delta quadrant using Borg technology!
> > >>>
> > >>> Nux!
> > >>> www.nux.ro
> > >>>
> > >>> ----- Original Message -----
> > >>>
> > >>>> From: "Rohit Yadav" <bh...@apache.org>
> > >>>> To: "Will Stevens" <ws...@cloudops.com>
> > >>>> Cc: "Edison Su" <Ed...@citrix.com>, "Wilder Rodrigues" <
> > >>>>
> > >>> WRodrigues@schubergphilis.com>, dev@cloudstack.apache.org,
> > >>>
> > >>>> "Leo Simons" <LS...@schubergphilis.com>
> > >>>> Sent: Tuesday, 2 December, 2014 07:36:21
> > >>>> Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
> > >>>>
> > >>>
> > >>> Hi Will,
> > >>>>
> > >>>> While we can use your first solution that checks exit status ($?)
> > >>>> but in case some is running the script with a "set -e", the script
> > >>>> will simply break when $? is not zero. I think if we know above
> > >>>> which qemu-img
> > >>>>
> > >>> version
> > >>>
> > >>>> we should we the compat option (1.7?) than that's great we should
> > >>>> we
> > >>>>
> > >>> that.
> > >>>
> > >>>> Thanks for looking into this.
> > >>>>
> > >>>> Regards.
> > >>>>
> > >>>> On Tue, Dec 2, 2014 at 8:01 AM, Will Stevens
> > >>>> <ws...@cloudops.com>
> > >>>>
> > >>> wrote:
> > >>>
> > >>>>
> > >>>> Thanks for that. Does that mean that we prefer to do an if else
> > >>>> based
> > >>>>> on
> > >>>>> version rather than basically doing a try catch?
> > >>>>>
> > >>>>> I have not been involved in previous bash scripting for CS, so I
> > >>>>> am willing to fall in line with the preferred method. What I
> > >>>>> proposed was based on how similar problems were solved in other
> > >>>>> code in the same
> > >>>>>
> > >>>> file.
> > >>>
> > >>>>
> > >>>>> If I check for version, do you agree that I should check for 1.7
> > >>>>> as the version since that is the version where the compact default
> > changed?
> > >>>>>
> > >>>>> I will try to get this resolved tomorrow.
> > >>>>>
> > >>>>> Cheers,
> > >>>>>
> > >>>>> Will
> > >>>>> On Dec 1, 2014 9:16 PM, "Edison Su" <Ed...@citrix.com> wrote:
> > >>>>>
> > >>>>> qemu-img |head -n 1|awk '{print $3}'
> > >>>>>>
> > >>>>>> should show the version of qemu-img
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> *From:* williamstevens@gmail.com
> > >>>>>> [mailto:williamstevens@gmail.com] *On Behalf Of *Will Stevens
> > >>>>>> *Sent:* Monday, December 01, 2014 2:17 PM
> > >>>>>> *To:* dev@cloudstack.apache.org
> > >>>>>> *Cc:* Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
> > >>>>>> *Subject:* Re: 05bec59c - kvm, qcow, systemvm qemu-img -o
> > compat
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Alright, so far I have found the following:
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb
> > >>>>>> 0
> > >>> 13f9b3dd1c
> > >>>
> > >>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Basically, if the qemu-img version is less than 1.7, we should
> > >>>>>> run the non 'compat' option version and if the version is 1.7 or
> > >>>>>> greater, we
> > >>>>>>
> > >>>>> should
> > >>>
> > >>>> run the new command with the 'compat' version.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Unfortunately I am not able to find a way to get the qemu-img
> > >>>>>> version from the command line.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> I am looking to basically add a conditional to try running with
> > >>>>>> the compat option and if that fails, then run it without the
> > >>>>>> compat option.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Basically, I would be replacing this:
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> > >>>>>> "${appliance_build_name}-kvm.qcow2"
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> With this:
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> set +e
> > >>>>>>
> > >>>>>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> > >>>>>> "${appliance_build_name}-kvm.qcow2"
> > >>>>>>
> > >>>>>> local qemuresult=$?
> > >>>>>>
> > >>>>>> set -e
> > >>>>>>
> > >>>>>> if [ ${qemuresult} != 0 ]; then
> > >>>>>>
> > >>>>>> log INFO "qemu-img convert failed, trying without compat
> option"
> > >>>>>>
> > >>>>>> qemu-img convert -f raw -c -O qcow2 raw.img
> > >>>>>> "${appliance_build_name}-kvm.qcow2"
> > >>>>>>
> > >>>>>> fi
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> What do you guys think? Is this a good enough solution? If you
> > >>>>>> guys agree I will implement it in master and make sure it works,
> > >>>>>> then we can merge the change back to 4.5 to fix that branch as
> > >>>>>> well.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Let me know if you have issues with this approach...
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Cheers,
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Will
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> *Will STEVENS*
> > >>>>>>
> > >>>>>> Lead Developer
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> *CloudOps | *Cloud Solutions Experts
> > >>>>>>
> > >>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > >>>>>>
> > >>>>>> w cloudops.com *|* tw @CloudOps_
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens
> > >>>>>> <ws...@cloudops.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>> Edison,
> > >>>>>>
> > >>>>>> According to this page the default compat option is 0.10:
> > >>>>>> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-
> > img.1.html
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Did you find that to not be the case and is that why you had to
> > >>>>>> add the compat option?
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> BTW, in an attempt to get the master system vms building again, I
> > >>>>>> committed a change to master to remove the compat option. We
> > now
> > >>>>>> have master system vms building correctly, but Rohit rightly
> > >>>>>> pointed out
> > >>>>>>
> > >>>>> that I
> > >>>
> > >>>> had basically reverted your change.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> I created a Jira ticket for this issue:
> > >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> I have not reverted my change in master at this point, so it is
> > >>>>>>
> > >>>>> building
> > >>>
> > >>>> right now, but I also did not make the change to 4.5, so that
> > >>>> branch is
> > >>>>>> currently failing to build system vms.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> I will see if I can track down in which version of qemu the
> > >>>>>> compat
> > >>>>>>
> > >>>>> option
> > >>>
> > >>>> was added so we can add some intelligent logic around this command.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Will
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> *Will STEVENS*
> > >>>>>>
> > >>>>>> Lead Developer
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> *CloudOps | *Cloud Solutions Experts
> > >>>>>>
> > >>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > >>>>>>
> > >>>>>> w cloudops.com *|* tw @CloudOps_
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com>
> > wrote:
> > >>>>>>
> > >>>>>> Hi Leo,
> > >>>>>> Our new internal build system are using ubuntu 14.04 or something
> > >>>>>> like that, which has qemu 1.x installed by default, that's why I
> > >>>>>> added the "compat" option in the build script, otherwise, the
> > >>>>>> image build by qemu 1.x, won't work on RHEL 6.x.
> > >>>>>> The fix would be, in the build script, check the version of
> > >>>>>> qemu-img, if it's 0.x, then don't add "compat" option. There are
> > >>>>>> a lot of people still using RHEL 6.x as KVM hypervisor, we have
> > >>>>>> to make sure the image we build can still work on these machines.
> > >>>>>>
> > >>>>>>
> > >>>>>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <
> > >>>>>>
> > >>>>> LSimons@schubergphilis.com>
> > >>>
> > >>>> wrote:
> > >>>>>>
> > >>>>>>> Hey Edison, all,
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>> https://github.com/apache/cloudstack/commit/
> > >>> 05bec59c1498dbcfb8a1089c86855fd3b433ea58
> > >>>
> > >>>>
> > >>>>>>> breaks our internal build with
> > >>>>>>>
> > >>>>>>> + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> > \
> > >>>>>>> systemvmtemplate-systemvm-persistent-config-4.5.0.124-
> > >>>>>>> kvm.qcow2
> > >>>>>>> Unknown option 'compat'
> > >>>>>>> qemu-img: Invalid options for file format 'qcow2'.
> > >>>>>>>
> > >>>>>>> This is on CentOS release 6.6, which has
> > >>>>>>>
> > >>>>>> qemu-img-0.15.0-1.el6.rfx.x86_64.
> > >>>>>>
> > >>>>>>>
> > >>>>>>> Based on
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>> http://secure-
> > web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue
> > >>>>>> 6-
> > >>> gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI
> > >>> 76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-
> > >>>
> > HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOl
> > derNew
> > >>> s
> > >>>
> > >>>> that seems like it is a really old qemu (august 2011).
> > >>>>>>>
> > >>>>>>> I'm guessing you have a newer OS / newer qemu? Can you please
> > >>>>>>> let me
> > >>>>>>>
> > >>>>>> know what OS, OS version and qemu(-img) version you are using?
> > >>>>>>
> > >>>>>>>
> > >>>>>>> Also, does anyone know if there some minimum version of qemu-
> > img
> > >>>>>>> that
> > >>>>>>>
> > >>>>>> should be used / cloudstack assumes? Is 0.15 still ok to do an
> > >>>>>>
> > >>>>> acceptable
> > >>>
> > >>>> image conversion with? (we currently don't have any kvm use
> > >>>> ourselves,
> > >>>>>>
> > >>>>> but,
> > >>>
> > >>>> I'd like for our build infra to produce useful kvm images
> nonetheless).
> > >>>>>>
> > >>>>>>>
> > >>>>>>> According to
> > >>>>>>>
> > >>>>>>>
> > >>>>>> http://secure-
> > web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-
> > >>>
> > Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6n
> > W6o9mB
> > >>> Z-
> > >>> AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93
> > >>> D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
> > >>>
> > >>>> the -o compat switch was introduced in 1.1.
> > >>>>>>>
> > >>>>>>> According to
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb
> > >>>>>> 0
> > >>> 13f9b3dd1c
> > >>>
> > >>>> the default format was changed from 0.10 to 1.1 in qemu 1.7 and
> > >>>>>>>
> > >>>>>> onwards.
> > >>>
> > >>>>
> > >>>>>>> The libvirt people
> > >>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=997977
> > >>>>>>> say they pass -o compat when qemu supports it (so when v >= 1.1
> > >>>>>>> I
> > >>>>>>>
> > >>>>>> imagine).
> > >>>>>>
> > >>>>>>>
> > >>>>>>> I think we should do the same in the build script and I'll make
> > >>>>>>> a
> > >>>>>>>
> > >>>>>> patch.
> > >>>
> > >>>>
> > >>>>>>> But, should we publish newer-format images too? According to
> > >>>>>>>
> > >>>>>>>
> > >>>>>> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-
> > >>> aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-
> > PlYYLhCZX0f768ZYK2fV1WbMFpQPz5
> > >>> JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-
> > >>> KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.
> > >>> org%2FFeatures%2FQcow3
> > >>>
> > >>>> the new format is much better so I imagine qcow/kvm users will
> > >>>> really
> > >>>>>>>
> > >>>>>> appreciate the newer formats.
> > >>>>>>
> > >>>>>>>
> > >>>>>>> Thoughts?
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> cheers,
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Leo
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Best Regards,
> > >>>>>> Edison
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>
> > >>
> > > --
> > > Regards,
> > > Rohit Yadav
> > > Software Architect, ShapeBlue
> > > M. +41 779015219 | rohit.yadav@shapeblue.com
> > >
> > > Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue
> > > and our range of CloudStack related services
> > >
> > > IaaS Cloud Design & Build<http://shapeblue.com/
> > > iaas-cloud-design-and-build//> CSForge – rapid IaaS deployment
> > > framework<http://shapeblue.com/csforge/>
> > > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> > > CloudStack Software
> > > Engineering<http://shapeblue.com/cloudstack-software-
> > > engineering/>
> > > CloudStack Infrastructure Support<http://shapeblue.com/
> > > cloudstack-infrastructure-support/>
> > > CloudStack Bootcamp Training Courses<http://shapeblue.com/
> > > cloudstack-training/>
> > >
> > > This email and any attachments to it may be confidential and are
> > > intended solely for the use of the individual to whom it is addressed.
> > > Any views or opinions expressed are solely those of the author and do
> > > not necessarily represent those of Shape Blue Ltd or related
> > > companies. If you are not the intended recipient of this email, you
> > > must neither take any action based upon its contents, nor copy or show
> > > it to anyone. Please contact the sender if you believe you have
> > > received this email in error. Shape Blue Ltd is a company incorporated
> > > in England & Wales. ShapeBlue Services India LLP is a company
> > > incorporated in India and is operated under license from Shape Blue
> > > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> > > Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA
> > > Pty Ltd is a company registered by The Republic of South Africa and is
> > traded under license from Shape Blue Ltd. ShapeBlue is a registered
> > trademark.
> > >
>
RE: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On
> Behalf Of Will Stevens
> Sent: Tuesday, December 02, 2014 8:06 AM
> To: dev@cloudstack.apache.org
> Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>
> I will do some testing and see what I can come up with.
>
> BTW, I would be doing it the reverse of what you said. I would try to specify
> the 'compat=0.10' option initially and if that is OK, then we are good. If it fails
> (it is currently failing on the 4.5 branch) this is most likely because the version
> of qemu-img does not support the 'compat' option being passed to the
> convert call (which is what is happening right now on our build systems). In
> that case, we will want to run the command without the compat option
> because if it is too old to support the 'compat' option then it will have the
> compat=0.10 default.
Seems the qemu version is a little bit of mess, I think above method is good enough.
BTW, thanks to look into the issue...
>
> Basically, I will try it the new way that Edison introduced. If that fails, I will fall
> back to the way it was being done before Edison's change...
>
> Will
>
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
> @CloudOps_
>
> On Tue, Dec 2, 2014 at 10:53 AM, Rohit Yadav <ro...@shapeblue.com>
> wrote:
>
> > Hi Will,
> >
> > Reply in-line;
> >
> > On Tuesday 02 December 2014 08:41 PM, Will Stevens wrote:
> >
> >> @Rohit: If the script is run with 'set -e' it should not be a
> >> problem because I was planning to do an explicit 'set +e' before the
> >> call and then a 'set -e' after the call. This should handle this case, no?
> >>
> >
> > Yes, I think that will work as well. If we are not sure about qemu
> > version, it should be alright to try once without compat and check $?
> > and retry again with compat option. The only corner case I can think
> > of is when qemu-img genuinely exits with a non-zero exit code during
> > image conversion.
> >
> >
> >> @Nux: When you say "EL7 comes with qemu-img 1.5.3 and it requires the
> >> compat option", what do you mean by that? Are you saying that if you
> >> run the build system vms job on EL7, that you have to specify the
> >> compat option even though it comes with qemu-img 1.5.3? That should
> >> not be the case because the default compat flag should not have been
> >> changed till 1.7. If you are talking about having a KVM host which
> >> runs EL7 and has a qemu-img version of 1.5.3, then that is not really
> >> relevant for this bash script.
> >> The version of qemu-img which this script checking is the version on
> >> the build system, not on the KVM host. It is validating if the
> >> compat option needs to be passed to the 'qemu-img convert' command in
> >> order for the image to have the 'compat=0.10' option set. We are not
> >> debating that the 'compat=0.10' option needs to be set, that is for
> >> sure. We are determining when it needs to be explicitly set on the
> >> build systems.
> >>
> >> In short...
> >> Jenkins kicks off a job to build the system vms. This build system
> >> could have one of many different qemu-img versions. When this script
> >> runs on the build system, we need to determine which command needs
> to
> >> be run in order to output a 'compat=0.10' compatible image.
> >>
> >> I will do some testing with Edison's suggestion and see if I can come
> >> up with a solution that way as well and then we can pick which we
> >> want to implement.
> >>
> >> Cheers,
> >>
> >> Will
> >>
> >>
> >> *Will STEVENS*
> >> Lead Developer
> >>
> >> *CloudOps* *| *Cloud Solutions Experts
> >>
> >> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> tw
> >> @CloudOps_
> >>
> >> On Tue, Dec 2, 2014 at 3:32 AM, Nux! <nu...@li.nux.ro> wrote:
> >>
> >> As I said above EL7 comes with qemu-img 1.5.3 and it requires the
> >> compat
> >>> option, so it's not only 1.7+.
> >>> Personally I'm in favour of Will's approach.
> >>>
> >>> Lucian
> >>>
> >>> --
> >>> Sent from the Delta quadrant using Borg technology!
> >>>
> >>> Nux!
> >>> www.nux.ro
> >>>
> >>> ----- Original Message -----
> >>>
> >>>> From: "Rohit Yadav" <bh...@apache.org>
> >>>> To: "Will Stevens" <ws...@cloudops.com>
> >>>> Cc: "Edison Su" <Ed...@citrix.com>, "Wilder Rodrigues" <
> >>>>
> >>> WRodrigues@schubergphilis.com>, dev@cloudstack.apache.org,
> >>>
> >>>> "Leo Simons" <LS...@schubergphilis.com>
> >>>> Sent: Tuesday, 2 December, 2014 07:36:21
> >>>> Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
> >>>>
> >>>
> >>> Hi Will,
> >>>>
> >>>> While we can use your first solution that checks exit status ($?)
> >>>> but in case some is running the script with a "set -e", the script
> >>>> will simply break when $? is not zero. I think if we know above
> >>>> which qemu-img
> >>>>
> >>> version
> >>>
> >>>> we should we the compat option (1.7?) than that's great we should
> >>>> we
> >>>>
> >>> that.
> >>>
> >>>> Thanks for looking into this.
> >>>>
> >>>> Regards.
> >>>>
> >>>> On Tue, Dec 2, 2014 at 8:01 AM, Will Stevens
> >>>> <ws...@cloudops.com>
> >>>>
> >>> wrote:
> >>>
> >>>>
> >>>> Thanks for that. Does that mean that we prefer to do an if else
> >>>> based
> >>>>> on
> >>>>> version rather than basically doing a try catch?
> >>>>>
> >>>>> I have not been involved in previous bash scripting for CS, so I
> >>>>> am willing to fall in line with the preferred method. What I
> >>>>> proposed was based on how similar problems were solved in other
> >>>>> code in the same
> >>>>>
> >>>> file.
> >>>
> >>>>
> >>>>> If I check for version, do you agree that I should check for 1.7
> >>>>> as the version since that is the version where the compact default
> changed?
> >>>>>
> >>>>> I will try to get this resolved tomorrow.
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>> Will
> >>>>> On Dec 1, 2014 9:16 PM, "Edison Su" <Ed...@citrix.com> wrote:
> >>>>>
> >>>>> qemu-img |head -n 1|awk '{print $3}'
> >>>>>>
> >>>>>> should show the version of qemu-img
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> *From:* williamstevens@gmail.com
> >>>>>> [mailto:williamstevens@gmail.com] *On Behalf Of *Will Stevens
> >>>>>> *Sent:* Monday, December 01, 2014 2:17 PM
> >>>>>> *To:* dev@cloudstack.apache.org
> >>>>>> *Cc:* Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
> >>>>>> *Subject:* Re: 05bec59c - kvm, qcow, systemvm qemu-img -o
> compat
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Alright, so far I have found the following:
> >>>>>>
> >>>>>>
> >>>>>>
> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb
> >>>>>> 0
> >>> 13f9b3dd1c
> >>>
> >>>>
> >>>>>>
> >>>>>>
> >>>>>> Basically, if the qemu-img version is less than 1.7, we should
> >>>>>> run the non 'compat' option version and if the version is 1.7 or
> >>>>>> greater, we
> >>>>>>
> >>>>> should
> >>>
> >>>> run the new command with the 'compat' version.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Unfortunately I am not able to find a way to get the qemu-img
> >>>>>> version from the command line.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> I am looking to basically add a conditional to try running with
> >>>>>> the compat option and if that fails, then run it without the
> >>>>>> compat option.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Basically, I would be replacing this:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> >>>>>> "${appliance_build_name}-kvm.qcow2"
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> With this:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> set +e
> >>>>>>
> >>>>>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> >>>>>> "${appliance_build_name}-kvm.qcow2"
> >>>>>>
> >>>>>> local qemuresult=$?
> >>>>>>
> >>>>>> set -e
> >>>>>>
> >>>>>> if [ ${qemuresult} != 0 ]; then
> >>>>>>
> >>>>>> log INFO "qemu-img convert failed, trying without compat option"
> >>>>>>
> >>>>>> qemu-img convert -f raw -c -O qcow2 raw.img
> >>>>>> "${appliance_build_name}-kvm.qcow2"
> >>>>>>
> >>>>>> fi
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> What do you guys think? Is this a good enough solution? If you
> >>>>>> guys agree I will implement it in master and make sure it works,
> >>>>>> then we can merge the change back to 4.5 to fix that branch as
> >>>>>> well.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Let me know if you have issues with this approach...
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Cheers,
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Will
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> *Will STEVENS*
> >>>>>>
> >>>>>> Lead Developer
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> *CloudOps | *Cloud Solutions Experts
> >>>>>>
> >>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >>>>>>
> >>>>>> w cloudops.com *|* tw @CloudOps_
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens
> >>>>>> <ws...@cloudops.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Edison,
> >>>>>>
> >>>>>> According to this page the default compat option is 0.10:
> >>>>>> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-
> img.1.html
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Did you find that to not be the case and is that why you had to
> >>>>>> add the compat option?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> BTW, in an attempt to get the master system vms building again, I
> >>>>>> committed a change to master to remove the compat option. We
> now
> >>>>>> have master system vms building correctly, but Rohit rightly
> >>>>>> pointed out
> >>>>>>
> >>>>> that I
> >>>
> >>>> had basically reverted your change.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> I created a Jira ticket for this issue:
> >>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> I have not reverted my change in master at this point, so it is
> >>>>>>
> >>>>> building
> >>>
> >>>> right now, but I also did not make the change to 4.5, so that
> >>>> branch is
> >>>>>> currently failing to build system vms.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> I will see if I can track down in which version of qemu the
> >>>>>> compat
> >>>>>>
> >>>>> option
> >>>
> >>>> was added so we can add some intelligent logic around this command.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Will
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> *Will STEVENS*
> >>>>>>
> >>>>>> Lead Developer
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> *CloudOps | *Cloud Solutions Experts
> >>>>>>
> >>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >>>>>>
> >>>>>> w cloudops.com *|* tw @CloudOps_
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com>
> wrote:
> >>>>>>
> >>>>>> Hi Leo,
> >>>>>> Our new internal build system are using ubuntu 14.04 or something
> >>>>>> like that, which has qemu 1.x installed by default, that's why I
> >>>>>> added the "compat" option in the build script, otherwise, the
> >>>>>> image build by qemu 1.x, won't work on RHEL 6.x.
> >>>>>> The fix would be, in the build script, check the version of
> >>>>>> qemu-img, if it's 0.x, then don't add "compat" option. There are
> >>>>>> a lot of people still using RHEL 6.x as KVM hypervisor, we have
> >>>>>> to make sure the image we build can still work on these machines.
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <
> >>>>>>
> >>>>> LSimons@schubergphilis.com>
> >>>
> >>>> wrote:
> >>>>>>
> >>>>>>> Hey Edison, all,
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> https://github.com/apache/cloudstack/commit/
> >>> 05bec59c1498dbcfb8a1089c86855fd3b433ea58
> >>>
> >>>>
> >>>>>>> breaks our internal build with
> >>>>>>>
> >>>>>>> + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> \
> >>>>>>> systemvmtemplate-systemvm-persistent-config-4.5.0.124-
> >>>>>>> kvm.qcow2
> >>>>>>> Unknown option 'compat'
> >>>>>>> qemu-img: Invalid options for file format 'qcow2'.
> >>>>>>>
> >>>>>>> This is on CentOS release 6.6, which has
> >>>>>>>
> >>>>>> qemu-img-0.15.0-1.el6.rfx.x86_64.
> >>>>>>
> >>>>>>>
> >>>>>>> Based on
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> http://secure-
> web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue
> >>>>>> 6-
> >>> gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI
> >>> 76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-
> >>>
> HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOl
> derNew
> >>> s
> >>>
> >>>> that seems like it is a really old qemu (august 2011).
> >>>>>>>
> >>>>>>> I'm guessing you have a newer OS / newer qemu? Can you please
> >>>>>>> let me
> >>>>>>>
> >>>>>> know what OS, OS version and qemu(-img) version you are using?
> >>>>>>
> >>>>>>>
> >>>>>>> Also, does anyone know if there some minimum version of qemu-
> img
> >>>>>>> that
> >>>>>>>
> >>>>>> should be used / cloudstack assumes? Is 0.15 still ok to do an
> >>>>>>
> >>>>> acceptable
> >>>
> >>>> image conversion with? (we currently don't have any kvm use
> >>>> ourselves,
> >>>>>>
> >>>>> but,
> >>>
> >>>> I'd like for our build infra to produce useful kvm images nonetheless).
> >>>>>>
> >>>>>>>
> >>>>>>> According to
> >>>>>>>
> >>>>>>>
> >>>>>> http://secure-
> web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-
> >>>
> Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6n
> W6o9mB
> >>> Z-
> >>> AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93
> >>> D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
> >>>
> >>>> the -o compat switch was introduced in 1.1.
> >>>>>>>
> >>>>>>> According to
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb
> >>>>>> 0
> >>> 13f9b3dd1c
> >>>
> >>>> the default format was changed from 0.10 to 1.1 in qemu 1.7 and
> >>>>>>>
> >>>>>> onwards.
> >>>
> >>>>
> >>>>>>> The libvirt people
> >>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=997977
> >>>>>>> say they pass -o compat when qemu supports it (so when v >= 1.1
> >>>>>>> I
> >>>>>>>
> >>>>>> imagine).
> >>>>>>
> >>>>>>>
> >>>>>>> I think we should do the same in the build script and I'll make
> >>>>>>> a
> >>>>>>>
> >>>>>> patch.
> >>>
> >>>>
> >>>>>>> But, should we publish newer-format images too? According to
> >>>>>>>
> >>>>>>>
> >>>>>> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-
> >>> aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-
> PlYYLhCZX0f768ZYK2fV1WbMFpQPz5
> >>> JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-
> >>> KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.
> >>> org%2FFeatures%2FQcow3
> >>>
> >>>> the new format is much better so I imagine qcow/kvm users will
> >>>> really
> >>>>>>>
> >>>>>> appreciate the newer formats.
> >>>>>>
> >>>>>>>
> >>>>>>> Thoughts?
> >>>>>>>
> >>>>>>>
> >>>>>>> cheers,
> >>>>>>>
> >>>>>>>
> >>>>>>> Leo
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best Regards,
> >>>>>> Edison
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>
> >>
> > --
> > Regards,
> > Rohit Yadav
> > Software Architect, ShapeBlue
> > M. +41 779015219 | rohit.yadav@shapeblue.com
> >
> > Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue
> > and our range of CloudStack related services
> >
> > IaaS Cloud Design & Build<http://shapeblue.com/
> > iaas-cloud-design-and-build//> CSForge – rapid IaaS deployment
> > framework<http://shapeblue.com/csforge/>
> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> > CloudStack Software
> > Engineering<http://shapeblue.com/cloudstack-software-
> > engineering/>
> > CloudStack Infrastructure Support<http://shapeblue.com/
> > cloudstack-infrastructure-support/>
> > CloudStack Bootcamp Training Courses<http://shapeblue.com/
> > cloudstack-training/>
> >
> > This email and any attachments to it may be confidential and are
> > intended solely for the use of the individual to whom it is addressed.
> > Any views or opinions expressed are solely those of the author and do
> > not necessarily represent those of Shape Blue Ltd or related
> > companies. If you are not the intended recipient of this email, you
> > must neither take any action based upon its contents, nor copy or show
> > it to anyone. Please contact the sender if you believe you have
> > received this email in error. Shape Blue Ltd is a company incorporated
> > in England & Wales. ShapeBlue Services India LLP is a company
> > incorporated in India and is operated under license from Shape Blue
> > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> > Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA
> > Pty Ltd is a company registered by The Republic of South Africa and is
> traded under license from Shape Blue Ltd. ShapeBlue is a registered
> trademark.
> >
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Will Stevens <ws...@cloudops.com>.
I will do some testing and see what I can come up with.
BTW, I would be doing it the reverse of what you said. I would try to
specify the 'compat=0.10' option initially and if that is OK, then we are
good. If it fails (it is currently failing on the 4.5 branch) this is most
likely because the version of qemu-img does not support the 'compat' option
being passed to the convert call (which is what is happening right now on
our build systems). In that case, we will want to run the command without
the compat option because if it is too old to support the 'compat' option
then it will have the compat=0.10 default.
Basically, I will try it the new way that Edison introduced. If that
fails, I will fall back to the way it was being done before Edison's
change...
Will
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Dec 2, 2014 at 10:53 AM, Rohit Yadav <ro...@shapeblue.com>
wrote:
> Hi Will,
>
> Reply in-line;
>
> On Tuesday 02 December 2014 08:41 PM, Will Stevens wrote:
>
>> @Rohit: If the script is run with 'set -e' it should not be a problem
>> because I was planning to do an explicit 'set +e' before the call and then
>> a 'set -e' after the call. This should handle this case, no?
>>
>
> Yes, I think that will work as well. If we are not sure about qemu
> version, it should be alright to try once without compat and check $?
> and retry again with compat option. The only corner case I can think of
> is when qemu-img genuinely exits with a non-zero exit code during image
> conversion.
>
>
>> @Nux: When you say "EL7 comes with qemu-img 1.5.3 and it requires the
>> compat option", what do you mean by that? Are you saying that if you run
>> the build system vms job on EL7, that you have to specify the compat
>> option
>> even though it comes with qemu-img 1.5.3? That should not be the case
>> because the default compat flag should not have been changed till 1.7. If
>> you are talking about having a KVM host which runs EL7 and has a qemu-img
>> version of 1.5.3, then that is not really relevant for this bash script.
>> The version of qemu-img which this script checking is the version on the
>> build system, not on the KVM host. It is validating if the compat option
>> needs to be passed to the 'qemu-img convert' command in order for the
>> image
>> to have the 'compat=0.10' option set. We are not debating that the
>> 'compat=0.10' option needs to be set, that is for sure. We are
>> determining
>> when it needs to be explicitly set on the build systems.
>>
>> In short...
>> Jenkins kicks off a job to build the system vms. This build system could
>> have one of many different qemu-img versions. When this script runs on
>> the
>> build system, we need to determine which command needs to be run in order
>> to output a 'compat=0.10' compatible image.
>>
>> I will do some testing with Edison's suggestion and see if I can come up
>> with a solution that way as well and then we can pick which we want to
>> implement.
>>
>> Cheers,
>>
>> Will
>>
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>>
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>>
>> On Tue, Dec 2, 2014 at 3:32 AM, Nux! <nu...@li.nux.ro> wrote:
>>
>> As I said above EL7 comes with qemu-img 1.5.3 and it requires the compat
>>> option, so it's not only 1.7+.
>>> Personally I'm in favour of Will's approach.
>>>
>>> Lucian
>>>
>>> --
>>> Sent from the Delta quadrant using Borg technology!
>>>
>>> Nux!
>>> www.nux.ro
>>>
>>> ----- Original Message -----
>>>
>>>> From: "Rohit Yadav" <bh...@apache.org>
>>>> To: "Will Stevens" <ws...@cloudops.com>
>>>> Cc: "Edison Su" <Ed...@citrix.com>, "Wilder Rodrigues" <
>>>>
>>> WRodrigues@schubergphilis.com>, dev@cloudstack.apache.org,
>>>
>>>> "Leo Simons" <LS...@schubergphilis.com>
>>>> Sent: Tuesday, 2 December, 2014 07:36:21
>>>> Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>>>>
>>>
>>> Hi Will,
>>>>
>>>> While we can use your first solution that checks exit status ($?) but in
>>>> case some is running the script with a "set -e", the script will simply
>>>> break when $? is not zero. I think if we know above which qemu-img
>>>>
>>> version
>>>
>>>> we should we the compat option (1.7?) than that's great we should we
>>>>
>>> that.
>>>
>>>> Thanks for looking into this.
>>>>
>>>> Regards.
>>>>
>>>> On Tue, Dec 2, 2014 at 8:01 AM, Will Stevens <ws...@cloudops.com>
>>>>
>>> wrote:
>>>
>>>>
>>>> Thanks for that. Does that mean that we prefer to do an if else based
>>>>> on
>>>>> version rather than basically doing a try catch?
>>>>>
>>>>> I have not been involved in previous bash scripting for CS, so I am
>>>>> willing to fall in line with the preferred method. What I proposed was
>>>>> based on how similar problems were solved in other code in the same
>>>>>
>>>> file.
>>>
>>>>
>>>>> If I check for version, do you agree that I should check for 1.7 as the
>>>>> version since that is the version where the compact default changed?
>>>>>
>>>>> I will try to get this resolved tomorrow.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Will
>>>>> On Dec 1, 2014 9:16 PM, "Edison Su" <Ed...@citrix.com> wrote:
>>>>>
>>>>> qemu-img |head -n 1|awk '{print $3}'
>>>>>>
>>>>>> should show the version of qemu-img
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* williamstevens@gmail.com [mailto:williamstevens@gmail.com]
>>>>>> *On
>>>>>> Behalf Of *Will Stevens
>>>>>> *Sent:* Monday, December 01, 2014 2:17 PM
>>>>>> *To:* dev@cloudstack.apache.org
>>>>>> *Cc:* Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
>>>>>> *Subject:* Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>>>>>>
>>>>>>
>>>>>>
>>>>>> Alright, so far I have found the following:
>>>>>>
>>>>>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb0
>>> 13f9b3dd1c
>>>
>>>>
>>>>>>
>>>>>>
>>>>>> Basically, if the qemu-img version is less than 1.7, we should run the
>>>>>> non 'compat' option version and if the version is 1.7 or greater, we
>>>>>>
>>>>> should
>>>
>>>> run the new command with the 'compat' version.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Unfortunately I am not able to find a way to get the qemu-img version
>>>>>> from the command line.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I am looking to basically add a conditional to try running with the
>>>>>> compat option and if that fails, then run it without the compat
>>>>>> option.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Basically, I would be replacing this:
>>>>>>
>>>>>>
>>>>>>
>>>>>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
>>>>>> "${appliance_build_name}-kvm.qcow2"
>>>>>>
>>>>>>
>>>>>>
>>>>>> With this:
>>>>>>
>>>>>>
>>>>>>
>>>>>> set +e
>>>>>>
>>>>>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
>>>>>> "${appliance_build_name}-kvm.qcow2"
>>>>>>
>>>>>> local qemuresult=$?
>>>>>>
>>>>>> set -e
>>>>>>
>>>>>> if [ ${qemuresult} != 0 ]; then
>>>>>>
>>>>>> log INFO "qemu-img convert failed, trying without compat option"
>>>>>>
>>>>>> qemu-img convert -f raw -c -O qcow2 raw.img
>>>>>> "${appliance_build_name}-kvm.qcow2"
>>>>>>
>>>>>> fi
>>>>>>
>>>>>>
>>>>>>
>>>>>> What do you guys think? Is this a good enough solution? If you guys
>>>>>> agree I will implement it in master and make sure it works, then we
>>>>>> can
>>>>>> merge the change back to 4.5 to fix that branch as well.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Let me know if you have issues with this approach...
>>>>>>
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Will
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Will STEVENS*
>>>>>>
>>>>>> Lead Developer
>>>>>>
>>>>>>
>>>>>>
>>>>>> *CloudOps | *Cloud Solutions Experts
>>>>>>
>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>>>>>
>>>>>> w cloudops.com *|* tw @CloudOps_
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <ws...@cloudops.com>
>>>>>> wrote:
>>>>>>
>>>>>> Edison,
>>>>>>
>>>>>> According to this page the default compat option is 0.10:
>>>>>> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
>>>>>>
>>>>>>
>>>>>>
>>>>>> Did you find that to not be the case and is that why you had to add
>>>>>> the
>>>>>> compat option?
>>>>>>
>>>>>>
>>>>>>
>>>>>> BTW, in an attempt to get the master system vms building again, I
>>>>>> committed a change to master to remove the compat option. We now have
>>>>>> master system vms building correctly, but Rohit rightly pointed out
>>>>>>
>>>>> that I
>>>
>>>> had basically reverted your change.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I created a Jira ticket for this issue:
>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
>>>>>>
>>>>>>
>>>>>>
>>>>>> I have not reverted my change in master at this point, so it is
>>>>>>
>>>>> building
>>>
>>>> right now, but I also did not make the change to 4.5, so that branch is
>>>>>> currently failing to build system vms.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I will see if I can track down in which version of qemu the compat
>>>>>>
>>>>> option
>>>
>>>> was added so we can add some intelligent logic around this command.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Will
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Will STEVENS*
>>>>>>
>>>>>> Lead Developer
>>>>>>
>>>>>>
>>>>>>
>>>>>> *CloudOps | *Cloud Solutions Experts
>>>>>>
>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>>>>>
>>>>>> w cloudops.com *|* tw @CloudOps_
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com> wrote:
>>>>>>
>>>>>> Hi Leo,
>>>>>> Our new internal build system are using ubuntu 14.04 or something like
>>>>>> that, which has qemu 1.x installed by default, that's why I added the
>>>>>> "compat" option in the build script, otherwise, the image build by
>>>>>> qemu 1.x, won't work on RHEL 6.x.
>>>>>> The fix would be, in the build script, check the version of qemu-img,
>>>>>> if it's 0.x, then don't add "compat" option. There are a lot of people
>>>>>> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
>>>>>> we build can still work on these machines.
>>>>>>
>>>>>>
>>>>>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <
>>>>>>
>>>>> LSimons@schubergphilis.com>
>>>
>>>> wrote:
>>>>>>
>>>>>>> Hey Edison, all,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> https://github.com/apache/cloudstack/commit/
>>> 05bec59c1498dbcfb8a1089c86855fd3b433ea58
>>>
>>>>
>>>>>>> breaks our internal build with
>>>>>>>
>>>>>>> + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
>>>>>>> systemvmtemplate-systemvm-persistent-config-4.5.0.124-
>>>>>>> kvm.qcow2
>>>>>>> Unknown option 'compat'
>>>>>>> qemu-img: Invalid options for file format 'qcow2'.
>>>>>>>
>>>>>>> This is on CentOS release 6.6, which has
>>>>>>>
>>>>>> qemu-img-0.15.0-1.el6.rfx.x86_64.
>>>>>>
>>>>>>>
>>>>>>> Based on
>>>>>>>
>>>>>>>
>>>>>> http://secure-web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue6-
>>> gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI
>>> 76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-
>>> HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOlderNews
>>>
>>>> that seems like it is a really old qemu (august 2011).
>>>>>>>
>>>>>>> I'm guessing you have a newer OS / newer qemu? Can you please let me
>>>>>>>
>>>>>> know what OS, OS version and qemu(-img) version you are using?
>>>>>>
>>>>>>>
>>>>>>> Also, does anyone know if there some minimum version of qemu-img that
>>>>>>>
>>>>>> should be used / cloudstack assumes? Is 0.15 still ok to do an
>>>>>>
>>>>> acceptable
>>>
>>>> image conversion with? (we currently don't have any kvm use ourselves,
>>>>>>
>>>>> but,
>>>
>>>> I'd like for our build infra to produce useful kvm images nonetheless).
>>>>>>
>>>>>>>
>>>>>>> According to
>>>>>>>
>>>>>>>
>>>>>> http://secure-web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-
>>> Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6nW6o9mBZ-
>>> AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93
>>> D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
>>>
>>>> the -o compat switch was introduced in 1.1.
>>>>>>>
>>>>>>> According to
>>>>>>>
>>>>>>>
>>>>>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb0
>>> 13f9b3dd1c
>>>
>>>> the default format was changed from 0.10 to 1.1 in qemu 1.7 and
>>>>>>>
>>>>>> onwards.
>>>
>>>>
>>>>>>> The libvirt people
>>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=997977
>>>>>>> say they pass -o compat when qemu supports it (so when v >= 1.1 I
>>>>>>>
>>>>>> imagine).
>>>>>>
>>>>>>>
>>>>>>> I think we should do the same in the build script and I'll make a
>>>>>>>
>>>>>> patch.
>>>
>>>>
>>>>>>> But, should we publish newer-format images too? According to
>>>>>>>
>>>>>>>
>>>>>> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-
>>> aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-PlYYLhCZX0f768ZYK2fV1WbMFpQPz5
>>> JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-
>>> KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.
>>> org%2FFeatures%2FQcow3
>>>
>>>> the new format is much better so I imagine qcow/kvm users will really
>>>>>>>
>>>>>> appreciate the newer formats.
>>>>>>
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>>
>>>>>>> cheers,
>>>>>>>
>>>>>>>
>>>>>>> Leo
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>> Edison
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>
>>
> --
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.yadav@shapeblue.com
>
> Blog: bhaisaab.org | Twitter: @_bhaisaab
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<http://shapeblue.com/
> iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-
> engineering/>
> CloudStack Infrastructure Support<http://shapeblue.com/
> cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<http://shapeblue.com/
> cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Will,
Reply in-line;
On Tuesday 02 December 2014 08:41 PM, Will Stevens wrote:
> @Rohit: If the script is run with 'set -e' it should not be a problem
> because I was planning to do an explicit 'set +e' before the call and then
> a 'set -e' after the call. This should handle this case, no?
Yes, I think that will work as well. If we are not sure about qemu
version, it should be alright to try once without compat and check $?
and retry again with compat option. The only corner case I can think of
is when qemu-img genuinely exits with a non-zero exit code during image
conversion.
>
> @Nux: When you say "EL7 comes with qemu-img 1.5.3 and it requires the
> compat option", what do you mean by that? Are you saying that if you run
> the build system vms job on EL7, that you have to specify the compat option
> even though it comes with qemu-img 1.5.3? That should not be the case
> because the default compat flag should not have been changed till 1.7. If
> you are talking about having a KVM host which runs EL7 and has a qemu-img
> version of 1.5.3, then that is not really relevant for this bash script.
> The version of qemu-img which this script checking is the version on the
> build system, not on the KVM host. It is validating if the compat option
> needs to be passed to the 'qemu-img convert' command in order for the image
> to have the 'compat=0.10' option set. We are not debating that the
> 'compat=0.10' option needs to be set, that is for sure. We are determining
> when it needs to be explicitly set on the build systems.
>
> In short...
> Jenkins kicks off a job to build the system vms. This build system could
> have one of many different qemu-img versions. When this script runs on the
> build system, we need to determine which command needs to be run in order
> to output a 'compat=0.10' compatible image.
>
> I will do some testing with Edison's suggestion and see if I can come up
> with a solution that way as well and then we can pick which we want to
> implement.
>
> Cheers,
>
> Will
>
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Tue, Dec 2, 2014 at 3:32 AM, Nux! <nu...@li.nux.ro> wrote:
>
>> As I said above EL7 comes with qemu-img 1.5.3 and it requires the compat
>> option, so it's not only 1.7+.
>> Personally I'm in favour of Will's approach.
>>
>> Lucian
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> ----- Original Message -----
>>> From: "Rohit Yadav" <bh...@apache.org>
>>> To: "Will Stevens" <ws...@cloudops.com>
>>> Cc: "Edison Su" <Ed...@citrix.com>, "Wilder Rodrigues" <
>> WRodrigues@schubergphilis.com>, dev@cloudstack.apache.org,
>>> "Leo Simons" <LS...@schubergphilis.com>
>>> Sent: Tuesday, 2 December, 2014 07:36:21
>>> Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>>
>>> Hi Will,
>>>
>>> While we can use your first solution that checks exit status ($?) but in
>>> case some is running the script with a "set -e", the script will simply
>>> break when $? is not zero. I think if we know above which qemu-img
>> version
>>> we should we the compat option (1.7?) than that's great we should we
>> that.
>>> Thanks for looking into this.
>>>
>>> Regards.
>>>
>>> On Tue, Dec 2, 2014 at 8:01 AM, Will Stevens <ws...@cloudops.com>
>> wrote:
>>>
>>>> Thanks for that. Does that mean that we prefer to do an if else based on
>>>> version rather than basically doing a try catch?
>>>>
>>>> I have not been involved in previous bash scripting for CS, so I am
>>>> willing to fall in line with the preferred method. What I proposed was
>>>> based on how similar problems were solved in other code in the same
>> file.
>>>>
>>>> If I check for version, do you agree that I should check for 1.7 as the
>>>> version since that is the version where the compact default changed?
>>>>
>>>> I will try to get this resolved tomorrow.
>>>>
>>>> Cheers,
>>>>
>>>> Will
>>>> On Dec 1, 2014 9:16 PM, "Edison Su" <Ed...@citrix.com> wrote:
>>>>
>>>>> qemu-img |head -n 1|awk '{print $3}'
>>>>>
>>>>> should show the version of qemu-img
>>>>>
>>>>>
>>>>>
>>>>> *From:* williamstevens@gmail.com [mailto:williamstevens@gmail.com] *On
>>>>> Behalf Of *Will Stevens
>>>>> *Sent:* Monday, December 01, 2014 2:17 PM
>>>>> *To:* dev@cloudstack.apache.org
>>>>> *Cc:* Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
>>>>> *Subject:* Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>>>>>
>>>>>
>>>>>
>>>>> Alright, so far I have found the following:
>>>>>
>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>>>>>
>>>>>
>>>>>
>>>>> Basically, if the qemu-img version is less than 1.7, we should run the
>>>>> non 'compat' option version and if the version is 1.7 or greater, we
>> should
>>>>> run the new command with the 'compat' version.
>>>>>
>>>>>
>>>>>
>>>>> Unfortunately I am not able to find a way to get the qemu-img version
>>>>> from the command line.
>>>>>
>>>>>
>>>>>
>>>>> I am looking to basically add a conditional to try running with the
>>>>> compat option and if that fails, then run it without the compat option.
>>>>>
>>>>>
>>>>>
>>>>> Basically, I would be replacing this:
>>>>>
>>>>>
>>>>>
>>>>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
>>>>> "${appliance_build_name}-kvm.qcow2"
>>>>>
>>>>>
>>>>>
>>>>> With this:
>>>>>
>>>>>
>>>>>
>>>>> set +e
>>>>>
>>>>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
>>>>> "${appliance_build_name}-kvm.qcow2"
>>>>>
>>>>> local qemuresult=$?
>>>>>
>>>>> set -e
>>>>>
>>>>> if [ ${qemuresult} != 0 ]; then
>>>>>
>>>>> log INFO "qemu-img convert failed, trying without compat option"
>>>>>
>>>>> qemu-img convert -f raw -c -O qcow2 raw.img
>>>>> "${appliance_build_name}-kvm.qcow2"
>>>>>
>>>>> fi
>>>>>
>>>>>
>>>>>
>>>>> What do you guys think? Is this a good enough solution? If you guys
>>>>> agree I will implement it in master and make sure it works, then we can
>>>>> merge the change back to 4.5 to fix that branch as well.
>>>>>
>>>>>
>>>>>
>>>>> Let me know if you have issues with this approach...
>>>>>
>>>>>
>>>>>
>>>>> Cheers,
>>>>>
>>>>>
>>>>>
>>>>> Will
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *Will STEVENS*
>>>>>
>>>>> Lead Developer
>>>>>
>>>>>
>>>>>
>>>>> *CloudOps | *Cloud Solutions Experts
>>>>>
>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>>>>
>>>>> w cloudops.com *|* tw @CloudOps_
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <ws...@cloudops.com>
>>>>> wrote:
>>>>>
>>>>> Edison,
>>>>>
>>>>> According to this page the default compat option is 0.10:
>>>>> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
>>>>>
>>>>>
>>>>>
>>>>> Did you find that to not be the case and is that why you had to add the
>>>>> compat option?
>>>>>
>>>>>
>>>>>
>>>>> BTW, in an attempt to get the master system vms building again, I
>>>>> committed a change to master to remove the compat option. We now have
>>>>> master system vms building correctly, but Rohit rightly pointed out
>> that I
>>>>> had basically reverted your change.
>>>>>
>>>>>
>>>>>
>>>>> I created a Jira ticket for this issue:
>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
>>>>>
>>>>>
>>>>>
>>>>> I have not reverted my change in master at this point, so it is
>> building
>>>>> right now, but I also did not make the change to 4.5, so that branch is
>>>>> currently failing to build system vms.
>>>>>
>>>>>
>>>>>
>>>>> I will see if I can track down in which version of qemu the compat
>> option
>>>>> was added so we can add some intelligent logic around this command.
>>>>>
>>>>>
>>>>>
>>>>> Will
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *Will STEVENS*
>>>>>
>>>>> Lead Developer
>>>>>
>>>>>
>>>>>
>>>>> *CloudOps | *Cloud Solutions Experts
>>>>>
>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>>>>
>>>>> w cloudops.com *|* tw @CloudOps_
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com> wrote:
>>>>>
>>>>> Hi Leo,
>>>>> Our new internal build system are using ubuntu 14.04 or something like
>>>>> that, which has qemu 1.x installed by default, that's why I added the
>>>>> "compat" option in the build script, otherwise, the image build by
>>>>> qemu 1.x, won't work on RHEL 6.x.
>>>>> The fix would be, in the build script, check the version of qemu-img,
>>>>> if it's 0.x, then don't add "compat" option. There are a lot of people
>>>>> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
>>>>> we build can still work on these machines.
>>>>>
>>>>>
>>>>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <
>> LSimons@schubergphilis.com>
>>>>> wrote:
>>>>>> Hey Edison, all,
>>>>>>
>>>>>>
>>>>>
>> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
>>>>>>
>>>>>> breaks our internal build with
>>>>>>
>>>>>> + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
>>>>>> systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
>>>>>> Unknown option 'compat'
>>>>>> qemu-img: Invalid options for file format 'qcow2'.
>>>>>>
>>>>>> This is on CentOS release 6.6, which has
>>>>> qemu-img-0.15.0-1.el6.rfx.x86_64.
>>>>>>
>>>>>> Based on
>>>>>>
>>>>>
>> http://secure-web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue6-gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOlderNews
>>>>>> that seems like it is a really old qemu (august 2011).
>>>>>>
>>>>>> I'm guessing you have a newer OS / newer qemu? Can you please let me
>>>>> know what OS, OS version and qemu(-img) version you are using?
>>>>>>
>>>>>> Also, does anyone know if there some minimum version of qemu-img that
>>>>> should be used / cloudstack assumes? Is 0.15 still ok to do an
>> acceptable
>>>>> image conversion with? (we currently don't have any kvm use ourselves,
>> but,
>>>>> I'd like for our build infra to produce useful kvm images nonetheless).
>>>>>>
>>>>>> According to
>>>>>>
>>>>>
>> http://secure-web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6nW6o9mBZ-AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
>>>>>> the -o compat switch was introduced in 1.1.
>>>>>>
>>>>>> According to
>>>>>>
>>>>>
>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>>>>>> the default format was changed from 0.10 to 1.1 in qemu 1.7 and
>> onwards.
>>>>>>
>>>>>> The libvirt people
>>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=997977
>>>>>> say they pass -o compat when qemu supports it (so when v >= 1.1 I
>>>>> imagine).
>>>>>>
>>>>>> I think we should do the same in the build script and I'll make a
>> patch.
>>>>>>
>>>>>> But, should we publish newer-format images too? According to
>>>>>>
>>>>>
>> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-PlYYLhCZX0f768ZYK2fV1WbMFpQPz5JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.org%2FFeatures%2FQcow3
>>>>>> the new format is much better so I imagine qcow/kvm users will really
>>>>> appreciate the newer formats.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>>
>>>>>> cheers,
>>>>>>
>>>>>>
>>>>>> Leo
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Edison
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>
>
--
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Will Stevens <ws...@cloudops.com>.
Interesting. Thanks for that...
So it seems that if you run the build system on a EL7 the version check
approach won't even help us because they have changed the default behavior
of the installed qemu-img version to make it default to '1.1'.
In light of this and considering what a PITA it is to compare version
numbers in bash, I will solve the problem with my original code. I will do
some testing on a local system to work out the kinks and try to validate
that Rohit's concerns are addressed.
I will report back later today.
Thanks for the help on this Nux... :)
Cheers,
Will
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Dec 2, 2014 at 10:27 AM, Nux! <nu...@li.nux.ro> wrote:
> Will, I might have gotten lost in the details, apologies if it's the case.
>
> What I wanted to point out is that the qemu-img in EL7 (v1.5.3 which is
> _lower_ than 1.7) will by default produce an image with compat = 1.1 which
> older qemu-kvm will not be able to use, such as the one in CentOS 6.
> So if you are running a systemvm build job on EL7 you should be aware of
> this. That's why I preferred your script, because it doesn't just rely on
> version numbers.
> I myself am not running any systemvm build jobs.
>
> qemu-img create -f raw test.raw 1G
> qemu-img convert -f raw test.raw -O qcow2 test.qcow2
> qemu-img info test.qcow2
> image: test.qcow2
> file format: qcow2
> virtual size: 1.0G (1073741824 bytes)
> disk size: 196K
> cluster_size: 65536
> Format specific information:
> compat: 1.1
> lazy refcounts: false
>
>
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Will Stevens" <ws...@cloudops.com>
> > To: "Nux!" <nu...@li.nux.ro>
> > Cc: dev@cloudstack.apache.org, "Edison Su" <Ed...@citrix.com>,
> "Wilder Rodrigues" <WR...@schubergphilis.com>,
> > "Leo Simons" <LS...@schubergphilis.com>
> > Sent: Tuesday, 2 December, 2014 15:11:40
> > Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>
> > @Rohit: If the script is run with 'set -e' it should not be a problem
> > because I was planning to do an explicit 'set +e' before the call and
> then
> > a 'set -e' after the call. This should handle this case, no?
> >
> > @Nux: When you say "EL7 comes with qemu-img 1.5.3 and it requires the
> > compat option", what do you mean by that? Are you saying that if you run
> > the build system vms job on EL7, that you have to specify the compat
> option
> > even though it comes with qemu-img 1.5.3? That should not be the case
> > because the default compat flag should not have been changed till 1.7.
> If
> > you are talking about having a KVM host which runs EL7 and has a qemu-img
> > version of 1.5.3, then that is not really relevant for this bash script.
> > The version of qemu-img which this script checking is the version on the
> > build system, not on the KVM host. It is validating if the compat option
> > needs to be passed to the 'qemu-img convert' command in order for the
> image
> > to have the 'compat=0.10' option set. We are not debating that the
> > 'compat=0.10' option needs to be set, that is for sure. We are
> determining
> > when it needs to be explicitly set on the build systems.
> >
> > In short...
> > Jenkins kicks off a job to build the system vms. This build system could
> > have one of many different qemu-img versions. When this script runs on
> the
> > build system, we need to determine which command needs to be run in order
> > to output a 'compat=0.10' compatible image.
> >
> > I will do some testing with Edison's suggestion and see if I can come up
> > with a solution that way as well and then we can pick which we want to
> > implement.
> >
> > Cheers,
> >
> > Will
> >
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Tue, Dec 2, 2014 at 3:32 AM, Nux! <nu...@li.nux.ro> wrote:
> >
> >> As I said above EL7 comes with qemu-img 1.5.3 and it requires the compat
> >> option, so it's not only 1.7+.
> >> Personally I'm in favour of Will's approach.
> >>
> >> Lucian
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >> ----- Original Message -----
> >> > From: "Rohit Yadav" <bh...@apache.org>
> >> > To: "Will Stevens" <ws...@cloudops.com>
> >> > Cc: "Edison Su" <Ed...@citrix.com>, "Wilder Rodrigues" <
> >> WRodrigues@schubergphilis.com>, dev@cloudstack.apache.org,
> >> > "Leo Simons" <LS...@schubergphilis.com>
> >> > Sent: Tuesday, 2 December, 2014 07:36:21
> >> > Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
> >>
> >> > Hi Will,
> >> >
> >> > While we can use your first solution that checks exit status ($?) but
> in
> >> > case some is running the script with a "set -e", the script will
> simply
> >> > break when $? is not zero. I think if we know above which qemu-img
> >> version
> >> > we should we the compat option (1.7?) than that's great we should we
> >> that.
> >> > Thanks for looking into this.
> >> >
> >> > Regards.
> >> >
> >> > On Tue, Dec 2, 2014 at 8:01 AM, Will Stevens <ws...@cloudops.com>
> >> wrote:
> >> >
> >> >> Thanks for that. Does that mean that we prefer to do an if else
> based on
> >> >> version rather than basically doing a try catch?
> >> >>
> >> >> I have not been involved in previous bash scripting for CS, so I am
> >> >> willing to fall in line with the preferred method. What I proposed
> was
> >> >> based on how similar problems were solved in other code in the same
> >> file.
> >> >>
> >> >> If I check for version, do you agree that I should check for 1.7 as
> the
> >> >> version since that is the version where the compact default changed?
> >> >>
> >> >> I will try to get this resolved tomorrow.
> >> >>
> >> >> Cheers,
> >> >>
> >> >> Will
> >> >> On Dec 1, 2014 9:16 PM, "Edison Su" <Ed...@citrix.com> wrote:
> >> >>
> >> >>> qemu-img |head -n 1|awk '{print $3}'
> >> >>>
> >> >>> should show the version of qemu-img
> >> >>>
> >> >>>
> >> >>>
> >> >>> *From:* williamstevens@gmail.com [mailto:williamstevens@gmail.com]
> *On
> >> >>> Behalf Of *Will Stevens
> >> >>> *Sent:* Monday, December 01, 2014 2:17 PM
> >> >>> *To:* dev@cloudstack.apache.org
> >> >>> *Cc:* Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
> >> >>> *Subject:* Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
> >> >>>
> >> >>>
> >> >>>
> >> >>> Alright, so far I have found the following:
> >> >>>
> >>
> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
> >> >>>
> >> >>>
> >> >>>
> >> >>> Basically, if the qemu-img version is less than 1.7, we should run
> the
> >> >>> non 'compat' option version and if the version is 1.7 or greater, we
> >> should
> >> >>> run the new command with the 'compat' version.
> >> >>>
> >> >>>
> >> >>>
> >> >>> Unfortunately I am not able to find a way to get the qemu-img
> version
> >> >>> from the command line.
> >> >>>
> >> >>>
> >> >>>
> >> >>> I am looking to basically add a conditional to try running with the
> >> >>> compat option and if that fails, then run it without the compat
> option.
> >> >>>
> >> >>>
> >> >>>
> >> >>> Basically, I would be replacing this:
> >> >>>
> >> >>>
> >> >>>
> >> >>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> >> >>> "${appliance_build_name}-kvm.qcow2"
> >> >>>
> >> >>>
> >> >>>
> >> >>> With this:
> >> >>>
> >> >>>
> >> >>>
> >> >>> set +e
> >> >>>
> >> >>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> >> >>> "${appliance_build_name}-kvm.qcow2"
> >> >>>
> >> >>> local qemuresult=$?
> >> >>>
> >> >>> set -e
> >> >>>
> >> >>> if [ ${qemuresult} != 0 ]; then
> >> >>>
> >> >>> log INFO "qemu-img convert failed, trying without compat option"
> >> >>>
> >> >>> qemu-img convert -f raw -c -O qcow2 raw.img
> >> >>> "${appliance_build_name}-kvm.qcow2"
> >> >>>
> >> >>> fi
> >> >>>
> >> >>>
> >> >>>
> >> >>> What do you guys think? Is this a good enough solution? If you
> guys
> >> >>> agree I will implement it in master and make sure it works, then we
> can
> >> >>> merge the change back to 4.5 to fix that branch as well.
> >> >>>
> >> >>>
> >> >>>
> >> >>> Let me know if you have issues with this approach...
> >> >>>
> >> >>>
> >> >>>
> >> >>> Cheers,
> >> >>>
> >> >>>
> >> >>>
> >> >>> Will
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> *Will STEVENS*
> >> >>>
> >> >>> Lead Developer
> >> >>>
> >> >>>
> >> >>>
> >> >>> *CloudOps | *Cloud Solutions Experts
> >> >>>
> >> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >> >>>
> >> >>> w cloudops.com *|* tw @CloudOps_
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <wstevens@cloudops.com
> >
> >> >>> wrote:
> >> >>>
> >> >>> Edison,
> >> >>>
> >> >>> According to this page the default compat option is 0.10:
> >> >>> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
> >> >>>
> >> >>>
> >> >>>
> >> >>> Did you find that to not be the case and is that why you had to add
> the
> >> >>> compat option?
> >> >>>
> >> >>>
> >> >>>
> >> >>> BTW, in an attempt to get the master system vms building again, I
> >> >>> committed a change to master to remove the compat option. We now
> have
> >> >>> master system vms building correctly, but Rohit rightly pointed out
> >> that I
> >> >>> had basically reverted your change.
> >> >>>
> >> >>>
> >> >>>
> >> >>> I created a Jira ticket for this issue:
> >> >>> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
> >> >>>
> >> >>>
> >> >>>
> >> >>> I have not reverted my change in master at this point, so it is
> >> building
> >> >>> right now, but I also did not make the change to 4.5, so that
> branch is
> >> >>> currently failing to build system vms.
> >> >>>
> >> >>>
> >> >>>
> >> >>> I will see if I can track down in which version of qemu the compat
> >> option
> >> >>> was added so we can add some intelligent logic around this command.
> >> >>>
> >> >>>
> >> >>>
> >> >>> Will
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> *Will STEVENS*
> >> >>>
> >> >>> Lead Developer
> >> >>>
> >> >>>
> >> >>>
> >> >>> *CloudOps | *Cloud Solutions Experts
> >> >>>
> >> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >> >>>
> >> >>> w cloudops.com *|* tw @CloudOps_
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com>
> wrote:
> >> >>>
> >> >>> Hi Leo,
> >> >>> Our new internal build system are using ubuntu 14.04 or something
> like
> >> >>> that, which has qemu 1.x installed by default, that's why I added
> the
> >> >>> "compat" option in the build script, otherwise, the image build by
> >> >>> qemu 1.x, won't work on RHEL 6.x.
> >> >>> The fix would be, in the build script, check the version of
> qemu-img,
> >> >>> if it's 0.x, then don't add "compat" option. There are a lot of
> people
> >> >>> still using RHEL 6.x as KVM hypervisor, we have to make sure the
> image
> >> >>> we build can still work on these machines.
> >> >>>
> >> >>>
> >> >>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <
> >> LSimons@schubergphilis.com>
> >> >>> wrote:
> >> >>> > Hey Edison, all,
> >> >>> >
> >> >>> >
> >> >>>
> >>
> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
> >> >>> >
> >> >>> > breaks our internal build with
> >> >>> >
> >> >>> > + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
> >> >>> >
> systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
> >> >>> > Unknown option 'compat'
> >> >>> > qemu-img: Invalid options for file format 'qcow2'.
> >> >>> >
> >> >>> > This is on CentOS release 6.6, which has
> >> >>> qemu-img-0.15.0-1.el6.rfx.x86_64.
> >> >>> >
> >> >>> > Based on
> >> >>> >
> >> >>>
> >>
> http://secure-web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue6-gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOlderNews
> >> >>> > that seems like it is a really old qemu (august 2011).
> >> >>> >
> >> >>> > I'm guessing you have a newer OS / newer qemu? Can you please let
> me
> >> >>> know what OS, OS version and qemu(-img) version you are using?
> >> >>> >
> >> >>> > Also, does anyone know if there some minimum version of qemu-img
> that
> >> >>> should be used / cloudstack assumes? Is 0.15 still ok to do an
> >> acceptable
> >> >>> image conversion with? (we currently don't have any kvm use
> ourselves,
> >> but,
> >> >>> I'd like for our build infra to produce useful kvm images
> nonetheless).
> >> >>> >
> >> >>> > According to
> >> >>> >
> >> >>>
> >>
> http://secure-web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6nW6o9mBZ-AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
> >> >>> > the -o compat switch was introduced in 1.1.
> >> >>> >
> >> >>> > According to
> >> >>> >
> >> >>>
> >>
> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
> >> >>> > the default format was changed from 0.10 to 1.1 in qemu 1.7 and
> >> onwards.
> >> >>> >
> >> >>> > The libvirt people
> >> >>> > https://bugzilla.redhat.com/show_bug.cgi?id=997977
> >> >>> > say they pass -o compat when qemu supports it (so when v >= 1.1 I
> >> >>> imagine).
> >> >>> >
> >> >>> > I think we should do the same in the build script and I'll make a
> >> patch.
> >> >>> >
> >> >>> > But, should we publish newer-format images too? According to
> >> >>> >
> >> >>>
> >>
> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-PlYYLhCZX0f768ZYK2fV1WbMFpQPz5JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.org%2FFeatures%2FQcow3
> >> >>> > the new format is much better so I imagine qcow/kvm users will
> really
> >> >>> appreciate the newer formats.
> >> >>> >
> >> >>> > Thoughts?
> >> >>> >
> >> >>> >
> >> >>> > cheers,
> >> >>> >
> >> >>> >
> >> >>> > Leo
> >> >>> >
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Best Regards,
> >> >>> Edison
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
>
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Nux! <nu...@li.nux.ro>.
Will, I might have gotten lost in the details, apologies if it's the case.
What I wanted to point out is that the qemu-img in EL7 (v1.5.3 which is _lower_ than 1.7) will by default produce an image with compat = 1.1 which older qemu-kvm will not be able to use, such as the one in CentOS 6.
So if you are running a systemvm build job on EL7 you should be aware of this. That's why I preferred your script, because it doesn't just rely on version numbers.
I myself am not running any systemvm build jobs.
qemu-img create -f raw test.raw 1G
qemu-img convert -f raw test.raw -O qcow2 test.qcow2
qemu-img info test.qcow2
image: test.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 196K
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Will Stevens" <ws...@cloudops.com>
> To: "Nux!" <nu...@li.nux.ro>
> Cc: dev@cloudstack.apache.org, "Edison Su" <Ed...@citrix.com>, "Wilder Rodrigues" <WR...@schubergphilis.com>,
> "Leo Simons" <LS...@schubergphilis.com>
> Sent: Tuesday, 2 December, 2014 15:11:40
> Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
> @Rohit: If the script is run with 'set -e' it should not be a problem
> because I was planning to do an explicit 'set +e' before the call and then
> a 'set -e' after the call. This should handle this case, no?
>
> @Nux: When you say "EL7 comes with qemu-img 1.5.3 and it requires the
> compat option", what do you mean by that? Are you saying that if you run
> the build system vms job on EL7, that you have to specify the compat option
> even though it comes with qemu-img 1.5.3? That should not be the case
> because the default compat flag should not have been changed till 1.7. If
> you are talking about having a KVM host which runs EL7 and has a qemu-img
> version of 1.5.3, then that is not really relevant for this bash script.
> The version of qemu-img which this script checking is the version on the
> build system, not on the KVM host. It is validating if the compat option
> needs to be passed to the 'qemu-img convert' command in order for the image
> to have the 'compat=0.10' option set. We are not debating that the
> 'compat=0.10' option needs to be set, that is for sure. We are determining
> when it needs to be explicitly set on the build systems.
>
> In short...
> Jenkins kicks off a job to build the system vms. This build system could
> have one of many different qemu-img versions. When this script runs on the
> build system, we need to determine which command needs to be run in order
> to output a 'compat=0.10' compatible image.
>
> I will do some testing with Edison's suggestion and see if I can come up
> with a solution that way as well and then we can pick which we want to
> implement.
>
> Cheers,
>
> Will
>
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Tue, Dec 2, 2014 at 3:32 AM, Nux! <nu...@li.nux.ro> wrote:
>
>> As I said above EL7 comes with qemu-img 1.5.3 and it requires the compat
>> option, so it's not only 1.7+.
>> Personally I'm in favour of Will's approach.
>>
>> Lucian
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> ----- Original Message -----
>> > From: "Rohit Yadav" <bh...@apache.org>
>> > To: "Will Stevens" <ws...@cloudops.com>
>> > Cc: "Edison Su" <Ed...@citrix.com>, "Wilder Rodrigues" <
>> WRodrigues@schubergphilis.com>, dev@cloudstack.apache.org,
>> > "Leo Simons" <LS...@schubergphilis.com>
>> > Sent: Tuesday, 2 December, 2014 07:36:21
>> > Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>>
>> > Hi Will,
>> >
>> > While we can use your first solution that checks exit status ($?) but in
>> > case some is running the script with a "set -e", the script will simply
>> > break when $? is not zero. I think if we know above which qemu-img
>> version
>> > we should we the compat option (1.7?) than that's great we should we
>> that.
>> > Thanks for looking into this.
>> >
>> > Regards.
>> >
>> > On Tue, Dec 2, 2014 at 8:01 AM, Will Stevens <ws...@cloudops.com>
>> wrote:
>> >
>> >> Thanks for that. Does that mean that we prefer to do an if else based on
>> >> version rather than basically doing a try catch?
>> >>
>> >> I have not been involved in previous bash scripting for CS, so I am
>> >> willing to fall in line with the preferred method. What I proposed was
>> >> based on how similar problems were solved in other code in the same
>> file.
>> >>
>> >> If I check for version, do you agree that I should check for 1.7 as the
>> >> version since that is the version where the compact default changed?
>> >>
>> >> I will try to get this resolved tomorrow.
>> >>
>> >> Cheers,
>> >>
>> >> Will
>> >> On Dec 1, 2014 9:16 PM, "Edison Su" <Ed...@citrix.com> wrote:
>> >>
>> >>> qemu-img |head -n 1|awk '{print $3}'
>> >>>
>> >>> should show the version of qemu-img
>> >>>
>> >>>
>> >>>
>> >>> *From:* williamstevens@gmail.com [mailto:williamstevens@gmail.com] *On
>> >>> Behalf Of *Will Stevens
>> >>> *Sent:* Monday, December 01, 2014 2:17 PM
>> >>> *To:* dev@cloudstack.apache.org
>> >>> *Cc:* Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
>> >>> *Subject:* Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>> >>>
>> >>>
>> >>>
>> >>> Alright, so far I have found the following:
>> >>>
>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>> >>>
>> >>>
>> >>>
>> >>> Basically, if the qemu-img version is less than 1.7, we should run the
>> >>> non 'compat' option version and if the version is 1.7 or greater, we
>> should
>> >>> run the new command with the 'compat' version.
>> >>>
>> >>>
>> >>>
>> >>> Unfortunately I am not able to find a way to get the qemu-img version
>> >>> from the command line.
>> >>>
>> >>>
>> >>>
>> >>> I am looking to basically add a conditional to try running with the
>> >>> compat option and if that fails, then run it without the compat option.
>> >>>
>> >>>
>> >>>
>> >>> Basically, I would be replacing this:
>> >>>
>> >>>
>> >>>
>> >>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
>> >>> "${appliance_build_name}-kvm.qcow2"
>> >>>
>> >>>
>> >>>
>> >>> With this:
>> >>>
>> >>>
>> >>>
>> >>> set +e
>> >>>
>> >>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
>> >>> "${appliance_build_name}-kvm.qcow2"
>> >>>
>> >>> local qemuresult=$?
>> >>>
>> >>> set -e
>> >>>
>> >>> if [ ${qemuresult} != 0 ]; then
>> >>>
>> >>> log INFO "qemu-img convert failed, trying without compat option"
>> >>>
>> >>> qemu-img convert -f raw -c -O qcow2 raw.img
>> >>> "${appliance_build_name}-kvm.qcow2"
>> >>>
>> >>> fi
>> >>>
>> >>>
>> >>>
>> >>> What do you guys think? Is this a good enough solution? If you guys
>> >>> agree I will implement it in master and make sure it works, then we can
>> >>> merge the change back to 4.5 to fix that branch as well.
>> >>>
>> >>>
>> >>>
>> >>> Let me know if you have issues with this approach...
>> >>>
>> >>>
>> >>>
>> >>> Cheers,
>> >>>
>> >>>
>> >>>
>> >>> Will
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> *Will STEVENS*
>> >>>
>> >>> Lead Developer
>> >>>
>> >>>
>> >>>
>> >>> *CloudOps | *Cloud Solutions Experts
>> >>>
>> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> >>>
>> >>> w cloudops.com *|* tw @CloudOps_
>> >>>
>> >>>
>> >>>
>> >>> On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <ws...@cloudops.com>
>> >>> wrote:
>> >>>
>> >>> Edison,
>> >>>
>> >>> According to this page the default compat option is 0.10:
>> >>> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
>> >>>
>> >>>
>> >>>
>> >>> Did you find that to not be the case and is that why you had to add the
>> >>> compat option?
>> >>>
>> >>>
>> >>>
>> >>> BTW, in an attempt to get the master system vms building again, I
>> >>> committed a change to master to remove the compat option. We now have
>> >>> master system vms building correctly, but Rohit rightly pointed out
>> that I
>> >>> had basically reverted your change.
>> >>>
>> >>>
>> >>>
>> >>> I created a Jira ticket for this issue:
>> >>> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
>> >>>
>> >>>
>> >>>
>> >>> I have not reverted my change in master at this point, so it is
>> building
>> >>> right now, but I also did not make the change to 4.5, so that branch is
>> >>> currently failing to build system vms.
>> >>>
>> >>>
>> >>>
>> >>> I will see if I can track down in which version of qemu the compat
>> option
>> >>> was added so we can add some intelligent logic around this command.
>> >>>
>> >>>
>> >>>
>> >>> Will
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> *Will STEVENS*
>> >>>
>> >>> Lead Developer
>> >>>
>> >>>
>> >>>
>> >>> *CloudOps | *Cloud Solutions Experts
>> >>>
>> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> >>>
>> >>> w cloudops.com *|* tw @CloudOps_
>> >>>
>> >>>
>> >>>
>> >>> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com> wrote:
>> >>>
>> >>> Hi Leo,
>> >>> Our new internal build system are using ubuntu 14.04 or something like
>> >>> that, which has qemu 1.x installed by default, that's why I added the
>> >>> "compat" option in the build script, otherwise, the image build by
>> >>> qemu 1.x, won't work on RHEL 6.x.
>> >>> The fix would be, in the build script, check the version of qemu-img,
>> >>> if it's 0.x, then don't add "compat" option. There are a lot of people
>> >>> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
>> >>> we build can still work on these machines.
>> >>>
>> >>>
>> >>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <
>> LSimons@schubergphilis.com>
>> >>> wrote:
>> >>> > Hey Edison, all,
>> >>> >
>> >>> >
>> >>>
>> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
>> >>> >
>> >>> > breaks our internal build with
>> >>> >
>> >>> > + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
>> >>> > systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
>> >>> > Unknown option 'compat'
>> >>> > qemu-img: Invalid options for file format 'qcow2'.
>> >>> >
>> >>> > This is on CentOS release 6.6, which has
>> >>> qemu-img-0.15.0-1.el6.rfx.x86_64.
>> >>> >
>> >>> > Based on
>> >>> >
>> >>>
>> http://secure-web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue6-gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOlderNews
>> >>> > that seems like it is a really old qemu (august 2011).
>> >>> >
>> >>> > I'm guessing you have a newer OS / newer qemu? Can you please let me
>> >>> know what OS, OS version and qemu(-img) version you are using?
>> >>> >
>> >>> > Also, does anyone know if there some minimum version of qemu-img that
>> >>> should be used / cloudstack assumes? Is 0.15 still ok to do an
>> acceptable
>> >>> image conversion with? (we currently don't have any kvm use ourselves,
>> but,
>> >>> I'd like for our build infra to produce useful kvm images nonetheless).
>> >>> >
>> >>> > According to
>> >>> >
>> >>>
>> http://secure-web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6nW6o9mBZ-AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
>> >>> > the -o compat switch was introduced in 1.1.
>> >>> >
>> >>> > According to
>> >>> >
>> >>>
>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>> >>> > the default format was changed from 0.10 to 1.1 in qemu 1.7 and
>> onwards.
>> >>> >
>> >>> > The libvirt people
>> >>> > https://bugzilla.redhat.com/show_bug.cgi?id=997977
>> >>> > say they pass -o compat when qemu supports it (so when v >= 1.1 I
>> >>> imagine).
>> >>> >
>> >>> > I think we should do the same in the build script and I'll make a
>> patch.
>> >>> >
>> >>> > But, should we publish newer-format images too? According to
>> >>> >
>> >>>
>> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-PlYYLhCZX0f768ZYK2fV1WbMFpQPz5JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.org%2FFeatures%2FQcow3
>> >>> > the new format is much better so I imagine qcow/kvm users will really
>> >>> appreciate the newer formats.
>> >>> >
>> >>> > Thoughts?
>> >>> >
>> >>> >
>> >>> > cheers,
>> >>> >
>> >>> >
>> >>> > Leo
>> >>> >
>> >>>
>> >>>
>> >>> --
>> >>> Best Regards,
>> >>> Edison
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Will Stevens <ws...@cloudops.com>.
@Rohit: If the script is run with 'set -e' it should not be a problem
because I was planning to do an explicit 'set +e' before the call and then
a 'set -e' after the call. This should handle this case, no?
@Nux: When you say "EL7 comes with qemu-img 1.5.3 and it requires the
compat option", what do you mean by that? Are you saying that if you run
the build system vms job on EL7, that you have to specify the compat option
even though it comes with qemu-img 1.5.3? That should not be the case
because the default compat flag should not have been changed till 1.7. If
you are talking about having a KVM host which runs EL7 and has a qemu-img
version of 1.5.3, then that is not really relevant for this bash script.
The version of qemu-img which this script checking is the version on the
build system, not on the KVM host. It is validating if the compat option
needs to be passed to the 'qemu-img convert' command in order for the image
to have the 'compat=0.10' option set. We are not debating that the
'compat=0.10' option needs to be set, that is for sure. We are determining
when it needs to be explicitly set on the build systems.
In short...
Jenkins kicks off a job to build the system vms. This build system could
have one of many different qemu-img versions. When this script runs on the
build system, we need to determine which command needs to be run in order
to output a 'compat=0.10' compatible image.
I will do some testing with Edison's suggestion and see if I can come up
with a solution that way as well and then we can pick which we want to
implement.
Cheers,
Will
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Tue, Dec 2, 2014 at 3:32 AM, Nux! <nu...@li.nux.ro> wrote:
> As I said above EL7 comes with qemu-img 1.5.3 and it requires the compat
> option, so it's not only 1.7+.
> Personally I'm in favour of Will's approach.
>
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -----
> > From: "Rohit Yadav" <bh...@apache.org>
> > To: "Will Stevens" <ws...@cloudops.com>
> > Cc: "Edison Su" <Ed...@citrix.com>, "Wilder Rodrigues" <
> WRodrigues@schubergphilis.com>, dev@cloudstack.apache.org,
> > "Leo Simons" <LS...@schubergphilis.com>
> > Sent: Tuesday, 2 December, 2014 07:36:21
> > Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>
> > Hi Will,
> >
> > While we can use your first solution that checks exit status ($?) but in
> > case some is running the script with a "set -e", the script will simply
> > break when $? is not zero. I think if we know above which qemu-img
> version
> > we should we the compat option (1.7?) than that's great we should we
> that.
> > Thanks for looking into this.
> >
> > Regards.
> >
> > On Tue, Dec 2, 2014 at 8:01 AM, Will Stevens <ws...@cloudops.com>
> wrote:
> >
> >> Thanks for that. Does that mean that we prefer to do an if else based on
> >> version rather than basically doing a try catch?
> >>
> >> I have not been involved in previous bash scripting for CS, so I am
> >> willing to fall in line with the preferred method. What I proposed was
> >> based on how similar problems were solved in other code in the same
> file.
> >>
> >> If I check for version, do you agree that I should check for 1.7 as the
> >> version since that is the version where the compact default changed?
> >>
> >> I will try to get this resolved tomorrow.
> >>
> >> Cheers,
> >>
> >> Will
> >> On Dec 1, 2014 9:16 PM, "Edison Su" <Ed...@citrix.com> wrote:
> >>
> >>> qemu-img |head -n 1|awk '{print $3}'
> >>>
> >>> should show the version of qemu-img
> >>>
> >>>
> >>>
> >>> *From:* williamstevens@gmail.com [mailto:williamstevens@gmail.com] *On
> >>> Behalf Of *Will Stevens
> >>> *Sent:* Monday, December 01, 2014 2:17 PM
> >>> *To:* dev@cloudstack.apache.org
> >>> *Cc:* Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
> >>> *Subject:* Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
> >>>
> >>>
> >>>
> >>> Alright, so far I have found the following:
> >>>
> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
> >>>
> >>>
> >>>
> >>> Basically, if the qemu-img version is less than 1.7, we should run the
> >>> non 'compat' option version and if the version is 1.7 or greater, we
> should
> >>> run the new command with the 'compat' version.
> >>>
> >>>
> >>>
> >>> Unfortunately I am not able to find a way to get the qemu-img version
> >>> from the command line.
> >>>
> >>>
> >>>
> >>> I am looking to basically add a conditional to try running with the
> >>> compat option and if that fails, then run it without the compat option.
> >>>
> >>>
> >>>
> >>> Basically, I would be replacing this:
> >>>
> >>>
> >>>
> >>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> >>> "${appliance_build_name}-kvm.qcow2"
> >>>
> >>>
> >>>
> >>> With this:
> >>>
> >>>
> >>>
> >>> set +e
> >>>
> >>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> >>> "${appliance_build_name}-kvm.qcow2"
> >>>
> >>> local qemuresult=$?
> >>>
> >>> set -e
> >>>
> >>> if [ ${qemuresult} != 0 ]; then
> >>>
> >>> log INFO "qemu-img convert failed, trying without compat option"
> >>>
> >>> qemu-img convert -f raw -c -O qcow2 raw.img
> >>> "${appliance_build_name}-kvm.qcow2"
> >>>
> >>> fi
> >>>
> >>>
> >>>
> >>> What do you guys think? Is this a good enough solution? If you guys
> >>> agree I will implement it in master and make sure it works, then we can
> >>> merge the change back to 4.5 to fix that branch as well.
> >>>
> >>>
> >>>
> >>> Let me know if you have issues with this approach...
> >>>
> >>>
> >>>
> >>> Cheers,
> >>>
> >>>
> >>>
> >>> Will
> >>>
> >>>
> >>>
> >>>
> >>> *Will STEVENS*
> >>>
> >>> Lead Developer
> >>>
> >>>
> >>>
> >>> *CloudOps | *Cloud Solutions Experts
> >>>
> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >>>
> >>> w cloudops.com *|* tw @CloudOps_
> >>>
> >>>
> >>>
> >>> On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <ws...@cloudops.com>
> >>> wrote:
> >>>
> >>> Edison,
> >>>
> >>> According to this page the default compat option is 0.10:
> >>> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
> >>>
> >>>
> >>>
> >>> Did you find that to not be the case and is that why you had to add the
> >>> compat option?
> >>>
> >>>
> >>>
> >>> BTW, in an attempt to get the master system vms building again, I
> >>> committed a change to master to remove the compat option. We now have
> >>> master system vms building correctly, but Rohit rightly pointed out
> that I
> >>> had basically reverted your change.
> >>>
> >>>
> >>>
> >>> I created a Jira ticket for this issue:
> >>> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
> >>>
> >>>
> >>>
> >>> I have not reverted my change in master at this point, so it is
> building
> >>> right now, but I also did not make the change to 4.5, so that branch is
> >>> currently failing to build system vms.
> >>>
> >>>
> >>>
> >>> I will see if I can track down in which version of qemu the compat
> option
> >>> was added so we can add some intelligent logic around this command.
> >>>
> >>>
> >>>
> >>> Will
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> *Will STEVENS*
> >>>
> >>> Lead Developer
> >>>
> >>>
> >>>
> >>> *CloudOps | *Cloud Solutions Experts
> >>>
> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >>>
> >>> w cloudops.com *|* tw @CloudOps_
> >>>
> >>>
> >>>
> >>> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com> wrote:
> >>>
> >>> Hi Leo,
> >>> Our new internal build system are using ubuntu 14.04 or something like
> >>> that, which has qemu 1.x installed by default, that's why I added the
> >>> "compat" option in the build script, otherwise, the image build by
> >>> qemu 1.x, won't work on RHEL 6.x.
> >>> The fix would be, in the build script, check the version of qemu-img,
> >>> if it's 0.x, then don't add "compat" option. There are a lot of people
> >>> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
> >>> we build can still work on these machines.
> >>>
> >>>
> >>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <
> LSimons@schubergphilis.com>
> >>> wrote:
> >>> > Hey Edison, all,
> >>> >
> >>> >
> >>>
> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
> >>> >
> >>> > breaks our internal build with
> >>> >
> >>> > + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
> >>> > systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
> >>> > Unknown option 'compat'
> >>> > qemu-img: Invalid options for file format 'qcow2'.
> >>> >
> >>> > This is on CentOS release 6.6, which has
> >>> qemu-img-0.15.0-1.el6.rfx.x86_64.
> >>> >
> >>> > Based on
> >>> >
> >>>
> http://secure-web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue6-gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOlderNews
> >>> > that seems like it is a really old qemu (august 2011).
> >>> >
> >>> > I'm guessing you have a newer OS / newer qemu? Can you please let me
> >>> know what OS, OS version and qemu(-img) version you are using?
> >>> >
> >>> > Also, does anyone know if there some minimum version of qemu-img that
> >>> should be used / cloudstack assumes? Is 0.15 still ok to do an
> acceptable
> >>> image conversion with? (we currently don't have any kvm use ourselves,
> but,
> >>> I'd like for our build infra to produce useful kvm images nonetheless).
> >>> >
> >>> > According to
> >>> >
> >>>
> http://secure-web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6nW6o9mBZ-AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
> >>> > the -o compat switch was introduced in 1.1.
> >>> >
> >>> > According to
> >>> >
> >>>
> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
> >>> > the default format was changed from 0.10 to 1.1 in qemu 1.7 and
> onwards.
> >>> >
> >>> > The libvirt people
> >>> > https://bugzilla.redhat.com/show_bug.cgi?id=997977
> >>> > say they pass -o compat when qemu supports it (so when v >= 1.1 I
> >>> imagine).
> >>> >
> >>> > I think we should do the same in the build script and I'll make a
> patch.
> >>> >
> >>> > But, should we publish newer-format images too? According to
> >>> >
> >>>
> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-PlYYLhCZX0f768ZYK2fV1WbMFpQPz5JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.org%2FFeatures%2FQcow3
> >>> > the new format is much better so I imagine qcow/kvm users will really
> >>> appreciate the newer formats.
> >>> >
> >>> > Thoughts?
> >>> >
> >>> >
> >>> > cheers,
> >>> >
> >>> >
> >>> > Leo
> >>> >
> >>>
> >>>
> >>> --
> >>> Best Regards,
> >>> Edison
> >>>
> >>>
> >>>
> >>>
> >>>
>
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Nux! <nu...@li.nux.ro>.
As I said above EL7 comes with qemu-img 1.5.3 and it requires the compat option, so it's not only 1.7+.
Personally I'm in favour of Will's approach.
Lucian
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Rohit Yadav" <bh...@apache.org>
> To: "Will Stevens" <ws...@cloudops.com>
> Cc: "Edison Su" <Ed...@citrix.com>, "Wilder Rodrigues" <WR...@schubergphilis.com>, dev@cloudstack.apache.org,
> "Leo Simons" <LS...@schubergphilis.com>
> Sent: Tuesday, 2 December, 2014 07:36:21
> Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
> Hi Will,
>
> While we can use your first solution that checks exit status ($?) but in
> case some is running the script with a "set -e", the script will simply
> break when $? is not zero. I think if we know above which qemu-img version
> we should we the compat option (1.7?) than that's great we should we that.
> Thanks for looking into this.
>
> Regards.
>
> On Tue, Dec 2, 2014 at 8:01 AM, Will Stevens <ws...@cloudops.com> wrote:
>
>> Thanks for that. Does that mean that we prefer to do an if else based on
>> version rather than basically doing a try catch?
>>
>> I have not been involved in previous bash scripting for CS, so I am
>> willing to fall in line with the preferred method. What I proposed was
>> based on how similar problems were solved in other code in the same file.
>>
>> If I check for version, do you agree that I should check for 1.7 as the
>> version since that is the version where the compact default changed?
>>
>> I will try to get this resolved tomorrow.
>>
>> Cheers,
>>
>> Will
>> On Dec 1, 2014 9:16 PM, "Edison Su" <Ed...@citrix.com> wrote:
>>
>>> qemu-img |head -n 1|awk '{print $3}'
>>>
>>> should show the version of qemu-img
>>>
>>>
>>>
>>> *From:* williamstevens@gmail.com [mailto:williamstevens@gmail.com] *On
>>> Behalf Of *Will Stevens
>>> *Sent:* Monday, December 01, 2014 2:17 PM
>>> *To:* dev@cloudstack.apache.org
>>> *Cc:* Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
>>> *Subject:* Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>>>
>>>
>>>
>>> Alright, so far I have found the following:
>>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>>>
>>>
>>>
>>> Basically, if the qemu-img version is less than 1.7, we should run the
>>> non 'compat' option version and if the version is 1.7 or greater, we should
>>> run the new command with the 'compat' version.
>>>
>>>
>>>
>>> Unfortunately I am not able to find a way to get the qemu-img version
>>> from the command line.
>>>
>>>
>>>
>>> I am looking to basically add a conditional to try running with the
>>> compat option and if that fails, then run it without the compat option.
>>>
>>>
>>>
>>> Basically, I would be replacing this:
>>>
>>>
>>>
>>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
>>> "${appliance_build_name}-kvm.qcow2"
>>>
>>>
>>>
>>> With this:
>>>
>>>
>>>
>>> set +e
>>>
>>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
>>> "${appliance_build_name}-kvm.qcow2"
>>>
>>> local qemuresult=$?
>>>
>>> set -e
>>>
>>> if [ ${qemuresult} != 0 ]; then
>>>
>>> log INFO "qemu-img convert failed, trying without compat option"
>>>
>>> qemu-img convert -f raw -c -O qcow2 raw.img
>>> "${appliance_build_name}-kvm.qcow2"
>>>
>>> fi
>>>
>>>
>>>
>>> What do you guys think? Is this a good enough solution? If you guys
>>> agree I will implement it in master and make sure it works, then we can
>>> merge the change back to 4.5 to fix that branch as well.
>>>
>>>
>>>
>>> Let me know if you have issues with this approach...
>>>
>>>
>>>
>>> Cheers,
>>>
>>>
>>>
>>> Will
>>>
>>>
>>>
>>>
>>> *Will STEVENS*
>>>
>>> Lead Developer
>>>
>>>
>>>
>>> *CloudOps | *Cloud Solutions Experts
>>>
>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>>
>>> w cloudops.com *|* tw @CloudOps_
>>>
>>>
>>>
>>> On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <ws...@cloudops.com>
>>> wrote:
>>>
>>> Edison,
>>>
>>> According to this page the default compat option is 0.10:
>>> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
>>>
>>>
>>>
>>> Did you find that to not be the case and is that why you had to add the
>>> compat option?
>>>
>>>
>>>
>>> BTW, in an attempt to get the master system vms building again, I
>>> committed a change to master to remove the compat option. We now have
>>> master system vms building correctly, but Rohit rightly pointed out that I
>>> had basically reverted your change.
>>>
>>>
>>>
>>> I created a Jira ticket for this issue:
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
>>>
>>>
>>>
>>> I have not reverted my change in master at this point, so it is building
>>> right now, but I also did not make the change to 4.5, so that branch is
>>> currently failing to build system vms.
>>>
>>>
>>>
>>> I will see if I can track down in which version of qemu the compat option
>>> was added so we can add some intelligent logic around this command.
>>>
>>>
>>>
>>> Will
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *Will STEVENS*
>>>
>>> Lead Developer
>>>
>>>
>>>
>>> *CloudOps | *Cloud Solutions Experts
>>>
>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>>
>>> w cloudops.com *|* tw @CloudOps_
>>>
>>>
>>>
>>> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com> wrote:
>>>
>>> Hi Leo,
>>> Our new internal build system are using ubuntu 14.04 or something like
>>> that, which has qemu 1.x installed by default, that's why I added the
>>> "compat" option in the build script, otherwise, the image build by
>>> qemu 1.x, won't work on RHEL 6.x.
>>> The fix would be, in the build script, check the version of qemu-img,
>>> if it's 0.x, then don't add "compat" option. There are a lot of people
>>> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
>>> we build can still work on these machines.
>>>
>>>
>>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <LS...@schubergphilis.com>
>>> wrote:
>>> > Hey Edison, all,
>>> >
>>> >
>>> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
>>> >
>>> > breaks our internal build with
>>> >
>>> > + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
>>> > systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
>>> > Unknown option 'compat'
>>> > qemu-img: Invalid options for file format 'qcow2'.
>>> >
>>> > This is on CentOS release 6.6, which has
>>> qemu-img-0.15.0-1.el6.rfx.x86_64.
>>> >
>>> > Based on
>>> >
>>> http://secure-web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue6-gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOlderNews
>>> > that seems like it is a really old qemu (august 2011).
>>> >
>>> > I'm guessing you have a newer OS / newer qemu? Can you please let me
>>> know what OS, OS version and qemu(-img) version you are using?
>>> >
>>> > Also, does anyone know if there some minimum version of qemu-img that
>>> should be used / cloudstack assumes? Is 0.15 still ok to do an acceptable
>>> image conversion with? (we currently don't have any kvm use ourselves, but,
>>> I'd like for our build infra to produce useful kvm images nonetheless).
>>> >
>>> > According to
>>> >
>>> http://secure-web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6nW6o9mBZ-AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
>>> > the -o compat switch was introduced in 1.1.
>>> >
>>> > According to
>>> >
>>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>>> > the default format was changed from 0.10 to 1.1 in qemu 1.7 and onwards.
>>> >
>>> > The libvirt people
>>> > https://bugzilla.redhat.com/show_bug.cgi?id=997977
>>> > say they pass -o compat when qemu supports it (so when v >= 1.1 I
>>> imagine).
>>> >
>>> > I think we should do the same in the build script and I'll make a patch.
>>> >
>>> > But, should we publish newer-format images too? According to
>>> >
>>> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-PlYYLhCZX0f768ZYK2fV1WbMFpQPz5JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.org%2FFeatures%2FQcow3
>>> > the new format is much better so I imagine qcow/kvm users will really
>>> appreciate the newer formats.
>>> >
>>> > Thoughts?
>>> >
>>> >
>>> > cheers,
>>> >
>>> >
>>> > Leo
>>> >
>>>
>>>
>>> --
>>> Best Regards,
>>> Edison
>>>
>>>
>>>
>>>
>>>
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Rohit Yadav <bh...@apache.org>.
Hi Will,
While we can use your first solution that checks exit status ($?) but in
case some is running the script with a "set -e", the script will simply
break when $? is not zero. I think if we know above which qemu-img version
we should we the compat option (1.7?) than that's great we should we that.
Thanks for looking into this.
Regards.
On Tue, Dec 2, 2014 at 8:01 AM, Will Stevens <ws...@cloudops.com> wrote:
> Thanks for that. Does that mean that we prefer to do an if else based on
> version rather than basically doing a try catch?
>
> I have not been involved in previous bash scripting for CS, so I am
> willing to fall in line with the preferred method. What I proposed was
> based on how similar problems were solved in other code in the same file.
>
> If I check for version, do you agree that I should check for 1.7 as the
> version since that is the version where the compact default changed?
>
> I will try to get this resolved tomorrow.
>
> Cheers,
>
> Will
> On Dec 1, 2014 9:16 PM, "Edison Su" <Ed...@citrix.com> wrote:
>
>> qemu-img |head -n 1|awk '{print $3}'
>>
>> should show the version of qemu-img
>>
>>
>>
>> *From:* williamstevens@gmail.com [mailto:williamstevens@gmail.com] *On
>> Behalf Of *Will Stevens
>> *Sent:* Monday, December 01, 2014 2:17 PM
>> *To:* dev@cloudstack.apache.org
>> *Cc:* Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
>> *Subject:* Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>>
>>
>>
>> Alright, so far I have found the following:
>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>>
>>
>>
>> Basically, if the qemu-img version is less than 1.7, we should run the
>> non 'compat' option version and if the version is 1.7 or greater, we should
>> run the new command with the 'compat' version.
>>
>>
>>
>> Unfortunately I am not able to find a way to get the qemu-img version
>> from the command line.
>>
>>
>>
>> I am looking to basically add a conditional to try running with the
>> compat option and if that fails, then run it without the compat option.
>>
>>
>>
>> Basically, I would be replacing this:
>>
>>
>>
>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
>> "${appliance_build_name}-kvm.qcow2"
>>
>>
>>
>> With this:
>>
>>
>>
>> set +e
>>
>> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
>> "${appliance_build_name}-kvm.qcow2"
>>
>> local qemuresult=$?
>>
>> set -e
>>
>> if [ ${qemuresult} != 0 ]; then
>>
>> log INFO "qemu-img convert failed, trying without compat option"
>>
>> qemu-img convert -f raw -c -O qcow2 raw.img
>> "${appliance_build_name}-kvm.qcow2"
>>
>> fi
>>
>>
>>
>> What do you guys think? Is this a good enough solution? If you guys
>> agree I will implement it in master and make sure it works, then we can
>> merge the change back to 4.5 to fix that branch as well.
>>
>>
>>
>> Let me know if you have issues with this approach...
>>
>>
>>
>> Cheers,
>>
>>
>>
>> Will
>>
>>
>>
>>
>> *Will STEVENS*
>>
>> Lead Developer
>>
>>
>>
>> *CloudOps | *Cloud Solutions Experts
>>
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>
>> w cloudops.com *|* tw @CloudOps_
>>
>>
>>
>> On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <ws...@cloudops.com>
>> wrote:
>>
>> Edison,
>>
>> According to this page the default compat option is 0.10:
>> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
>>
>>
>>
>> Did you find that to not be the case and is that why you had to add the
>> compat option?
>>
>>
>>
>> BTW, in an attempt to get the master system vms building again, I
>> committed a change to master to remove the compat option. We now have
>> master system vms building correctly, but Rohit rightly pointed out that I
>> had basically reverted your change.
>>
>>
>>
>> I created a Jira ticket for this issue:
>> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
>>
>>
>>
>> I have not reverted my change in master at this point, so it is building
>> right now, but I also did not make the change to 4.5, so that branch is
>> currently failing to build system vms.
>>
>>
>>
>> I will see if I can track down in which version of qemu the compat option
>> was added so we can add some intelligent logic around this command.
>>
>>
>>
>> Will
>>
>>
>>
>>
>>
>>
>>
>>
>> *Will STEVENS*
>>
>> Lead Developer
>>
>>
>>
>> *CloudOps | *Cloud Solutions Experts
>>
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>>
>> w cloudops.com *|* tw @CloudOps_
>>
>>
>>
>> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com> wrote:
>>
>> Hi Leo,
>> Our new internal build system are using ubuntu 14.04 or something like
>> that, which has qemu 1.x installed by default, that's why I added the
>> "compat" option in the build script, otherwise, the image build by
>> qemu 1.x, won't work on RHEL 6.x.
>> The fix would be, in the build script, check the version of qemu-img,
>> if it's 0.x, then don't add "compat" option. There are a lot of people
>> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
>> we build can still work on these machines.
>>
>>
>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <LS...@schubergphilis.com>
>> wrote:
>> > Hey Edison, all,
>> >
>> >
>> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
>> >
>> > breaks our internal build with
>> >
>> > + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
>> > systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
>> > Unknown option 'compat'
>> > qemu-img: Invalid options for file format 'qcow2'.
>> >
>> > This is on CentOS release 6.6, which has
>> qemu-img-0.15.0-1.el6.rfx.x86_64.
>> >
>> > Based on
>> >
>> http://secure-web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue6-gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOlderNews
>> > that seems like it is a really old qemu (august 2011).
>> >
>> > I'm guessing you have a newer OS / newer qemu? Can you please let me
>> know what OS, OS version and qemu(-img) version you are using?
>> >
>> > Also, does anyone know if there some minimum version of qemu-img that
>> should be used / cloudstack assumes? Is 0.15 still ok to do an acceptable
>> image conversion with? (we currently don't have any kvm use ourselves, but,
>> I'd like for our build infra to produce useful kvm images nonetheless).
>> >
>> > According to
>> >
>> http://secure-web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6nW6o9mBZ-AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
>> > the -o compat switch was introduced in 1.1.
>> >
>> > According to
>> >
>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>> > the default format was changed from 0.10 to 1.1 in qemu 1.7 and onwards.
>> >
>> > The libvirt people
>> > https://bugzilla.redhat.com/show_bug.cgi?id=997977
>> > say they pass -o compat when qemu supports it (so when v >= 1.1 I
>> imagine).
>> >
>> > I think we should do the same in the build script and I'll make a patch.
>> >
>> > But, should we publish newer-format images too? According to
>> >
>> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-PlYYLhCZX0f768ZYK2fV1WbMFpQPz5JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.org%2FFeatures%2FQcow3
>> > the new format is much better so I imagine qcow/kvm users will really
>> appreciate the newer formats.
>> >
>> > Thoughts?
>> >
>> >
>> > cheers,
>> >
>> >
>> > Leo
>> >
>>
>>
>> --
>> Best Regards,
>> Edison
>>
>>
>>
>>
>>
>
RE: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Will Stevens <ws...@cloudops.com>.
Thanks for that. Does that mean that we prefer to do an if else based on
version rather than basically doing a try catch?
I have not been involved in previous bash scripting for CS, so I am willing
to fall in line with the preferred method. What I proposed was based on how
similar problems were solved in other code in the same file.
If I check for version, do you agree that I should check for 1.7 as the
version since that is the version where the compact default changed?
I will try to get this resolved tomorrow.
Cheers,
Will
On Dec 1, 2014 9:16 PM, "Edison Su" <Ed...@citrix.com> wrote:
> qemu-img |head -n 1|awk '{print $3}'
>
> should show the version of qemu-img
>
>
>
> *From:* williamstevens@gmail.com [mailto:williamstevens@gmail.com] *On
> Behalf Of *Will Stevens
> *Sent:* Monday, December 01, 2014 2:17 PM
> *To:* dev@cloudstack.apache.org
> *Cc:* Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
> *Subject:* Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
>
>
>
> Alright, so far I have found the following:
> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>
>
>
> Basically, if the qemu-img version is less than 1.7, we should run the non
> 'compat' option version and if the version is 1.7 or greater, we should run
> the new command with the 'compat' version.
>
>
>
> Unfortunately I am not able to find a way to get the qemu-img version from
> the command line.
>
>
>
> I am looking to basically add a conditional to try running with the compat
> option and if that fails, then run it without the compat option.
>
>
>
> Basically, I would be replacing this:
>
>
>
> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> "${appliance_build_name}-kvm.qcow2"
>
>
>
> With this:
>
>
>
> set +e
>
> qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
> "${appliance_build_name}-kvm.qcow2"
>
> local qemuresult=$?
>
> set -e
>
> if [ ${qemuresult} != 0 ]; then
>
> log INFO "qemu-img convert failed, trying without compat option"
>
> qemu-img convert -f raw -c -O qcow2 raw.img
> "${appliance_build_name}-kvm.qcow2"
>
> fi
>
>
>
> What do you guys think? Is this a good enough solution? If you guys
> agree I will implement it in master and make sure it works, then we can
> merge the change back to 4.5 to fix that branch as well.
>
>
>
> Let me know if you have issues with this approach...
>
>
>
> Cheers,
>
>
>
> Will
>
>
>
>
> *Will STEVENS*
>
> Lead Developer
>
>
>
> *CloudOps | *Cloud Solutions Experts
>
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>
> w cloudops.com *|* tw @CloudOps_
>
>
>
> On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> Edison,
>
> According to this page the default compat option is 0.10:
> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
>
>
>
> Did you find that to not be the case and is that why you had to add the
> compat option?
>
>
>
> BTW, in an attempt to get the master system vms building again, I
> committed a change to master to remove the compat option. We now have
> master system vms building correctly, but Rohit rightly pointed out that I
> had basically reverted your change.
>
>
>
> I created a Jira ticket for this issue:
> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
>
>
>
> I have not reverted my change in master at this point, so it is building
> right now, but I also did not make the change to 4.5, so that branch is
> currently failing to build system vms.
>
>
>
> I will see if I can track down in which version of qemu the compat option
> was added so we can add some intelligent logic around this command.
>
>
>
> Will
>
>
>
>
>
>
>
>
> *Will STEVENS*
>
> Lead Developer
>
>
>
> *CloudOps | *Cloud Solutions Experts
>
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>
> w cloudops.com *|* tw @CloudOps_
>
>
>
> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com> wrote:
>
> Hi Leo,
> Our new internal build system are using ubuntu 14.04 or something like
> that, which has qemu 1.x installed by default, that's why I added the
> "compat" option in the build script, otherwise, the image build by
> qemu 1.x, won't work on RHEL 6.x.
> The fix would be, in the build script, check the version of qemu-img,
> if it's 0.x, then don't add "compat" option. There are a lot of people
> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
> we build can still work on these machines.
>
>
> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <LS...@schubergphilis.com>
> wrote:
> > Hey Edison, all,
> >
> >
> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
> >
> > breaks our internal build with
> >
> > + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
> > systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
> > Unknown option 'compat'
> > qemu-img: Invalid options for file format 'qcow2'.
> >
> > This is on CentOS release 6.6, which has
> qemu-img-0.15.0-1.el6.rfx.x86_64.
> >
> > Based on
> >
> http://secure-web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue6-gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOlderNews
> > that seems like it is a really old qemu (august 2011).
> >
> > I'm guessing you have a newer OS / newer qemu? Can you please let me
> know what OS, OS version and qemu(-img) version you are using?
> >
> > Also, does anyone know if there some minimum version of qemu-img that
> should be used / cloudstack assumes? Is 0.15 still ok to do an acceptable
> image conversion with? (we currently don't have any kvm use ourselves, but,
> I'd like for our build infra to produce useful kvm images nonetheless).
> >
> > According to
> >
> http://secure-web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6nW6o9mBZ-AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
> > the -o compat switch was introduced in 1.1.
> >
> > According to
> >
> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
> > the default format was changed from 0.10 to 1.1 in qemu 1.7 and onwards.
> >
> > The libvirt people
> > https://bugzilla.redhat.com/show_bug.cgi?id=997977
> > say they pass -o compat when qemu supports it (so when v >= 1.1 I
> imagine).
> >
> > I think we should do the same in the build script and I'll make a patch.
> >
> > But, should we publish newer-format images too? According to
> >
> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-PlYYLhCZX0f768ZYK2fV1WbMFpQPz5JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.org%2FFeatures%2FQcow3
> > the new format is much better so I imagine qcow/kvm users will really
> appreciate the newer formats.
> >
> > Thoughts?
> >
> >
> > cheers,
> >
> >
> > Leo
> >
>
>
> --
> Best Regards,
> Edison
>
>
>
>
>
RE: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Edison Su <Ed...@citrix.com>.
qemu-img |head -n 1|awk '{print $3}'
should show the version of qemu-img
From: williamstevens@gmail.com [mailto:williamstevens@gmail.com] On Behalf Of Will Stevens
Sent: Monday, December 01, 2014 2:17 PM
To: dev@cloudstack.apache.org
Cc: Leo Simons; Wilder Rodrigues; Rohit Yadav; Edison Su
Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Alright, so far I have found the following: https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
Basically, if the qemu-img version is less than 1.7, we should run the non 'compat' option version and if the version is 1.7 or greater, we should run the new command with the 'compat' version.
Unfortunately I am not able to find a way to get the qemu-img version from the command line.
I am looking to basically add a conditional to try running with the compat option and if that fails, then run it without the compat option.
Basically, I would be replacing this:
qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img "${appliance_build_name}-kvm.qcow2"
With this:
set +e
qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img "${appliance_build_name}-kvm.qcow2"
local qemuresult=$?
set -e
if [ ${qemuresult} != 0 ]; then
log INFO "qemu-img convert failed, trying without compat option"
qemu-img convert -f raw -c -O qcow2 raw.img "${appliance_build_name}-kvm.qcow2"
fi
What do you guys think? Is this a good enough solution? If you guys agree I will implement it in master and make sure it works, then we can merge the change back to 4.5 to fix that branch as well.
Let me know if you have issues with this approach...
Cheers,
Will
Will STEVENS
Lead Developer
CloudOps | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w cloudops.com<http://cloudops.com/> | tw @CloudOps_
On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <ws...@cloudops.com>> wrote:
Edison,
According to this page the default compat option is 0.10: http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
Did you find that to not be the case and is that why you had to add the compat option?
BTW, in an attempt to get the master system vms building again, I committed a change to master to remove the compat option. We now have master system vms building correctly, but Rohit rightly pointed out that I had basically reverted your change.
I created a Jira ticket for this issue: https://issues.apache.org/jira/browse/CLOUDSTACK-7959
I have not reverted my change in master at this point, so it is building right now, but I also did not make the change to 4.5, so that branch is currently failing to build system vms.
I will see if I can track down in which version of qemu the compat option was added so we can add some intelligent logic around this command.
Will
Will STEVENS
Lead Developer
CloudOps | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w cloudops.com<http://cloudops.com/> | tw @CloudOps_
On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com>> wrote:
Hi Leo,
Our new internal build system are using ubuntu 14.04 or something like
that, which has qemu 1.x installed by default, that's why I added the
"compat" option in the build script, otherwise, the image build by
qemu 1.x, won't work on RHEL 6.x.
The fix would be, in the build script, check the version of qemu-img,
if it's 0.x, then don't add "compat" option. There are a lot of people
still using RHEL 6.x as KVM hypervisor, we have to make sure the image
we build can still work on these machines.
On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <LS...@schubergphilis.com>> wrote:
> Hey Edison, all,
>
> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
>
> breaks our internal build with
>
> + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
> systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
> Unknown option 'compat'
> qemu-img: Invalid options for file format 'qcow2'.
>
> This is on CentOS release 6.6, which has qemu-img-0.15.0-1.el6.rfx.x86_64.
>
> Based on
> http://secure-web.cisco.com/1F5zlT8tpWX5PfbJbuzUoso6OWYSnPdIEm5Ue6-gXqJ1a69oLY46nrS2BKoFjZFPlr0DmQwfZHdm2XjfkVhbuMFrET2ilAqQyrI76EYyuxNVnTHaRHH_4bMCtgrtgasDwW1zTJWKDQp8d-HuwyPYokzHAiPFGp7w1CyCabrBpkEw/http%3A%2F%2Fwiki.qemu.org%2FOlderNews
> that seems like it is a really old qemu (august 2011).
>
> I'm guessing you have a newer OS / newer qemu? Can you please let me know what OS, OS version and qemu(-img) version you are using?
>
> Also, does anyone know if there some minimum version of qemu-img that should be used / cloudstack assumes? Is 0.15 still ok to do an acceptable image conversion with? (we currently don't have any kvm use ourselves, but, I'd like for our build infra to produce useful kvm images nonetheless).
>
> According to
> http://secure-web.cisco.com/1pFLbDg2BMTb54NUG5_lYr4mAwFcn9tHP-Weq28Sj5SFqgRKker_DkUKLf6RdwVXHI66uQQNVvD74D6rs6Pc0srpLX0Ejh6nW6o9mBZ-AhQzygMrB_OfUQNLtsB-myd-CA0rJowOhs8ZTsIVr27mgJM4v6ay93D_64JLEBebrlnM/http%3A%2F%2Fwiki.qemu.org%2FChangeLog%2F1.1
> the -o compat switch was introduced in 1.1.
>
> According to
> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
> the default format was changed from 0.10 to 1.1 in qemu 1.7 and onwards.
>
> The libvirt people
> https://bugzilla.redhat.com/show_bug.cgi?id=997977
> say they pass -o compat when qemu supports it (so when v >= 1.1 I imagine).
>
> I think we should do the same in the build script and I'll make a patch.
>
> But, should we publish newer-format images too? According to
> http://secure-web.cisco.com/193Tci801hAyVHNqtfHy56-aPW58HHhzwlnmfvPZQnrbtHxURQC0m3kDHxk-PlYYLhCZX0f768ZYK2fV1WbMFpQPz5JfmFt0S8cuzCSrykjOEx9isfBwORZu6I4XdU3jo-WdJcDgtoH-KOwKcGRX5TGZrVva_BDOYDQzF-7QAg8w/http%3A%2F%2Fwiki.qemu.org%2FFeatures%2FQcow3
> the new format is much better so I imagine qcow/kvm users will really appreciate the newer formats.
>
> Thoughts?
>
>
> cheers,
>
>
> Leo
>
--
Best Regards,
Edison
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Will Stevens <ws...@cloudops.com>.
Alright, so far I have found the following:
https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
Basically, if the qemu-img version is less than 1.7, we should run the non
'compat' option version and if the version is 1.7 or greater, we should run
the new command with the 'compat' version.
Unfortunately I am not able to find a way to get the qemu-img version from
the command line.
I am looking to basically add a conditional to try running with the compat
option and if that fails, then run it without the compat option.
Basically, I would be replacing this:
qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
"${appliance_build_name}-kvm.qcow2"
With this:
set +e
qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img
"${appliance_build_name}-kvm.qcow2"
local qemuresult=$?
set -e
if [ ${qemuresult} != 0 ]; then
log INFO "qemu-img convert failed, trying without compat option"
qemu-img convert -f raw -c -O qcow2 raw.img
"${appliance_build_name}-kvm.qcow2"
fi
What do you guys think? Is this a good enough solution? If you guys agree
I will implement it in master and make sure it works, then we can merge the
change back to 4.5 to fix that branch as well.
Let me know if you have issues with this approach...
Cheers,
Will
*Will STEVENS*
Lead Developer
*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_
On Mon, Dec 1, 2014 at 4:06 PM, Will Stevens <ws...@cloudops.com> wrote:
> Edison,
> According to this page the default compat option is 0.10:
> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
>
> Did you find that to not be the case and is that why you had to add the
> compat option?
>
> BTW, in an attempt to get the master system vms building again, I
> committed a change to master to remove the compat option. We now have
> master system vms building correctly, but Rohit rightly pointed out that I
> had basically reverted your change.
>
> I created a Jira ticket for this issue:
> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
>
> I have not reverted my change in master at this point, so it is building
> right now, but I also did not make the change to 4.5, so that branch is
> currently failing to build system vms.
>
> I will see if I can track down in which version of qemu the compat option
> was added so we can add some intelligent logic around this command.
>
> Will
>
>
>
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com> wrote:
>
>> Hi Leo,
>> Our new internal build system are using ubuntu 14.04 or something like
>> that, which has qemu 1.x installed by default, that's why I added the
>> "compat" option in the build script, otherwise, the image build by
>> qemu 1.x, won't work on RHEL 6.x.
>> The fix would be, in the build script, check the version of qemu-img,
>> if it's 0.x, then don't add "compat" option. There are a lot of people
>> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
>> we build can still work on these machines.
>>
>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <LS...@schubergphilis.com>
>> wrote:
>> > Hey Edison, all,
>> >
>> >
>> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
>> >
>> > breaks our internal build with
>> >
>> > + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
>> > systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
>> > Unknown option 'compat'
>> > qemu-img: Invalid options for file format 'qcow2'.
>> >
>> > This is on CentOS release 6.6, which has
>> qemu-img-0.15.0-1.el6.rfx.x86_64.
>> >
>> > Based on
>> > http://wiki.qemu.org/OlderNews
>> > that seems like it is a really old qemu (august 2011).
>> >
>> > I'm guessing you have a newer OS / newer qemu? Can you please let me
>> know what OS, OS version and qemu(-img) version you are using?
>> >
>> > Also, does anyone know if there some minimum version of qemu-img that
>> should be used / cloudstack assumes? Is 0.15 still ok to do an acceptable
>> image conversion with? (we currently don't have any kvm use ourselves, but,
>> I'd like for our build infra to produce useful kvm images nonetheless).
>> >
>> > According to
>> > http://wiki.qemu.org/ChangeLog/1.1
>> > the -o compat switch was introduced in 1.1.
>> >
>> > According to
>> >
>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>> > the default format was changed from 0.10 to 1.1 in qemu 1.7 and onwards.
>> >
>> > The libvirt people
>> > https://bugzilla.redhat.com/show_bug.cgi?id=997977
>> > say they pass -o compat when qemu supports it (so when v >= 1.1 I
>> imagine).
>> >
>> > I think we should do the same in the build script and I'll make a patch.
>> >
>> > But, should we publish newer-format images too? According to
>> > http://wiki.qemu.org/Features/Qcow3
>> > the new format is much better so I imagine qcow/kvm users will really
>> appreciate the newer formats.
>> >
>> > Thoughts?
>> >
>> >
>> > cheers,
>> >
>> >
>> > Leo
>> >
>>
>>
>>
>> --
>> Best Regards,
>> Edison
>>
>
>
Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
Posted by Nux! <nu...@li.nux.ro>.
Hi,
If it helps, RHEL 7 ships qemu-img 1.5.3 and by default it does not produce compatible images with older qemu, the compat option needs to be specified.
HTH
Lucian
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
----- Original Message -----
> From: "Will Stevens" <ws...@cloudops.com>
> To: dev@cloudstack.apache.org
> Cc: "Leo Simons" <LS...@schubergphilis.com>, "Wilder Rodrigues" <WR...@schubergphilis.com>
> Sent: Monday, 1 December, 2014 21:06:27
> Subject: Re: 05bec59c - kvm, qcow, systemvm qemu-img -o compat
> Edison,
> According to this page the default compat option is 0.10:
> http://manpages.ubuntu.com/manpages/trusty/man1/qemu-img.1.html
>
> Did you find that to not be the case and is that why you had to add the
> compat option?
>
> BTW, in an attempt to get the master system vms building again, I committed
> a change to master to remove the compat option. We now have master system
> vms building correctly, but Rohit rightly pointed out that I had basically
> reverted your change.
>
> I created a Jira ticket for this issue:
> https://issues.apache.org/jira/browse/CLOUDSTACK-7959
>
> I have not reverted my change in master at this point, so it is building
> right now, but I also did not make the change to 4.5, so that branch is
> currently failing to build system vms.
>
> I will see if I can track down in which version of qemu the compat option
> was added so we can add some intelligent logic around this command.
>
> Will
>
>
>
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Thu, Nov 20, 2014 at 1:07 PM, edison su <su...@gmail.com> wrote:
>
>> Hi Leo,
>> Our new internal build system are using ubuntu 14.04 or something like
>> that, which has qemu 1.x installed by default, that's why I added the
>> "compat" option in the build script, otherwise, the image build by
>> qemu 1.x, won't work on RHEL 6.x.
>> The fix would be, in the build script, check the version of qemu-img,
>> if it's 0.x, then don't add "compat" option. There are a lot of people
>> still using RHEL 6.x as KVM hypervisor, we have to make sure the image
>> we build can still work on these machines.
>>
>> On Thu, Nov 20, 2014 at 6:49 AM, Leo Simons <LS...@schubergphilis.com>
>> wrote:
>> > Hey Edison, all,
>> >
>> >
>> https://github.com/apache/cloudstack/commit/05bec59c1498dbcfb8a1089c86855fd3b433ea58
>> >
>> > breaks our internal build with
>> >
>> > + qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img \
>> > systemvmtemplate-systemvm-persistent-config-4.5.0.124-kvm.qcow2
>> > Unknown option 'compat'
>> > qemu-img: Invalid options for file format 'qcow2'.
>> >
>> > This is on CentOS release 6.6, which has
>> qemu-img-0.15.0-1.el6.rfx.x86_64.
>> >
>> > Based on
>> > http://wiki.qemu.org/OlderNews
>> > that seems like it is a really old qemu (august 2011).
>> >
>> > I'm guessing you have a newer OS / newer qemu? Can you please let me
>> know what OS, OS version and qemu(-img) version you are using?
>> >
>> > Also, does anyone know if there some minimum version of qemu-img that
>> should be used / cloudstack assumes? Is 0.15 still ok to do an acceptable
>> image conversion with? (we currently don't have any kvm use ourselves, but,
>> I'd like for our build infra to produce useful kvm images nonetheless).
>> >
>> > According to
>> > http://wiki.qemu.org/ChangeLog/1.1
>> > the -o compat switch was introduced in 1.1.
>> >
>> > According to
>> >
>> https://github.com/qemu/qemu/commit/9117b47717ad208b12786ce88eacb013f9b3dd1c
>> > the default format was changed from 0.10 to 1.1 in qemu 1.7 and onwards.
>> >
>> > The libvirt people
>> > https://bugzilla.redhat.com/show_bug.cgi?id=997977
>> > say they pass -o compat when qemu supports it (so when v >= 1.1 I
>> imagine).
>> >
>> > I think we should do the same in the build script and I'll make a patch.
>> >
>> > But, should we publish newer-format images too? According to
>> > http://wiki.qemu.org/Features/Qcow3
>> > the new format is much better so I imagine qcow/kvm users will really
>> appreciate the newer formats.
>> >
>> > Thoughts?
>> >
>> >
>> > cheers,
>> >
>> >
>> > Leo
>> >
>>
>>
>>
>> --
>> Best Regards,
>> Edison