You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by David Nalley <da...@gnsa.us> on 2012/07/04 22:02:51 UTC
[DISCUSS] systemvms
Hi folks,
I spent a bit of time going through the patches directory and working
on license headers today.
Historically CloudStack has made a Debian-based systemvm template
available for each hypervisor. However, my sense is that going forward
we will not be able to do this because of licensing issues. However,
the need for a systemvm will not be eliminated, so I'd like to come up
with some ideas on how to mitigate this.
My only initial thought is that we'd have only the body of code that
the systemvm uses exclusively for CloudStack (and remove all of the
other bits like vhdutil) and the config tree.) and that we would
additionally provide some packaging around those bits so that a user
could deploy that package to a debian or fedora-based system and have
a working systemvm.
Thoughts, comments, flames?
--David
RE: Re:Re: [DISCUSS] systemvms
Posted by Edison Su <Ed...@citrix.com>.
Need to download CloudStack source code, or you can find the latest file from:
https://git-wip-us.apache.org/repos/asf?p=incubator-cloudstack.git;a=blob;f=patches/systemvm/debian/buildsystemvm.sh;h=090a0827e6b13d56e48cc23b0c1cf67b5863b401;hb=HEAD
> -----Original Message-----
> From: yueluck@163.com [mailto:yueluck@163.com] On Behalf Of
> cloudstacks@163.com
> Sent: Sunday, July 22, 2012 10:39 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re:Re: [DISCUSS] systemvms
>
>
> where can i find patches/systemvm/debian/build ? my OS is centos5.6.
>
> At 2012-07-22 15:12:22,"Edison Su" <Ed...@citrix.com> wrote:
> >Build a systemvm is pretty easy: just run
> patches/systemvm/debian/build systemvm.sh on a Ubuntu machine
> >
> >Sent from my iPhone
> >
> >On Jul 21, 2012, at 7:05 AM, "yueluck" <yu...@163.com> wrote:
> >
> >> when users can get the documents about how to build systemVM step
> by step?
> >>
> >>
> >> At 2012-07-10 01:23:22,"Kevin Kluge" <Ke...@citrix.com> wrote:
> >>>> Are you saying that you think that Apache CloudStack can provide
> Debian-
> >>>> based systemVMs as part of a release (or in any other manner)?
> >>>> If so, I am missing something. And I would love for this to not
> be the case, I
> >>>> just don't see how we can ship a big blob of GPL software given
> the
> >>>> guidelines that we are expected to follow.
> >>>>
> >>>
> >>> We need to be careful with defining "ship". Clearly we aren't
> releasing GPL-licensed software. The system template isn't a binary
> dep in the same sense as the other deps we are changing or removing to
> comply with ASF release criteria. This may limit where we could host a
> built system VM template, but even with some limitations we shouldn't
> let that block an ACS release. If you have continued concerns about
> this we should surface them quickly as this would be a big addition to
> the ACS release scope (at least from what I was expecting). Ideally we
> would be able to have a discussion with whoever will ultimately decide
> if this is allowed.
> >>>
> >>>
> >>>> The only conversation I see on it from the proposal period is [1],
> am I missing
> >>>> a thread somewhere?
> >>>
> >>> These conversations occurred when I was working to understand what
> challenges would exist if there were an "Apache CloudStack" project,
> prior to a project submission. That discussion wasn't public.
> >>>
> >>> -kevin
Re:Re: [DISCUSS] systemvms
Posted by cl...@163.com.
where can i find patches/systemvm/debian/build ? my OS is centos5.6.
At 2012-07-22 15:12:22,"Edison Su" <Ed...@citrix.com> wrote:
>Build a systemvm is pretty easy: just run patches/systemvm/debian/build systemvm.sh on a Ubuntu machine
>
>Sent from my iPhone
>
>On Jul 21, 2012, at 7:05 AM, "yueluck" <yu...@163.com> wrote:
>
>> when users can get the documents about how to build systemVM step by step?
>>
>>
>> At 2012-07-10 01:23:22,"Kevin Kluge" <Ke...@citrix.com> wrote:
>>>> Are you saying that you think that Apache CloudStack can provide Debian-
>>>> based systemVMs as part of a release (or in any other manner)?
>>>> If so, I am missing something. And I would love for this to not be the case, I
>>>> just don't see how we can ship a big blob of GPL software given the
>>>> guidelines that we are expected to follow.
>>>>
>>>
>>> We need to be careful with defining "ship". Clearly we aren't releasing GPL-licensed software. The system template isn't a binary dep in the same sense as the other deps we are changing or removing to comply with ASF release criteria. This may limit where we could host a built system VM template, but even with some limitations we shouldn't let that block an ACS release. If you have continued concerns about this we should surface them quickly as this would be a big addition to the ACS release scope (at least from what I was expecting). Ideally we would be able to have a discussion with whoever will ultimately decide if this is allowed.
>>>
>>>
>>>> The only conversation I see on it from the proposal period is [1], am I missing
>>>> a thread somewhere?
>>>
>>> These conversations occurred when I was working to understand what challenges would exist if there were an "Apache CloudStack" project, prior to a project submission. That discussion wasn't public.
>>>
>>> -kevin
Re: [DISCUSS] systemvms
Posted by Edison Su <Ed...@citrix.com>.
Build a systemvm is pretty easy: just run patches/systemvm/debian/build systemvm.sh on a Ubuntu machine
Sent from my iPhone
On Jul 21, 2012, at 7:05 AM, "yueluck" <yu...@163.com> wrote:
> when users can get the documents about how to build systemVM step by step?
>
>
> At 2012-07-10 01:23:22,"Kevin Kluge" <Ke...@citrix.com> wrote:
>>> Are you saying that you think that Apache CloudStack can provide Debian-
>>> based systemVMs as part of a release (or in any other manner)?
>>> If so, I am missing something. And I would love for this to not be the case, I
>>> just don't see how we can ship a big blob of GPL software given the
>>> guidelines that we are expected to follow.
>>>
>>
>> We need to be careful with defining "ship". Clearly we aren't releasing GPL-licensed software. The system template isn't a binary dep in the same sense as the other deps we are changing or removing to comply with ASF release criteria. This may limit where we could host a built system VM template, but even with some limitations we shouldn't let that block an ACS release. If you have continued concerns about this we should surface them quickly as this would be a big addition to the ACS release scope (at least from what I was expecting). Ideally we would be able to have a discussion with whoever will ultimately decide if this is allowed.
>>
>>
>>> The only conversation I see on it from the proposal period is [1], am I missing
>>> a thread somewhere?
>>
>> These conversations occurred when I was working to understand what challenges would exist if there were an "Apache CloudStack" project, prior to a project submission. That discussion wasn't public.
>>
>> -kevin
Re:RE: [DISCUSS] systemvms
Posted by yueluck <yu...@163.com>.
when users can get the documents about how to build systemVM step by step?
At 2012-07-10 01:23:22,"Kevin Kluge" <Ke...@citrix.com> wrote:
>> Are you saying that you think that Apache CloudStack can provide Debian-
>> based systemVMs as part of a release (or in any other manner)?
>> If so, I am missing something. And I would love for this to not be the case, I
>> just don't see how we can ship a big blob of GPL software given the
>> guidelines that we are expected to follow.
>>
>
>We need to be careful with defining "ship". Clearly we aren't releasing GPL-licensed software. The system template isn't a binary dep in the same sense as the other deps we are changing or removing to comply with ASF release criteria. This may limit where we could host a built system VM template, but even with some limitations we shouldn't let that block an ACS release. If you have continued concerns about this we should surface them quickly as this would be a big addition to the ACS release scope (at least from what I was expecting). Ideally we would be able to have a discussion with whoever will ultimately decide if this is allowed.
>
>
>> The only conversation I see on it from the proposal period is [1], am I missing
>> a thread somewhere?
>
>These conversations occurred when I was working to understand what challenges would exist if there were an "Apache CloudStack" project, prior to a project submission. That discussion wasn't public.
>
>-kevin
RE: [DISCUSS] systemvms
Posted by Kevin Kluge <Ke...@citrix.com>.
> Are you saying that you think that Apache CloudStack can provide Debian-
> based systemVMs as part of a release (or in any other manner)?
> If so, I am missing something. And I would love for this to not be the case, I
> just don't see how we can ship a big blob of GPL software given the
> guidelines that we are expected to follow.
>
We need to be careful with defining "ship". Clearly we aren't releasing GPL-licensed software. The system template isn't a binary dep in the same sense as the other deps we are changing or removing to comply with ASF release criteria. This may limit where we could host a built system VM template, but even with some limitations we shouldn't let that block an ACS release. If you have continued concerns about this we should surface them quickly as this would be a big addition to the ACS release scope (at least from what I was expecting). Ideally we would be able to have a discussion with whoever will ultimately decide if this is allowed.
> The only conversation I see on it from the proposal period is [1], am I missing
> a thread somewhere?
These conversations occurred when I was working to understand what challenges would exist if there were an "Apache CloudStack" project, prior to a project submission. That discussion wasn't public.
-kevin
Re: [DISCUSS] systemvms
Posted by David Nalley <da...@gnsa.us>.
On Fri, Jul 6, 2012 at 1:02 PM, Kevin Kluge <Ke...@citrix.com> wrote:
> I had mentioned this concern (licensing and system VM presence) in the earliest Apache discussions, and I included it in the CloudStack project proposal at [1]. I had been told that it was a non-issue. That is, as long as the code in our repo is of allowed license this dependency is fine, and it won't block our first Apache release. There are some other interesting ideas later in this thread, but they seem lower priority to me than the other issues on the deps page [2].
>
> [1] http://wiki.apache.org/incubator/CloudStackProposal
> [2] http://confluence.cloudstack.org/display/dev/Moving+dependencies+to+ASF+approved+licenses
>
Are you saying that you think that Apache CloudStack can provide
Debian-based systemVMs as part of a release (or in any other manner)?
If so, I am missing something. And I would love for this to not be
the case, I just don't see how we can ship a big blob of GPL software
given the guidelines that we are expected to follow.
The only conversation I see on it from the proposal period is [1], am
I missing a thread somewhere?
--David
[1] http://s.apache.org/v3
RE: [DISCUSS] systemvms
Posted by Kevin Kluge <Ke...@citrix.com>.
I had mentioned this concern (licensing and system VM presence) in the earliest Apache discussions, and I included it in the CloudStack project proposal at [1]. I had been told that it was a non-issue. That is, as long as the code in our repo is of allowed license this dependency is fine, and it won't block our first Apache release. There are some other interesting ideas later in this thread, but they seem lower priority to me than the other issues on the deps page [2].
[1] http://wiki.apache.org/incubator/CloudStackProposal
[2] http://confluence.cloudstack.org/display/dev/Moving+dependencies+to+ASF+approved+licenses
> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: Wednesday, July 04, 2012 1:03 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: [DISCUSS] systemvms
>
> Hi folks,
>
> I spent a bit of time going through the patches directory and working on
> license headers today.
>
> Historically CloudStack has made a Debian-based systemvm template
> available for each hypervisor. However, my sense is that going forward we
> will not be able to do this because of licensing issues. However, the need for
> a systemvm will not be eliminated, so I'd like to come up with some ideas on
> how to mitigate this.
>
> My only initial thought is that we'd have only the body of code that the
> systemvm uses exclusively for CloudStack (and remove all of the other bits
> like vhdutil) and the config tree.) and that we would additionally provide
> some packaging around those bits so that a user could deploy that package to
> a debian or fedora-based system and have a working systemvm.
>
> Thoughts, comments, flames?
>
> --David
Re: [DISCUSS] systemvms
Posted by John Kinsella <jl...@stratosec.co>.
On Jul 6, 2012, at 8:25 AM, Wido den Hollander wrote:
> Correct! Due to licensing we are however not allowed to distribute it in the official releases. But we can always write a script:
>
> $ cloud-download-and-install-systemvm /srv/my-secondary-storage <url>
>
> If URL is not specified it will download the image from a default location.
I think you have to go this way…maybe have a "SystemVM URL" variable in global settings that gets passed through as needed - that'd deal with installations that are not connected to the Internet.
A good number of folks are already having issues just trying to get a simple CS install up and running - we really can't make it more complicated.
How do our "competitors" handle this?
John
Re: [DISCUSS] systemvms
Posted by Wido den Hollander <wi...@widodh.nl>.
On 07/06/2012 01:04 AM, Chiradeep Vittal wrote:
>
>
> On 7/5/12 11:58 AM, "David Nalley" <da...@gnsa.us> wrote:
>>>>
>>>> Right now you have to download this weird qcow2 from the CS website, but
>>>> that should be different I think:
>>>>
>>>> You set up CloudStack, configure your zone and then it will ask you to
>>>> provide the System VM template.
>>>>
>>>> We can still provide a System VM template we build from scratch and put
>>>> the image online somewhere, but users also have the freedom to upload
>>>> their own.
>>>
>>> +1. A default systemvm template is very important. In fact for
>>> regression
>>> tests, smoke tests etc, this would be used.
>>> Also, most *users* of cloudstack would have very little idea on how to
>>> roll their own systemvm.
>>
>> So the issue with a default systemVM is that Apache CloudStack can't
>> distribute it. (unless we move to a BSD). Or at least - in their
>> current form, Apache CloudStack can't distribute the systemVMs. Soooo
>> what do we do?
>
> Like Wido said, we ask for the location of this template at install time.
> Where the template is hosted is not necessarily Apache, just like a lot of
> the libraries we use are not hosted at Apache.
What I meant:
1. A user can upload his/her homebrew version of the System VM image
2. A user can upload a pre-build System VM image
>
>>
>> While .debs and .rpms are not necessarily mandates - what would we
>> ship? We essentially can't ship a linux-based VM.
>
> There's a shell script that builds the debian template. That's in the
> patches directory. Today that takes the scripts / config files in there
> and bundles it into the template after building the base image. So, the
> repo will have the build script, the scripts and packaging scripts to
> package the scripts into deb and rpm.
The Debs and RPM's would just package the stuff.
A user could do:
1. Set up a simple Debian VM with a 2GB disk
2. Add the CloudStack APT repo
3. apt-get install cloud-systemvm
4. Shut down the VM
5. Convert the disk to QCOW2
6. Upload it into CloudStack as a System VM template
In this you can replace apt by yum ofcourse and Debian by Fedora or even
RHEL.
A System VM is nothing more (or less) then a Debian installation with
Apache, nfs-common and the agent running inside.
With the proper packages and tutorial anyone can create a image for the
System VM.
>
> We obviously need to write a design document specifying the contract for
> the system vm so that interested parties can build / offer / distribute
> their own systemvms. But a default one is necessary IMO. Otherwise the
> out-of-the-box experience for first-timers is going to be very very iffy.
Correct! Due to licensing we are however not allowed to distribute it in
the official releases. But we can always write a script:
$ cloud-download-and-install-systemvm /srv/my-secondary-storage <url>
If URL is not specified it will download the image from a default location.
This way we also make it much easier for people to create their own
System VM's. Some people might prefer hardening them futher or stripping
them down.
Wido
Re: [DISCUSS] systemvms
Posted by Chiradeep Vittal <Ch...@citrix.com>.
On 7/5/12 11:58 AM, "David Nalley" <da...@gnsa.us> wrote:
>>>
>>>Right now you have to download this weird qcow2 from the CS website, but
>>>that should be different I think:
>>>
>>>You set up CloudStack, configure your zone and then it will ask you to
>>>provide the System VM template.
>>>
>>>We can still provide a System VM template we build from scratch and put
>>>the image online somewhere, but users also have the freedom to upload
>>>their own.
>>
>> +1. A default systemvm template is very important. In fact for
>>regression
>> tests, smoke tests etc, this would be used.
>> Also, most *users* of cloudstack would have very little idea on how to
>> roll their own systemvm.
>
>So the issue with a default systemVM is that Apache CloudStack can't
>distribute it. (unless we move to a BSD). Or at least - in their
>current form, Apache CloudStack can't distribute the systemVMs. Soooo
>what do we do?
Like Wido said, we ask for the location of this template at install time.
Where the template is hosted is not necessarily Apache, just like a lot of
the libraries we use are not hosted at Apache.
>
>While .debs and .rpms are not necessarily mandates - what would we
>ship? We essentially can't ship a linux-based VM.
There's a shell script that builds the debian template. That's in the
patches directory. Today that takes the scripts / config files in there
and bundles it into the template after building the base image. So, the
repo will have the build script, the scripts and packaging scripts to
package the scripts into deb and rpm.
We obviously need to write a design document specifying the contract for
the system vm so that interested parties can build / offer / distribute
their own systemvms. But a default one is necessary IMO. Otherwise the
out-of-the-box experience for first-timers is going to be very very iffy.
--
Chiradeep
Re: [DISCUSS] systemvms
Posted by David Nalley <da...@gnsa.us>.
On Thu, Jul 5, 2012 at 2:35 PM, Chiradeep Vittal
<Ch...@citrix.com> wrote:
> Ah, Wido puts it much better. David's email kind of alarmed me.
What did I say that alarmed you?? :)
If it makes you feel better your email alarms me :)
>
>
> On 7/5/12 7:24 AM, "Wido den Hollander" <wi...@widodh.nl> wrote:
> [snip]
>
>>
>>I agree. In essence the System VM's are more (talking KVM-wise!) than a
>>Debian installation with the Java agent running in them.
>
> In the future, there may be even be multiple system vm templates. T
>
>>
>>Right now you have to download this weird qcow2 from the CS website, but
>>that should be different I think:
>>
>>You set up CloudStack, configure your zone and then it will ask you to
>>provide the System VM template.
>>
>>We can still provide a System VM template we build from scratch and put
>>the image online somewhere, but users also have the freedom to upload
>>their own.
>
> +1. A default systemvm template is very important. In fact for regression
> tests, smoke tests etc, this would be used.
> Also, most *users* of cloudstack would have very little idea on how to
> roll their own systemvm.
So the issue with a default systemVM is that Apache CloudStack can't
distribute it. (unless we move to a BSD). Or at least - in their
current form, Apache CloudStack can't distribute the systemVMs. Soooo
what do we do?
>
>
>>
>>In the repository we should only keep what we really need. When building
>>RPM's and Deb's we build the proper packages for Debian and Fedora which
>>you can install and depend on everything you need.
>>
>>Those packages can install the correct init scripts which are needed for
>>starting everything from the first time.
>>
>>These init script will mount the second disk in the System VM and
>>retrieve all the relevant configuration from there.
>>
>>We should start with actually building .debs and .rpms for the System
>>VM's and not have these scripts just floating around and magically
>>finding their way into the System VM's.
>
> +1. Although this could take a long time and could gate the 4.0 release if
> made a prerequisite.
While .debs and .rpms are not necessarily mandates - what would we
ship? We essentially can't ship a linux-based VM.
>
>>
>>The scripts inside the repository will probably comply with the Apache
>>License, although that's for KVM, I'm not sure about Xen.
>
> There's very little xen-specific stuff inside the vm. Some of it may
> relate to xen-tools. That can be removed AFAIK, since it actually hinders
> the xenserver upgrade process.
Re: [DISCUSS] systemvms
Posted by Chiradeep Vittal <Ch...@citrix.com>.
Ah, Wido puts it much better. David's email kind of alarmed me.
On 7/5/12 7:24 AM, "Wido den Hollander" <wi...@widodh.nl> wrote:
[snip]
>
>I agree. In essence the System VM's are more (talking KVM-wise!) than a
>Debian installation with the Java agent running in them.
In the future, there may be even be multiple system vm templates. T
>
>Right now you have to download this weird qcow2 from the CS website, but
>that should be different I think:
>
>You set up CloudStack, configure your zone and then it will ask you to
>provide the System VM template.
>
>We can still provide a System VM template we build from scratch and put
>the image online somewhere, but users also have the freedom to upload
>their own.
+1. A default systemvm template is very important. In fact for regression
tests, smoke tests etc, this would be used.
Also, most *users* of cloudstack would have very little idea on how to
roll their own systemvm.
>
>In the repository we should only keep what we really need. When building
>RPM's and Deb's we build the proper packages for Debian and Fedora which
>you can install and depend on everything you need.
>
>Those packages can install the correct init scripts which are needed for
>starting everything from the first time.
>
>These init script will mount the second disk in the System VM and
>retrieve all the relevant configuration from there.
>
>We should start with actually building .debs and .rpms for the System
>VM's and not have these scripts just floating around and magically
>finding their way into the System VM's.
+1. Although this could take a long time and could gate the 4.0 release if
made a prerequisite.
>
>The scripts inside the repository will probably comply with the Apache
>License, although that's for KVM, I'm not sure about Xen.
There's very little xen-specific stuff inside the vm. Some of it may
relate to xen-tools. That can be removed AFAIK, since it actually hinders
the xenserver upgrade process.
--
Chiradeep
Re: [DISCUSS] systemvms
Posted by Wido den Hollander <wi...@widodh.nl>.
On 04-07-12 22:02, David Nalley wrote:
> Hi folks,
>
> I spent a bit of time going through the patches directory and working
> on license headers today.
>
> Historically CloudStack has made a Debian-based systemvm template
> available for each hypervisor. However, my sense is that going forward
> we will not be able to do this because of licensing issues. However,
> the need for a systemvm will not be eliminated, so I'd like to come up
> with some ideas on how to mitigate this.
Hooray!
>
> My only initial thought is that we'd have only the body of code that
> the systemvm uses exclusively for CloudStack (and remove all of the
> other bits like vhdutil) and the config tree.) and that we would
> additionally provide some packaging around those bits so that a user
> could deploy that package to a debian or fedora-based system and have
> a working systemvm.
>
> Thoughts, comments, flames?
>
I agree. In essence the System VM's are more (talking KVM-wise!) than a
Debian installation with the Java agent running in them.
Right now you have to download this weird qcow2 from the CS website, but
that should be different I think:
You set up CloudStack, configure your zone and then it will ask you to
provide the System VM template.
We can still provide a System VM template we build from scratch and put
the image online somewhere, but users also have the freedom to upload
their own.
In the repository we should only keep what we really need. When building
RPM's and Deb's we build the proper packages for Debian and Fedora which
you can install and depend on everything you need.
Those packages can install the correct init scripts which are needed for
starting everything from the first time.
These init script will mount the second disk in the System VM and
retrieve all the relevant configuration from there.
We should start with actually building .debs and .rpms for the System
VM's and not have these scripts just floating around and magically
finding their way into the System VM's.
The scripts inside the repository will probably comply with the Apache
License, although that's for KVM, I'm not sure about Xen.
Just my to €0,02
Wido