You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Rohit Yadav <ro...@shapeblue.com> on 2017/07/23 16:08:13 UTC
[DISCUSS] Move to Debian9 systemvmtemplate
All,
Just want to kick an initial discussion around migration to Debian9 based systemvmtemplate, and get your feedback on the same.
Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
Regards.
rohit.yadav@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Rohit Yadav <ro...@shapeblue.com>.
Only though testing we can know if there are any regressions, with Debian7 to be unsupported somewhere in 2018 we need to start some work around this.
Distros such as Arch Linux are not solid enough or work out of the box and have bleeding edge unstable software. Debian 9 "stretch" finally have openjdk-8-jre-headless (which surprising was not in debian8, though was in debian8's backports) so we can move away from using Azul's distro as well, along with having updated stable dependencies/versions.
Debian distributions are usually gold standard for security with other distros such as Tails and Kali Linux build on top of it, maybe a reason why systemvm templates were originally build using Debian as base. Moving to a new distribution will be a lot more work given highly assumed and dependent code (bash/python scripts) that targets Debian based systemvmtemplate. Perhaps a more modular and modern approach would be to split systemvm services and run them in containers, using a light base such as Alpine Linux, however that's a different discussion and a much bigger project to undertake.
- Rohit
________________________________
From: Jayapal Uradi <ja...@accelerite.com>
Sent: Monday, July 24, 2017 6:34:08 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
First one come to my mind is VPN feature.
But other general issues will depend on what are changed in debain9.
Thanks,
Jayapal
rohit.yadav@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
> On Jul 24, 2017, at 9:54 AM, Satki Badal <sa...@gmail.com> wrote:
>
> Hi Jayapal,
>
> Do we know what all features re covered by Marvin and what all will require manual testing ?
> I guess without this list we can never be sure is all the features got tested or not.
>
> Thanks,
> Satki
>
>> On 24-Jul-2017, at 9:48 AM, Jayapal Uradi <ja...@accelerite.com> wrote:
>>
>> Hi,
>>
>> We should throughly check all the VR features to see feature breakage because of Debian9.
>>
>> Thanks,
>> Jayapal
>>
>>
>>> On Jul 23, 2017, at 9:38 PM, Rohit Yadav <ro...@shapeblue.com> wrote:
>>>
>>> All,
>>>
>>>
>>> Just want to kick an initial discussion around migration to Debian9 based systemvmtemplate, and get your feedback on the same.
>>>
>>> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
>>>
>>>
>>> Regards.
>>>
>>> rohit.yadav@shapeblue.com
>>> www.shapeblue.com<http://www.shapeblue.com>
>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>> @shapeblue
>>>
>>>
>>>
>>
>> DISCLAIMER
>> ==========
>> This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
>>
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Jayapal Uradi <ja...@accelerite.com>.
First one come to my mind is VPN feature.
But other general issues will depend on what are changed in debain9.
Thanks,
Jayapal
> On Jul 24, 2017, at 9:54 AM, Satki Badal <sa...@gmail.com> wrote:
>
> Hi Jayapal,
>
> Do we know what all features re covered by Marvin and what all will require manual testing ?
> I guess without this list we can never be sure is all the features got tested or not.
>
> Thanks,
> Satki
>
>> On 24-Jul-2017, at 9:48 AM, Jayapal Uradi <ja...@accelerite.com> wrote:
>>
>> Hi,
>>
>> We should throughly check all the VR features to see feature breakage because of Debian9.
>>
>> Thanks,
>> Jayapal
>>
>>
>>> On Jul 23, 2017, at 9:38 PM, Rohit Yadav <ro...@shapeblue.com> wrote:
>>>
>>> All,
>>>
>>>
>>> Just want to kick an initial discussion around migration to Debian9 based systemvmtemplate, and get your feedback on the same.
>>>
>>> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
>>>
>>>
>>> Regards.
>>>
>>> rohit.yadav@shapeblue.com
>>> www.shapeblue.com
>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>>> @shapeblue
>>>
>>>
>>>
>>
>> DISCLAIMER
>> ==========
>> This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
>>
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Satki Badal <sa...@gmail.com>.
Hi Jayapal,
Do we know what all features re covered by Marvin and what all will require manual testing ?
I guess without this list we can never be sure is all the features got tested or not.
Thanks,
Satki
> On 24-Jul-2017, at 9:48 AM, Jayapal Uradi <ja...@accelerite.com> wrote:
>
> Hi,
>
> We should throughly check all the VR features to see feature breakage because of Debian9.
>
> Thanks,
> Jayapal
>
>
>> On Jul 23, 2017, at 9:38 PM, Rohit Yadav <ro...@shapeblue.com> wrote:
>>
>> All,
>>
>>
>> Just want to kick an initial discussion around migration to Debian9 based systemvmtemplate, and get your feedback on the same.
>>
>> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
>>
>>
>> Regards.
>>
>> rohit.yadav@shapeblue.com
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
>> @shapeblue
>>
>>
>>
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Jayapal Uradi <ja...@accelerite.com>.
Hi,
We should throughly check all the VR features to see feature breakage because of Debian9.
Thanks,
Jayapal
> On Jul 23, 2017, at 9:38 PM, Rohit Yadav <ro...@shapeblue.com> wrote:
>
> All,
>
>
> Just want to kick an initial discussion around migration to Debian9 based systemvmtemplate, and get your feedback on the same.
>
> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
>
>
> Regards.
>
> rohit.yadav@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite, a Persistent Systems business. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent Systems business does not accept any liability for virus infected mails.
RE: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Paul Angus <pa...@shapeblue.com>.
Can you update the documentation in confluence
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Build+Your+Own+SystemVM+Templates
to reflect what's required now.
Its woefully short on information, and links don't work anymore.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
Sent: 28 July 2017 20:30
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
I think we can move to packer once we can get the Debian9 based systemvmtemplate to work.
I think we should focus on doing this first and then focus on migration to a new build system as a next step.
I spent some time today and with some help from veewee authors, I could get a base template up and running:
https://github.com/apache/cloudstack/pull/2211
The above PR branch is pushed on ASF remote and allows for cross-collaboration with all ACS committers. Please collaborate with me on this and feel free to push changes on the branch as separate commits and/or make changes to the PR. Thanks.
- Rohit
________________________________
From: Tim Mackey <tm...@gmail.com>
Sent: Friday, July 28, 2017 3:59:36 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
Syed,
I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It shouldn't be hard to validate with newest XS and updated Packer - I just lack the infra to do the testing.
[1] https://github.com/xenserverarmy/packer
-tim
rohit.yadav@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO.
> I've used Packer before and I really like it, the only downside that I
> see is that Packer lacks support for XenServer VHD images. There is
> some work on a XenServer plugin but I haven't tested that. If the
> community decides to use Packer, I can do some initial validation of it on XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser <ma...@renemoser.net>:
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to
> > > > Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.com/apache/
> > cloudstack/pull/2198
> > >
> > > Have you considered to replace veewee by packer?
> > >
> >
> > Packer is really nice indeed. We use it to build our templates [0]
> > which we use on CloudStack.
> >
> > Building the SSVM using Packer should be rather easy I think.
> >
> > [0]: https://github.com/pcextreme/packer-templates
> >
> > > Our friends from schubergphilis have already done some work here
> > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > >
> > > However there would be also an official way to convert the
> > > definitions https://www.packer.io/guides/veewee-to-packer.html
> > >
> > > Regards René
> >
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi René,
Thanks for your comments, this will be a strictly maintenance project at the moment I think we should consider expanding its scope to include any new feature/integration. The current aim and scope of this work are to migrate the base image to Debian9 and ensure no regressions. However, feel free to discuss/work on the proposed features/integrations separately.
Regards.
________________________________
From: Rene Moser <ma...@renemoser.net>
Sent: Tuesday, November 21, 2017 8:37:49 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
Hi Rohit
First, thank you very much for the effort!
On 11/21/2017 01:07 PM, Rohit Yadav wrote:
> Please advise if you can collaborate with me on this, especially around testing. Thanks!
I have some 2 "nice to have"s, AFAICS these haven't been addressed yet:
- SNMP readonly listen on "linklocalip"
- HAProxy stats (or even exporter
https://github.com/prometheus/haproxy_exporter) listen on linklocalip
Regards
René
rohit.yadav@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Rene Moser <ma...@renemoser.net>.
Hi Rohit
First, thank you very much for the effort!
On 11/21/2017 01:07 PM, Rohit Yadav wrote:
> Please advise if you can collaborate with me on this, especially around testing. Thanks!
I have some 2 "nice to have"s, AFAICS these haven't been addressed yet:
- SNMP readonly listen on "linklocalip"
- HAProxy stats (or even exporter
https://github.com/prometheus/haproxy_exporter) listen on linklocalip
Regards
René
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Rohit Yadav <ro...@shapeblue.com>.
All,
I wanted to revive this thread and bring recent progress to your attention:
- We're now able to build debian9 based systemvmtemplates successfully that work okay-ish as CPVM/SSVM host and some initial networking support (dns, dhcp, basic zone, advanced zone: firewall, pf, snat etc.) for VirtualRouter
- Several optimization improvements around disk size, services load time, a lot of work still to be done
- Several services migrated to use systemd, pending optimizations
- PR: https://github.com/apache/cloudstack/pull/2211
I could finally deploy a test KVM environment (advanced zone) and run some smoke tests on it, they are available in the above-mentioned PR. Most failures are around networking and services.
Given Debian7 has EOL-ed, and Java7 too, I think it's about time to work on this and aim to get this done for ACS 4.11 (end of Q1/2018). Please share your feedback and comments.
Using Trillian we can test the "migration to a debian9/java8 based template" PR against KVM, XenServer and VMware, however, what other hypervisors should we be testing/supporting - HyperV, baremetal, OVM etc? What are the status of these hypervisor plugins - baremetal, hyperv, lxc, ovm/ovm3, ucs?
Please advise if you can collaborate with me on this, especially around testing. Thanks!
Regards.
________________________________
From: Rohit Yadav <ro...@shapeblue.com>
Sent: Monday, July 31, 2017 3:05:30 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
Moving to a packer based build system can still reuse the same scripts and recipes to build a systemvmtemplate which is agnostic of who/how you build it. With lack of time and resources, moving to debian9 based systemvmtemplate is a much needed effort and moving to a new build system can be a next step.
The current build system may not be perfect, may be tricky to setup at first. Most recently updated docs are at tools/appliance/README.md, if anyone has issues you may ping me and I may be able to help.
Now back to the topic of supporting debian9 as systemvmtemplate base, I was able to get something up and running this weekend and get serial console. I could also verify some patching done by the systemvm.iso file however I'm facing issues with running cloud-early-config, postinit and cloud services in a certain order and I need help around the systemd scripts.
Here's the PR:
https://github.com/apache/cloudstack/pull/2211
The PR branch is pushed on ASF remote so any committer can collaborate with me by pushing changes/fixes as a separate commit on that branch, feel free to do so. Thanks.
- Rohit
________________________________
From: Paul Angus <pa...@shapeblue.com>
Sent: Monday, July 31, 2017 11:11:53 AM
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] Move to Debian9 systemvmtemplate
Depending on the timescales that we looking at, if we can get an agreement to use Packer going forward, there is an argument to say that spending time getting the Debian 9 template to work on VeeWee and then on Packer is wasted effort and that we should just use this as the opportunity to move over to Packer/Debian9.
Having spent the weekend fighting with RVM/VeeWee/Ruby. And finding that we've hard linked rvm to Ruby 2.1.1 when it's now on 2.4, Veewee hasn't been updated for years and other mismatches. I'm very interested to see other options explored.
Veewee doesn't do the disk conversions at them moment, so we can still keep that a separate process for corner cases that Packer (or something else) can't manage..
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
rohit.yadav@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
rohit.yadav@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
Sent: 28 July 2017 20:30
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
I think we can move to packer once we can get the Debian9 based systemvmtemplate to work.
I think we should focus on doing this first and then focus on migration to a new build system as a next step.
I spent some time today and with some help from veewee authors, I could get a base template up and running:
https://github.com/apache/cloudstack/pull/2211
The above PR branch is pushed on ASF remote and allows for cross-collaboration with all ACS committers. Please collaborate with me on this and feel free to push changes on the branch as separate commits and/or make changes to the PR. Thanks.
- Rohit
________________________________
From: Tim Mackey <tm...@gmail.com>
Sent: Friday, July 28, 2017 3:59:36 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
Syed,
I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It shouldn't be hard to validate with newest XS and updated Packer - I just lack the infra to do the testing.
[1] https://github.com/xenserverarmy/packer
-tim
rohit.yadav@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO.
> I've used Packer before and I really like it, the only downside that I
> see is that Packer lacks support for XenServer VHD images. There is
> some work on a XenServer plugin but I haven't tested that. If the
> community decides to use Packer, I can do some initial validation of it on XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser <ma...@renemoser.net>:
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to
> > > > Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.com/apache/
> > cloudstack/pull/2198
> > >
> > > Have you considered to replace veewee by packer?
> > >
> >
> > Packer is really nice indeed. We use it to build our templates [0]
> > which we use on CloudStack.
> >
> > Building the SSVM using Packer should be rather easy I think.
> >
> > [0]: https://github.com/pcextreme/packer-templates
> >
> > > Our friends from schubergphilis have already done some work here
> > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > >
> > > However there would be also an official way to convert the
> > > definitions https://www.packer.io/guides/veewee-to-packer.html
> > >
> > > Regards René
> >
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Rohit Yadav <ro...@shapeblue.com>.
Moving to a packer based build system can still reuse the same scripts and recipes to build a systemvmtemplate which is agnostic of who/how you build it. With lack of time and resources, moving to debian9 based systemvmtemplate is a much needed effort and moving to a new build system can be a next step.
The current build system may not be perfect, may be tricky to setup at first. Most recently updated docs are at tools/appliance/README.md, if anyone has issues you may ping me and I may be able to help.
Now back to the topic of supporting debian9 as systemvmtemplate base, I was able to get something up and running this weekend and get serial console. I could also verify some patching done by the systemvm.iso file however I'm facing issues with running cloud-early-config, postinit and cloud services in a certain order and I need help around the systemd scripts.
Here's the PR:
https://github.com/apache/cloudstack/pull/2211
The PR branch is pushed on ASF remote so any committer can collaborate with me by pushing changes/fixes as a separate commit on that branch, feel free to do so. Thanks.
- Rohit
________________________________
From: Paul Angus <pa...@shapeblue.com>
Sent: Monday, July 31, 2017 11:11:53 AM
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] Move to Debian9 systemvmtemplate
Depending on the timescales that we looking at, if we can get an agreement to use Packer going forward, there is an argument to say that spending time getting the Debian 9 template to work on VeeWee and then on Packer is wasted effort and that we should just use this as the opportunity to move over to Packer/Debian9.
Having spent the weekend fighting with RVM/VeeWee/Ruby. And finding that we've hard linked rvm to Ruby 2.1.1 when it's now on 2.4, Veewee hasn't been updated for years and other mismatches. I'm very interested to see other options explored.
Veewee doesn't do the disk conversions at them moment, so we can still keep that a separate process for corner cases that Packer (or something else) can't manage..
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
rohit.yadav@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
Sent: 28 July 2017 20:30
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
I think we can move to packer once we can get the Debian9 based systemvmtemplate to work.
I think we should focus on doing this first and then focus on migration to a new build system as a next step.
I spent some time today and with some help from veewee authors, I could get a base template up and running:
https://github.com/apache/cloudstack/pull/2211
The above PR branch is pushed on ASF remote and allows for cross-collaboration with all ACS committers. Please collaborate with me on this and feel free to push changes on the branch as separate commits and/or make changes to the PR. Thanks.
- Rohit
________________________________
From: Tim Mackey <tm...@gmail.com>
Sent: Friday, July 28, 2017 3:59:36 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
Syed,
I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It shouldn't be hard to validate with newest XS and updated Packer - I just lack the infra to do the testing.
[1] https://github.com/xenserverarmy/packer
-tim
rohit.yadav@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO.
> I've used Packer before and I really like it, the only downside that I
> see is that Packer lacks support for XenServer VHD images. There is
> some work on a XenServer plugin but I haven't tested that. If the
> community decides to use Packer, I can do some initial validation of it on XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser <ma...@renemoser.net>:
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to
> > > > Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.com/apache/
> > cloudstack/pull/2198
> > >
> > > Have you considered to replace veewee by packer?
> > >
> >
> > Packer is really nice indeed. We use it to build our templates [0]
> > which we use on CloudStack.
> >
> > Building the SSVM using Packer should be rather easy I think.
> >
> > [0]: https://github.com/pcextreme/packer-templates
> >
> > > Our friends from schubergphilis have already done some work here
> > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > >
> > > However there would be also an official way to convert the
> > > definitions https://www.packer.io/guides/veewee-to-packer.html
> > >
> > > Regards René
> >
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Satki Badal <sa...@gmail.com>.
The whole ruby platform is running out of steam, high time to move to Packer while moving system VMS to Debian9.
- Satki
> On 31-Jul-2017, at 2:41 PM, Paul Angus <pa...@shapeblue.com> wrote:
>
> Depending on the timescales that we looking at, if we can get an agreement to use Packer going forward, there is an argument to say that spending time getting the Debian 9 template to work on VeeWee and then on Packer is wasted effort and that we should just use this as the opportunity to move over to Packer/Debian9.
>
> Having spent the weekend fighting with RVM/VeeWee/Ruby. And finding that we've hard linked rvm to Ruby 2.1.1 when it's now on 2.4, Veewee hasn't been updated for years and other mismatches. I'm very interested to see other options explored.
>
> Veewee doesn't do the disk conversions at them moment, so we can still keep that a separate process for corner cases that Packer (or something else) can't manage..
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
> Sent: 28 July 2017 20:30
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
>
> I think we can move to packer once we can get the Debian9 based systemvmtemplate to work.
>
> I think we should focus on doing this first and then focus on migration to a new build system as a next step.
>
>
> I spent some time today and with some help from veewee authors, I could get a base template up and running:
>
> https://github.com/apache/cloudstack/pull/2211
>
>
> The above PR branch is pushed on ASF remote and allows for cross-collaboration with all ACS committers. Please collaborate with me on this and feel free to push changes on the branch as separate commits and/or make changes to the PR. Thanks.
>
>
> - Rohit
>
> ________________________________
> From: Tim Mackey <tm...@gmail.com>
> Sent: Friday, July 28, 2017 3:59:36 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
>
> Syed,
>
> I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
> My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It shouldn't be hard to validate with newest XS and updated Packer - I just lack the infra to do the testing.
>
> [1] https://github.com/xenserverarmy/packer
>
> -tim
>
>
> rohit.yadav@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>
>
>
> On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>
>> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO.
>> I've used Packer before and I really like it, the only downside that I
>> see is that Packer lacks support for XenServer VHD images. There is
>> some work on a XenServer plugin but I haven't tested that. If the
>> community decides to use Packer, I can do some initial validation of it on XenServer.
>>
>> Thanks,
>> -Syed
>>
>> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <wi...@widodh.nl>
>> wrote:
>>
>>>
>>>> Op 24 juli 2017 om 19:07 schreef Rene Moser <ma...@renemoser.net>:
>>>>
>>>>
>>>> Hi Rohit
>>>>
>>>>
>>>> On 07/23/2017 06:08 PM, Rohit Yadav wrote:
>>>>> All,
>>>>>
>>>>>
>>>>> Just want to kick an initial discussion around migration to
>>>>> Debian9
>>> based systemvmtemplate, and get your feedback on the same.
>>>>>
>>>>> Here's a work-in-progress PR: https://github.com/apache/
>>> cloudstack/pull/2198
>>>>
>>>> Have you considered to replace veewee by packer?
>>>>
>>>
>>> Packer is really nice indeed. We use it to build our templates [0]
>>> which we use on CloudStack.
>>>
>>> Building the SSVM using Packer should be rather easy I think.
>>>
>>> [0]: https://github.com/pcextreme/packer-templates
>>>
>>>> Our friends from schubergphilis have already done some work here
>>>> https://github.com/MissionCriticalCloud/systemvm-packer.
>>>>
>>>> However there would be also an official way to convert the
>>>> definitions https://www.packer.io/guides/veewee-to-packer.html
>>>>
>>>> Regards René
>>>
>>
>
RE: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Paul Angus <pa...@shapeblue.com>.
Depending on the timescales that we looking at, if we can get an agreement to use Packer going forward, there is an argument to say that spending time getting the Debian 9 template to work on VeeWee and then on Packer is wasted effort and that we should just use this as the opportunity to move over to Packer/Debian9.
Having spent the weekend fighting with RVM/VeeWee/Ruby. And finding that we've hard linked rvm to Ruby 2.1.1 when it's now on 2.4, Veewee hasn't been updated for years and other mismatches. I'm very interested to see other options explored.
Veewee doesn't do the disk conversions at them moment, so we can still keep that a separate process for corner cases that Packer (or something else) can't manage..
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
Sent: 28 July 2017 20:30
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
I think we can move to packer once we can get the Debian9 based systemvmtemplate to work.
I think we should focus on doing this first and then focus on migration to a new build system as a next step.
I spent some time today and with some help from veewee authors, I could get a base template up and running:
https://github.com/apache/cloudstack/pull/2211
The above PR branch is pushed on ASF remote and allows for cross-collaboration with all ACS committers. Please collaborate with me on this and feel free to push changes on the branch as separate commits and/or make changes to the PR. Thanks.
- Rohit
________________________________
From: Tim Mackey <tm...@gmail.com>
Sent: Friday, July 28, 2017 3:59:36 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
Syed,
I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It shouldn't be hard to validate with newest XS and updated Packer - I just lack the infra to do the testing.
[1] https://github.com/xenserverarmy/packer
-tim
rohit.yadav@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO.
> I've used Packer before and I really like it, the only downside that I
> see is that Packer lacks support for XenServer VHD images. There is
> some work on a XenServer plugin but I haven't tested that. If the
> community decides to use Packer, I can do some initial validation of it on XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser <ma...@renemoser.net>:
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to
> > > > Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.com/apache/
> > cloudstack/pull/2198
> > >
> > > Have you considered to replace veewee by packer?
> > >
> >
> > Packer is really nice indeed. We use it to build our templates [0]
> > which we use on CloudStack.
> >
> > Building the SSVM using Packer should be rather easy I think.
> >
> > [0]: https://github.com/pcextreme/packer-templates
> >
> > > Our friends from schubergphilis have already done some work here
> > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > >
> > > However there would be also an official way to convert the
> > > definitions https://www.packer.io/guides/veewee-to-packer.html
> > >
> > > Regards René
> >
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Rohit Yadav <ro...@shapeblue.com>.
I think we can move to packer once we can get the Debian9 based systemvmtemplate to work.
I think we should focus on doing this first and then focus on migration to a new build system as a next step.
I spent some time today and with some help from veewee authors, I could get a base template up and running:
https://github.com/apache/cloudstack/pull/2211
The above PR branch is pushed on ASF remote and allows for cross-collaboration with all ACS committers. Please collaborate with me on this and feel free to push changes on the branch as separate commits and/or make changes to the PR. Thanks.
- Rohit
________________________________
From: Tim Mackey <tm...@gmail.com>
Sent: Friday, July 28, 2017 3:59:36 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
Syed,
I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It
shouldn't be hard to validate with newest XS and updated Packer - I just
lack the infra to do the testing.
[1] https://github.com/xenserverarmy/packer
-tim
rohit.yadav@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. I've
> used Packer before and I really like it, the only downside that I see is
> that Packer lacks support for XenServer VHD images. There is some work on a
> XenServer plugin but I haven't tested that. If the community decides to use
> Packer, I can do some initial validation of it on XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser <ma...@renemoser.net>:
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.com/apache/
> > cloudstack/pull/2198
> > >
> > > Have you considered to replace veewee by packer?
> > >
> >
> > Packer is really nice indeed. We use it to build our templates [0] which
> > we use on CloudStack.
> >
> > Building the SSVM using Packer should be rather easy I think.
> >
> > [0]: https://github.com/pcextreme/packer-templates
> >
> > > Our friends from schubergphilis have already done some work here
> > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > >
> > > However there would be also an official way to convert the definitions
> > > https://www.packer.io/guides/veewee-to-packer.html
> > >
> > > Regards René
> >
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Syed Ahmed <sa...@cloudops.com>.
This is awesome Tim. Any chance we merge it upstream with Packer?
On Thu, Jul 27, 2017 at 9:59 PM, Tim Mackey <tm...@gmail.com> wrote:
> Syed,
>
> I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
> My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It
> shouldn't be hard to validate with newest XS and updated Packer - I just
> lack the infra to do the testing.
>
> [1] https://github.com/xenserverarmy/packer
>
> -tim
>
> On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sa...@cloudops.com> wrote:
>
> > -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. I've
> > used Packer before and I really like it, the only downside that I see is
> > that Packer lacks support for XenServer VHD images. There is some work
> on a
> > XenServer plugin but I haven't tested that. If the community decides to
> use
> > Packer, I can do some initial validation of it on XenServer.
> >
> > Thanks,
> > -Syed
> >
> > On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <wi...@widodh.nl>
> > wrote:
> >
> > >
> > > > Op 24 juli 2017 om 19:07 schreef Rene Moser <ma...@renemoser.net>:
> > > >
> > > >
> > > > Hi Rohit
> > > >
> > > >
> > > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > > All,
> > > > >
> > > > >
> > > > > Just want to kick an initial discussion around migration to Debian9
> > > based systemvmtemplate, and get your feedback on the same.
> > > > >
> > > > > Here's a work-in-progress PR: https://github.com/apache/
> > > cloudstack/pull/2198
> > > >
> > > > Have you considered to replace veewee by packer?
> > > >
> > >
> > > Packer is really nice indeed. We use it to build our templates [0]
> which
> > > we use on CloudStack.
> > >
> > > Building the SSVM using Packer should be rather easy I think.
> > >
> > > [0]: https://github.com/pcextreme/packer-templates
> > >
> > > > Our friends from schubergphilis have already done some work here
> > > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > > >
> > > > However there would be also an official way to convert the
> definitions
> > > > https://www.packer.io/guides/veewee-to-packer.html
> > > >
> > > > Regards René
> > >
> >
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Tim Mackey <tm...@gmail.com>.
Syed,
I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It
shouldn't be hard to validate with newest XS and updated Packer - I just
lack the infra to do the testing.
[1] https://github.com/xenserverarmy/packer
-tim
On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sa...@cloudops.com> wrote:
> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. I've
> used Packer before and I really like it, the only downside that I see is
> that Packer lacks support for XenServer VHD images. There is some work on a
> XenServer plugin but I haven't tested that. If the community decides to use
> Packer, I can do some initial validation of it on XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <wi...@widodh.nl>
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser <ma...@renemoser.net>:
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.com/apache/
> > cloudstack/pull/2198
> > >
> > > Have you considered to replace veewee by packer?
> > >
> >
> > Packer is really nice indeed. We use it to build our templates [0] which
> > we use on CloudStack.
> >
> > Building the SSVM using Packer should be rather easy I think.
> >
> > [0]: https://github.com/pcextreme/packer-templates
> >
> > > Our friends from schubergphilis have already done some work here
> > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > >
> > > However there would be also an official way to convert the definitions
> > > https://www.packer.io/guides/veewee-to-packer.html
> > >
> > > Regards René
> >
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Syed Ahmed <sa...@cloudops.com>.
-1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. I've
used Packer before and I really like it, the only downside that I see is
that Packer lacks support for XenServer VHD images. There is some work on a
XenServer plugin but I haven't tested that. If the community decides to use
Packer, I can do some initial validation of it on XenServer.
Thanks,
-Syed
On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <wi...@widodh.nl> wrote:
>
> > Op 24 juli 2017 om 19:07 schreef Rene Moser <ma...@renemoser.net>:
> >
> >
> > Hi Rohit
> >
> >
> > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > All,
> > >
> > >
> > > Just want to kick an initial discussion around migration to Debian9
> based systemvmtemplate, and get your feedback on the same.
> > >
> > > Here's a work-in-progress PR: https://github.com/apache/
> cloudstack/pull/2198
> >
> > Have you considered to replace veewee by packer?
> >
>
> Packer is really nice indeed. We use it to build our templates [0] which
> we use on CloudStack.
>
> Building the SSVM using Packer should be rather easy I think.
>
> [0]: https://github.com/pcextreme/packer-templates
>
> > Our friends from schubergphilis have already done some work here
> > https://github.com/MissionCriticalCloud/systemvm-packer.
> >
> > However there would be also an official way to convert the definitions
> > https://www.packer.io/guides/veewee-to-packer.html
> >
> > Regards René
>
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Wido den Hollander <wi...@widodh.nl>.
> Op 24 juli 2017 om 19:07 schreef Rene Moser <ma...@renemoser.net>:
>
>
> Hi Rohit
>
>
> On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > All,
> >
> >
> > Just want to kick an initial discussion around migration to Debian9 based systemvmtemplate, and get your feedback on the same.
> >
> > Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
>
> Have you considered to replace veewee by packer?
>
Packer is really nice indeed. We use it to build our templates [0] which we use on CloudStack.
Building the SSVM using Packer should be rather easy I think.
[0]: https://github.com/pcextreme/packer-templates
> Our friends from schubergphilis have already done some work here
> https://github.com/MissionCriticalCloud/systemvm-packer.
>
> However there would be also an official way to convert the definitions
> https://www.packer.io/guides/veewee-to-packer.html
>
> Regards René
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Rene Moser <ma...@renemoser.net>.
Hi Rohit
On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> All,
>
>
> Just want to kick an initial discussion around migration to Debian9 based systemvmtemplate, and get your feedback on the same.
>
> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
Have you considered to replace veewee by packer?
Our friends from schubergphilis have already done some work here
https://github.com/MissionCriticalCloud/systemvm-packer.
However there would be also an official way to convert the definitions
https://www.packer.io/guides/veewee-to-packer.html
Regards René
Re: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Rene Moser <ma...@renemoser.net>.
Hi
On 07/23/2017 06:21 PM, Paul Angus wrote:
>
> I think that we should look at lighter Distros like Arch Linux in order to get the boot times of the system VMs down.
> That in conjunction with improving the configuration will gives as a much leaner, meaner System VM.
I tend to -1 for Arch.
Debian is a proven mature Distro. It is used as base for many critical
systems e.g. VyOS or also cumulus OS, not to mention Ubuntu.
Arch linux was not designed as a server OS. It has a rolling release
policy and it is great if you want to use the latest releases, you have
to update often.
Because we usually do not update VR OS that often, Arch is IMHO not a
good candidate. On the other hand Debian is a good one.
I would also doubt that Arch "boots" faster than Debian. IMHO the ways
_we_ configure the VR is still the performance killer.
Let's stick with Debian.
RE: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Wido den Hollander <wi...@widodh.nl>.
> Op 23 juli 2017 om 18:21 schreef Paul Angus <pa...@shapeblue.com>:
>
>
>
> I think that we should look at lighter Distros like Arch Linux in order to get the boot times of the system VMs down.
Debian 9 uses systemd and that makes it boot a lot faster.
> That in conjunction with improving the configuration will gives as a much leaner, meaner System VM.
A lot of things can be done, but going to Debian 9 isn't that difficult.
Yes, we will need to fix a few things around systemd and such, but Debian is a proven and stable distribution.
Let's not try to do two things at the same time.
Our current Debian 7 is dangerously old and needs to be replaced. Debian 9 seems like a very good candidate to me.
Wido
>
> Kind regards,
>
> Paul Angus
>
> paul.angus@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK
> @shapeblue
>
>
>
>
> -----Original Message-----
> From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
> Sent: 23 July 2017 17:08
> To: dev@cloudstack.apache.org
> Subject: [DISCUSS] Move to Debian9 systemvmtemplate
>
> All,
>
>
> Just want to kick an initial discussion around migration to Debian9 based systemvmtemplate, and get your feedback on the same.
>
> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
>
>
> Regards.
>
> rohit.yadav@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
>
>
>
RE: [DISCUSS] Move to Debian9 systemvmtemplate
Posted by Paul Angus <pa...@shapeblue.com>.
I think that we should look at lighter Distros like Arch Linux in order to get the boot times of the system VMs down.
That in conjunction with improving the configuration will gives as a much leaner, meaner System VM.
Kind regards,
Paul Angus
paul.angus@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
-----Original Message-----
From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
Sent: 23 July 2017 17:08
To: dev@cloudstack.apache.org
Subject: [DISCUSS] Move to Debian9 systemvmtemplate
All,
Just want to kick an initial discussion around migration to Debian9 based systemvmtemplate, and get your feedback on the same.
Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
Regards.
rohit.yadav@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue