You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Chiradeep Vittal <Ch...@citrix.com> on 2014/04/29 20:24:36 UTC
[DISCUSS] LXC systemvm
Hi,
I was playing around with LXC and I think I have the systemvm build
figured out. Currently [1] the LXC system requires a KVM/Xen-based
systemvm.
I¹ve tried this out on Ubuntu 14.04 (trusty)
Download these 2 files:
create-lxc-systemvm.sh https://gist.github.com/chiradeep/11388262
provision.rb https://gist.github.com/chiradeep/11407727#file-provision-rb
Execute:
./create-lxc-systemvm.sh
The Ruby script called by the shell script is used to ssh into a minimal
lxc wheezy container.
* Downloads a modified Œpostinstall.sh¹ from
https://gist.github.com/chiradeep/11388137
This is modified from http://goo.gl/28QYa2
* Executes the postinstall script.
Then the shell script tars up the rootfs.
What¹s missing is fixes to cloud-early-config. This should be much easier
than for other hypervisors since the host has write access to
/var/cache/cmdline inside the container.
I guess this is too late for 4.4, but worthwhile for 4.5.
Comments?
[1] https://cwiki.apache.org/confluence/x/oJNMAg
Re: [DISCUSS] LXC systemvm
Posted by Chiradeep Vittal <Ch...@citrix.com>.
Right now the intention is to work just like any other hypervisor: using VLAN or perhaps VxLAN/GRE for isolation.
Live migration isn’t possible, but fast restart should be possible as long as shared storage is used.
Some of the more interesting use cases are : web tier on LXC, app tier on hypervisor and DB tier on baremetal.
Of course a single LXC container on a single host should be equivalent to baremetal.
From: Rohit Yadav <bh...@apache.org>>
Reply-To: "dev@cloudstack.apache.org<ma...@cloudstack.apache.org>" <de...@cloudstack.apache.org>>
Date: Tuesday, April 29, 2014 at 12:07 PM
To: "dev@cloudstack.apache.org<ma...@cloudstack.apache.org>" <de...@cloudstack.apache.org>>
Subject: Re: [DISCUSS] LXC systemvm
On Tue, Apr 29, 2014 at 11:54 PM, Chiradeep Vittal <
Chiradeep.Vittal@citrix.com<ma...@citrix.com>> wrote:
Hi,
I was playing around with LXC and I think I have the systemvm build
figured out. Currently [1] the LXC system requires a KVM/Xen-based
systemvm.
w00t!
I¹ve tried this out on Ubuntu 14.04 (trusty)
Download these 2 files:
create-lxc-systemvm.sh https://gist.github.com/chiradeep/11388262
provision.rb https://gist.github.com/chiradeep/11407727#file-provision-rb
Execute:
./create-lxc-systemvm.sh
The Ruby script called by the shell script is used to ssh into a minimal
lxc wheezy container.
* Downloads a modified OEpostinstall.sh¹ from
https://gist.github.com/chiradeep/11388137
This is modified from http://goo.gl/28QYa2
* Executes the postinstall script.
Then the shell script tars up the rootfs.
What¹s missing is fixes to cloud-early-config. This should be much easier
than for other hypervisors since the host has write access to
/var/cache/cmdline inside the container.
I guess this is too late for 4.4, but worthwhile for 4.5.
Comments?
This looks great, Chiradeep can you share with us how the networking will
work in case of using systemvms in LXC containers, how it will communicate
with outside world and with other instances running in shared/separate LXC
containers. Will we need a loadbalancer (like nginx) for reverse proxying
and example or use case of a infra deployment. Lastly, will we have feature
of moving containers across machines and some distributed locking service
that the systemvms or agent may implement?
Regards.
[1] https://cwiki.apache.org/confluence/x/oJNMAg
Re: [DISCUSS] LXC systemvm
Posted by Rohit Yadav <bh...@apache.org>.
Thanks Chiradeep and Pedro for your replies. My $dayjob requires me to work
with baremetal servers, in fact almost all our infra consists of baremetal
servers which at times is difficult to work with, therefore I'm exploring
ways to introduce docker/lxc (containers), mesos (cluster management), etcd
(conf/locking) and a manager/controller (cloudstack?).
Pedro's idea is great, I've been reading about the concept of idempotent
servers (I think it was Chad Fowler and others) where you deploy a server
once and it sort of becomes immutable so you don't upgrade it, change its
config once in production and in case you need some changes you create a
new one and destroy old ones.
On Wed, Apr 30, 2014 at 10:20 AM, Pedro Roque Marques <
pedro.r.marques@gmail.com> wrote:
> Rohit,
>
> On Apr 29, 2014, at 12:07 PM, Rohit Yadav <bh...@apache.org> wrote:
> > This looks great, Chiradeep can you share with us how the networking will
> > work in case of using systemvms in LXC containers, how it will
> communicate
> > with outside world and with other instances running in shared/separate
> LXC
> > containers.
>
> LXC networking can be configured to work just like a VM; one typically
> associates the LXC with a linux network namespace and uses a "veth" port
> instead of a "tap" interface.
>
> For examples of how configure OpenContrail to use netns see:
> 1. https://github.com/pedro-r-marques/opencontrail-netns; (no
> orchestration system)
> 2.
> https://github.com/pedro-r-marques/nova-docker/blob/opencontrail/novadocker/virt/docker/opencontrail.py;
> (openstack + docker).
>
> The same recipes can be used to configure bridged interfaces or OVS;
>
> In the case of using OpenContrail you get the added benefit that it
> integrates functionality such as "security-groups", dhcp, vpc support all
> in one... so you don't need to worry about ip-chains and L3 agents.
>
> > Will we need a loadbalancer (like nginx) for reverse proxying
> > and example or use case of a infra deployment.
>
> The OpenStack crowd typically implements the load balancer and external
> router (aka cloudstack domain router) as an app running inside a netns.
>
> > Lastly, will we have feature
> > of moving containers across machines and some distributed locking service
> > that the systemvms or agent may implement?
>
> The neat thing about containers is how fast they start (and how little
> overhead they have) compared to VMs. To move containers, just kill one and
> start the other;
>
> Pedro.
Re: [DISCUSS] LXC systemvm
Posted by Pedro Roque Marques <pe...@gmail.com>.
Rohit,
On Apr 29, 2014, at 12:07 PM, Rohit Yadav <bh...@apache.org> wrote:
> This looks great, Chiradeep can you share with us how the networking will
> work in case of using systemvms in LXC containers, how it will communicate
> with outside world and with other instances running in shared/separate LXC
> containers.
LXC networking can be configured to work just like a VM; one typically associates the LXC with a linux network namespace and uses a "veth" port instead of a "tap" interface.
For examples of how configure OpenContrail to use netns see:
1. https://github.com/pedro-r-marques/opencontrail-netns; (no orchestration system)
2. https://github.com/pedro-r-marques/nova-docker/blob/opencontrail/novadocker/virt/docker/opencontrail.py; (openstack + docker).
The same recipes can be used to configure bridged interfaces or OVS;
In the case of using OpenContrail you get the added benefit that it integrates functionality such as "security-groups", dhcp, vpc support all in one... so you don't need to worry about ip-chains and L3 agents.
> Will we need a loadbalancer (like nginx) for reverse proxying
> and example or use case of a infra deployment.
The OpenStack crowd typically implements the load balancer and external router (aka cloudstack domain router) as an app running inside a netns.
> Lastly, will we have feature
> of moving containers across machines and some distributed locking service
> that the systemvms or agent may implement?
The neat thing about containers is how fast they start (and how little overhead they have) compared to VMs. To move containers, just kill one and start the other;
Pedro.
Re: [DISCUSS] LXC systemvm
Posted by Rohit Yadav <bh...@apache.org>.
On Tue, Apr 29, 2014 at 11:54 PM, Chiradeep Vittal <
Chiradeep.Vittal@citrix.com> wrote:
> Hi,
>
> I was playing around with LXC and I think I have the systemvm build
> figured out. Currently [1] the LXC system requires a KVM/Xen-based
> systemvm.
>
w00t!
>
> I¹ve tried this out on Ubuntu 14.04 (trusty)
>
> Download these 2 files:
> create-lxc-systemvm.sh https://gist.github.com/chiradeep/11388262
> provision.rb https://gist.github.com/chiradeep/11407727#file-provision-rb
>
> Execute:
> ./create-lxc-systemvm.sh
>
> The Ruby script called by the shell script is used to ssh into a minimal
> lxc wheezy container.
> * Downloads a modified OEpostinstall.sh¹ from
> https://gist.github.com/chiradeep/11388137
> This is modified from http://goo.gl/28QYa2
> * Executes the postinstall script.
>
> Then the shell script tars up the rootfs.
>
> What¹s missing is fixes to cloud-early-config. This should be much easier
> than for other hypervisors since the host has write access to
> /var/cache/cmdline inside the container.
>
> I guess this is too late for 4.4, but worthwhile for 4.5.
> Comments?
>
This looks great, Chiradeep can you share with us how the networking will
work in case of using systemvms in LXC containers, how it will communicate
with outside world and with other instances running in shared/separate LXC
containers. Will we need a loadbalancer (like nginx) for reverse proxying
and example or use case of a infra deployment. Lastly, will we have feature
of moving containers across machines and some distributed locking service
that the systemvms or agent may implement?
Regards.
>
> [1] https://cwiki.apache.org/confluence/x/oJNMAg
>
>
>