[jira] [Updated] (MESOS-181) Virtual Machine Isolation Module

["Charles Earl (Updated) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MESOS-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Charles Earl updated MESOS-181:
-------------------------------

    Description: 
Earlier in the year I implemented a virtual machine isolation module. This module uses lib-virt to launch and manage virtual machine containers. The code is still rough and have done basic testing with the Spark example. 

This code works with the KVM (http://www.linux-kvm.org/page/Main_Page) virtual machine manager. I've placed the relevant code in a branch called mesos-vm, for now located at https://github.com/charlescearl/VirtualMesos. The code is based upon the mesos lxc isolation module that is located in src/slave/lxc_isolation_module.cpp/.hpp. My code based on the mesos master branch dated Wed Nov 23 12:02:07 2011 -0800, commit 059aabb2ec5bd7b20ed08ab9c439531a352ba3ec. I'll generate a patch soon for this. Suggestions appreciated on whether this is the appropriate branch/commit to patch against.

Most of the implementation is contained in vm_isolation_module.cpp and vm_isolation_module.hpp and there are some minor additions in launcher to handle setup of the environment for the virtual machine. I use the libvirt (http://libvirt.org/) library, to manage the virtual machine container in which the jobs are executed.

Dependencies
The code has been tested on Ubuntu 11.04 and 11.10 and depends on libpython2.6 and libvirt0

Configuration of the virtual machine container
The virtual machine invocation depends upon a few configuration assumptions:
 1.   ssh public keys installed on the container. I assume that the container is setup to allow password-less secure access.
  2.  Directory structure on the container matches the servant machine. For example, in invoking a spark executor, assume that the paths match the setup on the container host.

Running it
In the $MESOS_HOME/conf/mesos.conf file add the line 
   isolation=vm
to use the virtual machine isolation.

The Mesos slave is invoked with the isolation parameter set to vm. For example
     sudo bin/mesos-slave -m mesos://master@mesos-host:5050 -w 9839 --isolation=vm

Rough description of how it works

The `vm_isolation_module` class forks a process that in turn launches a virtual machine.  A routine  located in bin called find_addr.pl is responsible for figuring out the IP address of the launched virtual machine. This is probably not portable since it is explicitly looking for entry in the virbr0 network.

A script vmLauncherTemplate.sh located in bin assists the the vmLauncher method to setup the environment for launching tasks inside of the virtual machine. The vmLauncher method uses vmLauncherTemplate.sh  to create a tasks specific shell vmLauncherTemplate-<task_id>.sh, which is copied to the running guest and used to run the executor inside the VM. This communicates with the slave on the host.

Comments and suggestions on improvements and next directions are appreciated!

  was:
Earlier in the year I implemented a virtual machine isolation module. This module uses lib-virt to launch and manage virtual machine containers. The code is still rough and have done basic testing with the Spark example. I will add the more description and also the patch over the next few days. 
Comments and suggestions on improvements and next directions are appreciated.

    Environment: Ubuntu 11.04, Ubuntu 11.10
    
> Virtual Machine Isolation Module
> --------------------------------
>
>                 Key: MESOS-181
>                 URL: https://issues.apache.org/jira/browse/MESOS-181
>             Project: Mesos
>          Issue Type: New Feature
>          Components: isolation, slave
>         Environment: Ubuntu 11.04, Ubuntu 11.10
>            Reporter: Charles Earl
>            Priority: Minor
>              Labels: virtualiztion
>
> Earlier in the year I implemented a virtual machine isolation module. This module uses lib-virt to launch and manage virtual machine containers. The code is still rough and have done basic testing with the Spark example. 
> This code works with the KVM (http://www.linux-kvm.org/page/Main_Page) virtual machine manager. I've placed the relevant code in a branch called mesos-vm, for now located at https://github.com/charlescearl/VirtualMesos. The code is based upon the mesos lxc isolation module that is located in src/slave/lxc_isolation_module.cpp/.hpp. My code based on the mesos master branch dated Wed Nov 23 12:02:07 2011 -0800, commit 059aabb2ec5bd7b20ed08ab9c439531a352ba3ec. I'll generate a patch soon for this. Suggestions appreciated on whether this is the appropriate branch/commit to patch against.
> Most of the implementation is contained in vm_isolation_module.cpp and vm_isolation_module.hpp and there are some minor additions in launcher to handle setup of the environment for the virtual machine. I use the libvirt (http://libvirt.org/) library, to manage the virtual machine container in which the jobs are executed.
> Dependencies
> The code has been tested on Ubuntu 11.04 and 11.10 and depends on libpython2.6 and libvirt0
> Configuration of the virtual machine container
> The virtual machine invocation depends upon a few configuration assumptions:
>  1.   ssh public keys installed on the container. I assume that the container is setup to allow password-less secure access.
>   2.  Directory structure on the container matches the servant machine. For example, in invoking a spark executor, assume that the paths match the setup on the container host.
> Running it
> In the $MESOS_HOME/conf/mesos.conf file add the line 
>    isolation=vm
> to use the virtual machine isolation.
> The Mesos slave is invoked with the isolation parameter set to vm. For example
>      sudo bin/mesos-slave -m mesos://master@mesos-host:5050 -w 9839 --isolation=vm
> Rough description of how it works
> The `vm_isolation_module` class forks a process that in turn launches a virtual machine.  A routine  located in bin called find_addr.pl is responsible for figuring out the IP address of the launched virtual machine. This is probably not portable since it is explicitly looking for entry in the virbr0 network.
> A script vmLauncherTemplate.sh located in bin assists the the vmLauncher method to setup the environment for launching tasks inside of the virtual machine. The vmLauncher method uses vmLauncherTemplate.sh  to create a tasks specific shell vmLauncherTemplate-<task_id>.sh, which is copied to the running guest and used to run the executor inside the VM. This communicates with the slave on the host.
> Comments and suggestions on improvements and next directions are appreciated!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira