Stage multiple jobs/tasks on the same host

[Rogier Dikkes <ro...@surfsara.nl>]

Hello all,

Recently i have been playing with docker containers on Aurora. Now i 
have a few questions, the first of many:

Is it possible to combine/stage Aurora jobs to run on the same machine, 
or (preferred) is it possible to run some tasks inside the docker 
container and others outside while staged on the same machine?
Reasons could be:

- Mounting a block device on the host machine to expose to the container
- Setup a network with for example calico for container A to use
- Stage a second container with other processes to use

For me its the ability to setup a calico network per 'workload' container.

Thank you for your answers!

-- 
Rogier Dikkes
Systeem Programmeur Hadoop & HPC Cloud
SURFsara | Science Park 140 | 1098 XG Amsterdam

Re: Stage multiple jobs/tasks on the same host

[Bill Farner <wf...@apache.org>]

You could do this in an ad-hoc way with constraints (i.e. constrain the
tasks to the same host name or other attribute of your choice).

An approach that would require more heavy lifting on your end is to build a
custom executor that is responsible for launching the containers (likely
interacting directly with docker).  This would allow the scheduler to treat
the multiple containers as one schedulable 'unit' while the executor
decides how to manage the components of it.

On Sat, Nov 21, 2015 at 3:58 AM, Rogier Dikkes <ro...@surfsara.nl>
wrote:

> Hello all,
>
> Recently i have been playing with docker containers on Aurora. Now i have
> a few questions, the first of many:
>
> Is it possible to combine/stage Aurora jobs to run on the same machine, or
> (preferred) is it possible to run some tasks inside the docker container
> and others outside while staged on the same machine?
> Reasons could be:
>
> - Mounting a block device on the host machine to expose to the container
> - Setup a network with for example calico for container A to use
> - Stage a second container with other processes to use
>
> For me its the ability to setup a calico network per 'workload' container.
>
> Thank you for your answers!
>
> --
> Rogier Dikkes
> Systeem Programmeur Hadoop & HPC Cloud
> SURFsara | Science Park 140 | 1098 XG Amsterdam
>
>