Mesos clusters and security boundaries

[Guido Bakker <gu...@gmail.com>]

Hi,

At the company I work for we are in the process of setting up/designing a
Mesos cluster. We do use it already for several projects & POC. But not in
a production setup yet.
We believe in the idea of DCOS and we would like to have the ability to
fully utilize all the available resources depending on demand, priority and
constraints.
But we're struggling a bit with the right separation of security boundaries
and amount of clusters that you need to setup.
Currently we have DTAXP environments that are logically separated by
network hardware. Within each environment we make the separation of DMZ,
Front-end, Back-end & Management. Again within each environment multiple
VLAN's.
Theoretically you could run Mesos on every server in your DC and have 1 big
cluster. But how do you maintain the same level of security, either
physical of logical?
Statically partition Mesos slave's for each DTAXP or have separate Mesos
clusters? Add overlay networking? The amount of options seem to increase
every week...

Maybe I'm missing the obvious, but I would love to hear about use-cases and
directions other people are going...

Regards,
Guido

Re: Mesos clusters and security boundaries

[Maxime Brugidou <ma...@gmail.com>]

You would have to break the vlans and push security into the
apps/frameworks. Vlans are a way to control communication between machines.
With mesos, the machine is not the unit of work since they tend to all
become identical and managed by a single team. What matters is the security
of the frameworks running over mesos. This requires big changes in many
apps relying on vlans or firewall to "secure" their TCP connections.

You would have to start working on authenticating connections, use TLS...
Etc. The hardest one is that vlans prevent you from DOS (either accidental
or not) from another unauthorized vlan. This is harder to do with Mesos
since at the network level you will receive the DOS connections and have to
reject them higher in the stack.

Anyway you could also work out a solution with tagged vlans and iptables
that restrict access to the vlan from a specific user. Then you authorize
some framework user to use a vlan on all slaves or not.
On Mar 18, 2015 1:20 PM, "Guido Bakker" <gu...@gmail.com> wrote:

> Hi,
>
> At the company I work for we are in the process of setting up/designing a
> Mesos cluster. We do use it already for several projects & POC. But not in
> a production setup yet.
> We believe in the idea of DCOS and we would like to have the ability to
> fully utilize all the available resources depending on demand, priority and
> constraints.
> But we're struggling a bit with the right separation of security
> boundaries and amount of clusters that you need to setup.
> Currently we have DTAXP environments that are logically separated by
> network hardware. Within each environment we make the separation of DMZ,
> Front-end, Back-end & Management. Again within each environment multiple
> VLAN's.
> Theoretically you could run Mesos on every server in your DC and have 1
> big cluster. But how do you maintain the same level of security, either
> physical of logical?
> Statically partition Mesos slave's for each DTAXP or have separate Mesos
> clusters? Add overlay networking? The amount of options seem to increase
> every week...
>
> Maybe I'm missing the obvious, but I would love to hear about use-cases
> and directions other people are going...
>
> Regards,
> Guido
>

Re: Resource allocation module

[Alex Rukletsov <al...@mesosphere.io>]

Hi Gidon,

and thanks for your interest. As you have already noticed, the work is
currently in progress and should land in master branch in around 2 weeks.
It will also be part of 0.23 release. There is no documentation so far, but
we plan to document the API once the patches land. Right now you may want
to look at the Allocator interface and check the DRF implementation for
more details.

—Alex

On Wed, Mar 18, 2015 at 7:05 AM, Gidon Gershinsky <GI...@il.ibm.com> wrote:

> We need to develop a new resource allocation module, replacing the
> off-the-shelf DRF.
> As I understand, the current mechanism
> http://mesos.apache.org/documentation/latest/allocation-module/
>
> is being replaced with a less intrusive module architecture,
> https://issues.apache.org/jira/browse/MESOS-2160
>
> The capabilities of the new mechanism have real advantages for us.
> However, it is not clear when it will be released. The jira has an 'in
> progress' status. What is the current target / horizon for making this
> available to the users? Also, is there any documentation on the SPIs /
> technical interfaces of these modules (what info is passed from slaves,
> frameworks, offers; what  calls can be made by the modules; etc)?
>
> Regards, Gidon

Resource allocation module

[Gidon Gershinsky <GI...@il.ibm.com>]

We need to develop a new resource allocation module, replacing the 
off-the-shelf DRF. 
As I understand, the current mechanism
http://mesos.apache.org/documentation/latest/allocation-module/

is being replaced with a less intrusive module architecture,
https://issues.apache.org/jira/browse/MESOS-2160

The capabilities of the new mechanism have real advantages for us. 
However, it is not clear when it will be released. The jira has an 'in 
progress' status. What is the current target / horizon for making this 
available to the users? Also, is there any documentation on the SPIs / 
technical interfaces of these modules (what info is passed from slaves, 
frameworks, offers; what  calls can be made by the modules; etc)?

Regards, Gidon