using Mesos in a multi-users environment

[Charles-François Natali <cf...@gmail.com>]

Hi,

We've been contemplating using Mesos at the company I work for.
Basically we would be writing our own framework, and various users
would submit jobs to it.
However a remaining problem to us is the security model: anyone which
is allowed to connect to the Mesos cluster can start tasks as any user
- by setting the "user" in the CommandInfo. Which means that the whole
framework code would basically have to be trusted, which would be
problematic for various reasons.
Is there no better way to ensure that a task is actually executed by
the submitting user?

1. A potential solution would be to write a standalone audited proxy
process which would validate the submitted TaskInfo and forward it to
the Mesos cluster, but it's a bit awkward.

2. Another potential solution involves a custom executor: for example
we want to write our own executor anyway to have end-to-end validation
of user commands, i.e. have each user sign its job before submitting
it, and have the executor validate the signature and check it's
running as the correct user before executing the task. The above
problem would be solved if there was a way to restrict the agents to
only execute this trusted executor - is that possible short of hacking
the source code?

Any suggestion is welcome!

Cheers,

Charles

Re: using Mesos in a multi-users environment

[Vinod Kone <vi...@apache.org>]

Hi  Charles,


> > We've been contemplating using Mesos at the company I work for.
> > Basically we would be writing our own framework, and various users
> > would submit jobs to it.
> > However a remaining problem to us is the security model: anyone which
> > is allowed to connect to the Mesos cluster can start tasks as any user
> > - by setting the "user" in the CommandInfo. Which means that the whole
> > framework code would basically have to be trusted, which would be
> > problematic for various reasons.
> > Is there no better way to ensure that a task is actually executed by
> > the submitting user?
>

I don't completely follow your threat model. Are you concerned that a
malicious user can connect to Mesos as a framework and launch tasks? You
can restrict which principal can connect to Mesos as a framework using the
`register_frameworks` ACL and the built-in authorizer. See the
authorization documentation here
<http://mesos.apache.org/documentation/latest/authorization/>. There's also
a `run_tasks` ACL that you could leverage to restrict which OS users a
framework can run tasks as. To authenticate a given principal you could use
the built-in authenticator. If the built-in authenticator/authorizer turn
out to be too basic for your use case, I would recommend writing your
authorizer/authenticator module to plugin in to your company's security
infrastructure.

HTH,

Re: using Mesos in a multi-users environment

[Charles-François Natali <cf...@gmail.com>]

Thinking about it more, I think that the best solution might be to run
the agents as an unprivileged user, and use the default command
executor to execute a setuid root process written and audited by us,
which would verify the user's job signature and then setuid
 to the target user and for/exec the target command.

This way we don't need to trust the framework, we don't rely on Mesos
authentication, and don't have to write our own executor.


Le sam. 19 oct. 2019 à 12:20, Charles-François Natali
<cf...@gmail.com> a écrit :
>
> Hi,
>
> We've been contemplating using Mesos at the company I work for.
> Basically we would be writing our own framework, and various users
> would submit jobs to it.
> However a remaining problem to us is the security model: anyone which
> is allowed to connect to the Mesos cluster can start tasks as any user
> - by setting the "user" in the CommandInfo. Which means that the whole
> framework code would basically have to be trusted, which would be
> problematic for various reasons.
> Is there no better way to ensure that a task is actually executed by
> the submitting user?
>
> 1. A potential solution would be to write a standalone audited proxy
> process which would validate the submitted TaskInfo and forward it to
> the Mesos cluster, but it's a bit awkward.
>
> 2. Another potential solution involves a custom executor: for example
> we want to write our own executor anyway to have end-to-end validation
> of user commands, i.e. have each user sign its job before submitting
> it, and have the executor validate the signature and check it's
> running as the correct user before executing the task. The above
> problem would be solved if there was a way to restrict the agents to
> only execute this trusted executor - is that possible short of hacking
> the source code?
>
> Any suggestion is welcome!
>
> Cheers,
>
> Charles