[jira] [Comment Edited] (MESOS-3711) Docker containers running as other than root can't access sandbox

["Ian Downes (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954109#comment-14954109 ] 

Ian Downes edited comment on MESOS-3711 at 10/13/15 12:24 AM:
--------------------------------------------------------------

The task work directory (the top-level sandbox) is created and chown'ed to the user specified by the task, assuming one is specified and {{--switch_user}} is true (default), otherwise it's created and chown'ed to the user running the slave. See {{paths::createExecutorDirectory()}} as used during {{launchExecutor()}} in {{slave/slave.cpp}}.

This doesn't seem to match your stated behavior: if the task does specify the user then the top-level sandbox will be chown'ed to that user and Thermos, running as that user, should be okay to create the sandbox subdirectory.

*BUT*, note the TODO comment from [~nnielsen] (nnielsen) in the implementation of {{createExecutorDirectory()}} and the implications: the user *name* is used for the chown so the chown call will fail if the name doesn't resolve to a UID on the host (i.e., is present in {{/etc/passwd}}). Furthermore, *it's only logged with a warning and the slave will continue with the executor work directory owned by the user running the slave*, I assume this is root? So, what I think is happening is you are specifying a user name that (at some point) was valid inside the container filesystem (mapped to a UID) but is not valid on the host when launching the container (not present in the host's {{/etc/passwd}}). Does that make sense?


was (Author: idownes):
The task work directory (the top-level sandbox) is created and chown'ed to the user specified by the task, assuming one is specified and {{--switch_user is true}} (default), otherwise it's created and chown'ed to the user running the slave. See {{paths::createExecutorDirectory()}} as used during {{launchExecutor()}} in {{slave/slave.cpp}}.

This doesn't seem to match your stated behavior: if the task does specify the user then the top-level sandbox will be chown'ed to that user and Thermos, running as that user, should be okay to create the sandbox subdirectory.

*BUT*, note the TODO comment from [~nnielsen] (nnielsen) in the implementation of {{createExecutorDirectory()}} and the implications: the user *name* is used for the chown so the chown call will fail if the name doesn't resolve to a UID on the host (i.e., is present in {{/etc/passwd}}). Furthermore, *it's only logged with a warning and the slave will continue with the executor work directory owned by the user running the slave*, I assume this is root? So, what I think is happening is you are specifying a user name that (at some point) was valid inside the container filesystem (mapped to a UID) but is not valid on the host when launching the container (not present in the host's {{/etc/passwd}}). Does that make sense?

> Docker containers running as other than root can't access sandbox
> -----------------------------------------------------------------
>
>                 Key: MESOS-3711
>                 URL: https://issues.apache.org/jira/browse/MESOS-3711
>             Project: Mesos
>          Issue Type: Bug
>          Components: containerization, docker
>            Reporter: Vasilis Vasaitis
>
> (Disclaimer: I'm not the one running the Mesos infrastructure in my org, and I don't necessarily fully understand how all the moving parts fit together, so please bear with me if there any gaps in my understanding of the issues at hand.)
> We have a setup here where we deploy Docker-based tasks on Mesos, using Aurora (and thus the Thermos executor, on the agent side). As part of the process of launching a task, it looks like the Mesos agent creates / volume-mounts an {{/mnt/mesos/sandbox}} directory, which is what's used as the task's sandbox. Thermos then creates a {{sandbox}} subdirectory _inside_ that, and the aggregate {{/mnt/mesos/sandbox/sandbox}} is in fact the directory that the user application is given. So far so good.
> Now, Docker has the option, during the creation of a Docker image, to specify the _user_ that any container launched using this image will be run as. This is a useful feature, because often the image is built so that only one particular user has ownership of important files etc. One could of course sidestep this issue by always launching the container as root, but that can be unsavoury for its own reasons.
> However, with the setup I described above, specifying a user for the Docker container quickly goes south: the Thermos executor itself is launched as that user, tries to create that extra {{sandbox}} directory, and fails, because the parent directory is owned by root.
> I won't claim to know whether this is the _best_ approach, but one possible solution to this problem is to chmod 1777 the parent sandbox directory (i.e., set the sticky bit, like {{/tmp}}) after creating it; this way any user can create files/directories under it, without compromising the isolation between users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)