[jira] [Updated] (MESOS-3711) Docker containers running as other than root can't access sandbox

["Vasilis Vasaitis (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MESOS-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vasilis Vasaitis updated MESOS-3711:
------------------------------------
    Description: 
(Disclaimer: I'm not the one running the Mesos infrastructure in my org, and I don't necessarily fully understand how all the moving parts fit together, so please bear with me if there any gaps in my understanding of the issues at hand.)

We have a setup here where we deploy Docker-based tasks on Mesos, using Aurora (and thus the Thermos executor, on the agent side). As part of the process of launching a task, it looks like the Mesos agent creates / volume-mounts an {{/mnt/mesos/sandbox}} directory, which is what's used as the task's sandbox. Thermos then creates a {{sandbox}} subdirectory _inside_ that, and the aggregate {{/mnt/mesos/sandbox/sandbox}} is in fact the directory that the user application is given. So far so good.

Now, Docker has the option, during the creation of a Docker image, to specify the _user_ that any container launched using this image will be run as. This is a useful feature, because often the image is built so that only one particular user has ownership of important files etc. One could of course sidestep this issue by always launching the container as root, but that can be unsavoury for its own reasons.

However, with the setup I described above, specifying a user for the Docker container quickly goes south: the Thermos executor itself is launched as that user, tries to create that extra {{sandbox}} directory, and fails, because the parent directory is owned by root.

I won't claim to know whether this is the _best_ approach, but one possible solution to this problem is to chmod 1777 the parent sandbox directory (i.e., set the sticky bit, like {{/tmp}}) after creating it; this way any user can create files/directories under it, without compromising the isolation between users.

  was:
(Disclaimer: I'm not the one running the Mesos infrastructure in my org, and I don't necessarily fully understand how all the moving parts fit together, so please bear with me if there any gaps in my understanding of the issues at hand.)

We have a setup here where we deploy Docker-based tasks on Mesos, using Aurora (and thus the Thermos executor, on the agent side). As part of the process of launching a task, it looks like the Mesos agent creates / volume-mounts an {{/mnt/mesos/sandbox}} directory, which is what's used as the task's sandbox. Thermos then creates a {{sandbox}} subdirectory _inside_ that, and the aggregate {{/mnt/mesos/sandbox/sandbox}} is in fact the directory that the user application is given. So far so good.

Now, Docker has the option, during the creation of a Docker image, to specify the _user_ that any container launched using this image will be run as. This is a useful feature, because often the image is built so that only one particular user has ownership of important files etc. One could of course sidestep this issue by always launching the container as root, but that can be unsavoury for its own reasons.

However, with the setup I described above, specifying a user for the Docker container quickly goes south: the Thermos executor itself is launched as that user, tries to create that extra {{sandbox}} directory, and fails, because the parent directory is owned by root.

I won't claim to know whether this is the _best_ approach, but one possible solution to this problem is to chmod 1777 the parent sandbox directory (i.e., set the sticky bit, like {{/tmp}}) after creating it; this way any user can create files/directories under it, without compromising the isolation between users. I plan to attach a small patch which demonstrates this change.


> Docker containers running as other than root can't access sandbox
> -----------------------------------------------------------------
>
>                 Key: MESOS-3711
>                 URL: https://issues.apache.org/jira/browse/MESOS-3711
>             Project: Mesos
>          Issue Type: Bug
>          Components: containerization, docker
>            Reporter: Vasilis Vasaitis
>
> (Disclaimer: I'm not the one running the Mesos infrastructure in my org, and I don't necessarily fully understand how all the moving parts fit together, so please bear with me if there any gaps in my understanding of the issues at hand.)
> We have a setup here where we deploy Docker-based tasks on Mesos, using Aurora (and thus the Thermos executor, on the agent side). As part of the process of launching a task, it looks like the Mesos agent creates / volume-mounts an {{/mnt/mesos/sandbox}} directory, which is what's used as the task's sandbox. Thermos then creates a {{sandbox}} subdirectory _inside_ that, and the aggregate {{/mnt/mesos/sandbox/sandbox}} is in fact the directory that the user application is given. So far so good.
> Now, Docker has the option, during the creation of a Docker image, to specify the _user_ that any container launched using this image will be run as. This is a useful feature, because often the image is built so that only one particular user has ownership of important files etc. One could of course sidestep this issue by always launching the container as root, but that can be unsavoury for its own reasons.
> However, with the setup I described above, specifying a user for the Docker container quickly goes south: the Thermos executor itself is launched as that user, tries to create that extra {{sandbox}} directory, and fails, because the parent directory is owned by root.
> I won't claim to know whether this is the _best_ approach, but one possible solution to this problem is to chmod 1777 the parent sandbox directory (i.e., set the sticky bit, like {{/tmp}}) after creating it; this way any user can create files/directories under it, without compromising the isolation between users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)