[jira] [Comment Edited] (MESOS-6340) Set HOME for Mesos tasks

["Joshua Cohen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15562768#comment-15562768 ] 

Joshua Cohen edited comment on MESOS-6340 at 10/10/16 4:49 PM:
---------------------------------------------------------------

Yes, this is relatively accurate. Thermos currently has some pretty janky logic to set {{$HOME}}, which amounts to:

* If running a task with an isolated filesystem (i.e. unified containerizer with an {{Image}} set), then use {{/mnt/mesos/sandbox}} (which is bind mounted to the rootfs for the isolated task filesystem).
* Otherwise use {{$MESOS_DIRECTORY/sandbox}} (i.e. a path we previously created under {{$MESOS_DIRECTORY}}).

I think given the current implementation of filesystem isolation in Thermos, the first bit of logic is pretty specific to our implementation though (we don't set the filesystem {{Image}} on the {{ContainerInfo}}, instead we just add it as a volume and Thermos uses {{mesos-containerizer launch}} to pivot/chroot into that filesystem after the task is launched. This allows our executor to rely on dependencies from the host filesystem (e.g. Python) without requiring that all filesystem images include those dependencies.

Long story short, I think from Mesos's perspective, just setting {{$HOME}} to be {{$MESOS_SANDBOX}} would be acceptable (but that's based on the assumption that {{$MESOS_SANDBOX}} and {{$MESOS_DIRECTORY}} point to the same path when not running under filesystem isolation). That said, for the case of non-isolated tasks we'd still need to modify {{$HOME}} to point to the task sandbox directory that we created.





was (Author: joshua.cohen):
Yes, this is relatively accurate. Thermos currently has some pretty janky logic to set {{$HOME}}, which amounts to:

* If running a task with an isolated filesystem (i.e. unified containerizer with an {{Image}} set), then use {{/mnt/mesos/sandbox}} (which is bind mounted to the rootfs for the isolated task filesystem).
* Otherwise use {{$MESOS_SANDBOX}}.

I think given the current implementation of filesystem isolation in Thermos, the first bit of logic is pretty specific to your implementation though (we don't set the filesystem {{Image}} on the {{ContainerInfo}}, instead we just add it as a volume and Thermos uses {{mesos-containerizer launch}} to pivot/chroot into that filesystem after the task is launched. This allows our executor to rely on dependencies from the host filesystem (e.g. Python) without requiring that all filesystem images include those dependencies.

Long story short, I think from Mesos's perspective, just setting {{$HOME}} to be {{$MESOS_SANDBOX}} would be acceptable (but that's based on the assumption that {{$MESOS_SANDBOX}} and {{$MESOS_DIRECTORY}} point to the same path when not running under filesystem isolation.




> Set HOME for Mesos tasks
> ------------------------
>
>                 Key: MESOS-6340
>                 URL: https://issues.apache.org/jira/browse/MESOS-6340
>             Project: Mesos
>          Issue Type: Bug
>          Components: containerization, slave
>            Reporter: Cody Maloney
>
> Quite a few programs assume {{$HOME}} points to a user-editable data file directory.
> One example is PYTHON, which tries to look up $HOME to find user-installed pacakges, and if that fails it tries to look up the user in the passwd database which often goes badly (The container is running under the `nobody` user):
> {code}
>     if i == 1:
>         if 'HOME' not in os.environ:
>             import pwd
>             userhome = pwd.getpwuid(os.getuid()).pw_dir
>         else:
>             userhome = os.environ['HOME']
> {code}
> Just setting HOME by default to WORK_DIR would enable more software to work correctly out of the box. Software which needs to specialize / change it (or schedulers with specific preferences), should still be able to set it arbitrarily and anything a scheduler explicitly sets should overwrite the default value of $WORK_DIR



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)