[jira] [Commented] (YARN-4266) Allow whitelisted users to disable user re-mapping/squashing when launching docker containers

["Eric Badger (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063931#comment-16063931 ] 

Eric Badger commented on YARN-4266:
-----------------------------------

I'm working on a new patch based on the one from [~tangzhankun] and [~luhuichun], but would like some comments. In the patch I use the {{--user}} option to set the uid:gid of the user in the {{docker run}} command. After massaging the environment variables, installing the necessary packages in the image, and etc. this almost works perfectly. However, MRAppMaster tries to do a user lookup and fails because the user has no name, only a uid. So for jobs that run through the MRAppMaster, they will fail when trying to use a specific uid in the container. This doesn't necessarily apply to other jobs that could be run in the docker container on YARN, so I think that this change is still an improvement over the current implementation for some users. 

But clearly, my goal here is to run through the MRAppMaster with an arbitrary uid:gid pair. To do this, I propose mounting /var/run/nscd so that the docker container can lookup users via the host according to whatever method is defined in nsswitch.conf on the host. glibc will automatically go to the nscd socket to see if there is a service listening. If a service is listening (such as nscd or sssd on the host), then the lookup can leverage them to do the lookup. This gives us the ability to do remote authentication via ldap. The downside, of course, is that we're now bind mounting another directory, and a socket nonetheless. So I'm very interested in comments on this approach.

cc [~shanekumpf@gmail.com], [~templedf], [~vvasudev], [~sidharta-s]

> Allow whitelisted users to disable user re-mapping/squashing when launching docker containers
> ---------------------------------------------------------------------------------------------
>
>                 Key: YARN-4266
>                 URL: https://issues.apache.org/jira/browse/YARN-4266
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Sidharta Seethana
>            Assignee: luhuichun
>         Attachments: YARN-4266.001.patch, YARN-4266.001.patch, YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping.pdf, YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping_v2.pdf, YARN-4266_Allow_whitelisted_users_to_disable_user_re-mapping_v3.pdf, YARN-4266-branch-2.8.001.patch
>
>
> Docker provides a mechanism (the --user switch) that enables us to specify the user the container processes should run as. We use this mechanism today when launching docker containers . In non-secure mode, we run the docker container based on `yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user` and in secure mode, as the submitting user. However, this mechanism breaks down with a large number of 'pre-created' images which don't necessarily have the users available within the image. Examples of such images include shared images that need to be used by multiple users. We need a way in which we can allow a pre-defined set of users to run containers based on existing images, without using the --user switch. There are some implications of disabling this user squashing that we'll need to work through : log aggregation, artifact deletion etc.,



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org