You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Shane Kumpf (JIRA)" <ji...@apache.org> on 2018/01/31 02:23:00 UTC
[jira] [Comment Edited] (YARN-7516) Security check for trusted
docker image
[ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346144#comment-16346144 ]
Shane Kumpf edited comment on YARN-7516 at 1/31/18 2:22 AM:
------------------------------------------------------------
Thanks for the quick response and feedback, Eric. At this point, I think it's really just how we convey the feature to users. IMO, we need to try to keep this simply and avoid having too many "if you enable this, this gets disabled automatically" settings.
{quote}When using the double list check, how does a user switch the image to run in default-mode, if the image also exists in yarn-mode list?
{quote}
The yarn-mode would take precedence if the same image prefix exists in both configs. But really the user needs to pick one. It shouldn't exist in both places. :) By moving to the image level, they can get as granular as needed to avoid that scenario.
{quote}There are use cases to pass launch command in default mode. I don't think we can limit launch command for default mode for practical purpose of how docker operates, but I like to hear from others as well.
{quote}
With default-mode, all mounts are disabled. This means that the launch script simply can't run. Any "launch_command" set by native services gets injected into a launch script that is localized and run, AFAIK. I know with enabling entry points we fix some of this, but my thought behind default-mode containers is that we don't override anything, keep it simple. If you need YARN to override the entry point, it's a yarn-mode container.
{quote}For user parameter, I prefer to have it configurable via privileged:true flag (i.e. YARN-7446).
{quote}
We have now restricted the container pretty seriously, I can't see how the user in the container could matter, but I'm open to discussion here. I'm in favor of removing --user for default-mode, again, no unnecessary overrides. I'm also still not sure I agree with disabling user with --privileged. I'm not sure the two are mutually exclusive. We can continue the discussion in YARN-7446.
Thanks again.
was (Author: shanekumpf@gmail.com):
Thanks for the quick response and feedback, Eric. At this point, I think it's really just how we convey the feature to users. IMO, we need to try to keep this simply and avoid having too many "if you enable this, this gets disabled automatically" settings.
{quote}When using the double list check, how does a user switch the image to run in default-mode, if the image also exists in yarn-mode list?
{quote}
The yarn-mode would take precedence if the same image prefix exists in both configs. But really the user needs to pick one. It shouldn't exist in both places. :) By moving to the image level, they can get as granular as needed to avoid that scenario.
{quote}There are use cases to pass launch command in default mode. I don't think we can limit launch command for default mode for practical purpose of how docker operates, but I like to hear from others as well.
{quote}
With default-mode, all mounts are disabled. This means that the launch script simply can't run. Any "launch_command" set by native services gets injected into a launch script that is localized and run, AFAIK. I know with enabling entry points we fix some of this, but my thought behind default containers is that we don't override anything, keep it simple. If you need us to override the entry point, it's a yarn-mode container.
{quote}For user parameter, I prefer to have it configurable via privileged:true flag (i.e. YARN-7446).
{quote}
We have now restricted the container pretty seriously, I can't see how the user in the container could matter, but I'm open to discussion here. I'm in favor of removing --user for default-mode, again, no unnecessary overrides. I'm also still not sure I agree with disabling user with --privileged. I'm not sure the two are mutually exclusive. We can continue the discussion in YARN-7446.
Thanks again.
> Security check for trusted docker image
> ---------------------------------------
>
> Key: YARN-7516
> URL: https://issues.apache.org/jira/browse/YARN-7516
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Major
> Attachments: YARN-7516.001.patch, YARN-7516.002.patch, YARN-7516.003.patch, YARN-7516.004.patch, YARN-7516.005.patch, YARN-7516.006.patch, YARN-7516.007.patch, YARN-7516.008.patch, YARN-7516.009.patch, YARN-7516.010.patch, YARN-7516.011.patch, YARN-7516.012.patch, YARN-7516.013.patch, YARN-7516.014.patch, YARN-7516.015.patch
>
>
> Hadoop YARN Services can support using private docker registry image or docker image from docker hub. In current implementation, Hadoop security is enforced through username and group membership, and enforce uid:gid consistency in docker container and distributed file system. There is cloud use case for having ability to run untrusted docker image on the same cluster for testing.
> The basic requirement for untrusted container is to ensure all kernel and root privileges are dropped, and there is no interaction with distributed file system to avoid contamination. We can probably enforce detection of untrusted docker image by checking the following:
> # If docker image is from public docker hub repository, the container is automatically flagged as insecure, and disk volume mount are disabled automatically, and drop all kernel capabilities.
> # If docker image is from private repository in docker hub, and there is a white list to allow the private repository, disk volume mount is allowed, kernel capabilities follows the allowed list.
> # If docker image is from private trusted registry with image name like "private.registry.local:5000/centos", and white list allows this private trusted repository. Disk volume mount is allowed, kernel capabilities follows the allowed list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org