You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by eb...@apache.org on 2019/03/14 19:40:53 UTC
[hadoop] branch trunk updated: YARN-8376. Separate white list for
docker.trusted.registries and docker.privileged-container.registries.
Contributed by Eric Yang
This is an automated email from the ASF dual-hosted git repository.
ebadger pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/hadoop.git
The following commit(s) were added to refs/heads/trunk by this push:
new 688b177 YARN-8376. Separate white list for docker.trusted.registries and docker.privileged-container.registries. Contributed by Eric Yang
688b177 is described below
commit 688b177fc6402fc9b00b1312bffbe8226c5ea502
Author: Eric Badger <eb...@oath.com>
AuthorDate: Thu Mar 14 19:39:00 2019 +0000
YARN-8376. Separate white list for docker.trusted.registries and docker.privileged-container.registries. Contributed by Eric Yang
---
.../main/native/container-executor/impl/utils/docker-util.c | 11 ++++++++++-
.../native/container-executor/test/utils/test_docker_util.cc | 4 +++-
.../hadoop-yarn-site/src/site/markdown/DockerContainers.md | 11 +++++++++++
3 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c
index 090d2fc..1339c67 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c
@@ -115,7 +115,15 @@ int check_trusted_image(const struct configuration *command_config, const struct
int ret = 0;
int no_registry_prefix_in_image_name = 0;
char *image_name = get_configuration_value("image", DOCKER_COMMAND_FILE_SECTION, command_config);
- char **privileged_registry = get_configuration_values_delimiter("docker.trusted.registries", CONTAINER_EXECUTOR_CFG_DOCKER_SECTION, conf, ",");
+ char *privileged = NULL;
+ char **privileged_registry = NULL;
+ privileged = get_configuration_value("privileged", DOCKER_COMMAND_FILE_SECTION, command_config);
+ if (privileged != NULL && strcasecmp(privileged, "true") == 0 ) {
+ privileged_registry = get_configuration_values_delimiter("docker.privileged-containers.registries", CONTAINER_EXECUTOR_CFG_DOCKER_SECTION, conf, ",");
+ }
+ if (privileged_registry == NULL) {
+ privileged_registry = get_configuration_values_delimiter("docker.trusted.registries", CONTAINER_EXECUTOR_CFG_DOCKER_SECTION, conf, ",");
+ }
char *registry_ptr = NULL;
if (image_name == NULL) {
ret = INVALID_DOCKER_IMAGE_NAME;
@@ -158,6 +166,7 @@ int check_trusted_image(const struct configuration *command_config, const struct
}
free_and_exit:
+ free(privileged);
free(image_name);
free_values(privileged_registry);
return ret;
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_docker_util.cc b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_docker_util.cc
index 3d052e4..d8f609d 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_docker_util.cc
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_docker_util.cc
@@ -782,7 +782,9 @@ namespace ContainerExecutor {
struct configuration container_cfg, cmd_cfg;
struct args buff = ARGS_INITIAL_VALUE;
int ret = 0;
- std::string container_executor_cfg_contents[] = {"[docker]\n docker.privileged-containers.enabled=1\n docker.trusted.registries=hadoop",
+ std::string container_executor_cfg_contents[] = {"[docker]\n docker.privileged-containers.enabled=1\n"
+ " docker.trusted.registries=library\n"
+ " docker.privileged-containers.registries=hadoop",
"[docker]\n docker.privileged-containers.enabled=true\n docker.trusted.registries=hadoop",
"[docker]\n docker.privileged-containers.enabled=True\n docker.trusted.registries=hadoop",
"[docker]\n docker.privileged-containers.enabled=0",
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md
index ffb5ebc..64988c1 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md
@@ -280,6 +280,7 @@ are allowed. It contains the following properties:
| `docker.allowed.volume-drivers` | Comma separated list of volume drivers which are allowed to be used. By default, no volume drivers are allowed. |
| `docker.host-pid-namespace.enabled` | Set to "true" or "false" to enable or disable using the host's PID namespace. Default value is "false". |
| `docker.privileged-containers.enabled` | Set to "true" or "false" to enable or disable launching privileged containers. Default value is "false". |
+| `docker.privileged-containers.registries` | Comma separated list of privileged docker registries for running privileged docker containers. By default, no registries are defined. |
| `docker.trusted.registries` | Comma separated list of trusted docker registries for running trusted privileged docker containers. By default, no registries are defined. |
| `docker.inspect.max.retries` | Integer value to check docker container readiness. Each inspection is set with 3 seconds delay. Default value of 10 will wait 30 seconds for docker container to become ready before marked as container failed. |
| `docker.no-new-privileges.enabled` | Enable/disable the no-new-privileges flag for docker run. Set to "true" to enable, disabled by default. |
@@ -306,6 +307,7 @@ yarn.nodemanager.linux-container-executor.group=yarn
[docker]
module.enabled=true
docker.privileged-containers.enabled=true
+ docker.privileged-containers.registries=local
docker.trusted.registries=centos
docker.allowed.capabilities=SYS_CHROOT,MKNOD,SETFCAP,SETPCAP,FSETID,CHOWN,AUDIT_WRITE,SETGID,NET_RAW,FOWNER,SETUID,DAC_OVERRIDE,KILL,NET_BIND_SERVICE
docker.allowed.networks=bridge,host,none
@@ -650,6 +652,15 @@ When docker images have been certified by developers and testers to be trustwort
docker.trusted.registries=library
```
+Fine grained access control can also be defined using `docker.privileged-containers.registries` to allow only a subset of Docker images to run as privileged containers. If `docker.privileged-containers.registries` is not defined, YARN will fall back to use `docker.trusted.registries` as access control for privileged Docker images. Fine grained access control example:
+
+```
+[docker]
+ docker.privileged-containers.enabled=true
+ docker.privileged-containers.registries=local/centos:latest
+ docker.trusted.registries=library
+```
+
In development environment, local images can be tagged with a repository name prefix to enable trust. The recommendation of choosing a repository name is using a local hostname and port number to prevent accidentially pulling docker images from Docker Hub or use reserved Docker Hub keyword: "local". Docker run will look for docker images on Docker Hub, if the image does not exist locally. Using a local hostname and port in image name can prevent accidental pulling of canonical images [...]
```
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org