You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by su...@apache.org on 2018/08/08 20:11:56 UTC
[15/50] [abbrv] hadoop git commit: YARN-8287. Update documentation
and yarn-default related to the Docker runtime. Contributed by Craig Condit
YARN-8287. Update documentation and yarn-default related to the Docker runtime. Contributed by Craig Condit
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/bcfc9851
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/bcfc9851
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/bcfc9851
Branch: refs/heads/HDFS-12943
Commit: bcfc9851f4c6d37d9a119c16ea7d4d253d3b9614
Parents: 48673bc
Author: Shane Kumpf <sk...@apache.org>
Authored: Fri Aug 3 18:20:49 2018 -0600
Committer: Shane Kumpf <sk...@apache.org>
Committed: Fri Aug 3 18:20:49 2018 -0600
----------------------------------------------------------------------
.../src/main/resources/yarn-default.xml | 13 ++--
.../src/site/markdown/DockerContainers.md | 66 +++++++++++++++++---
2 files changed, 64 insertions(+), 15 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hadoop/blob/bcfc9851/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
index 2cc842f..72e42d8 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
@@ -1729,16 +1729,19 @@
<description>This configuration setting determines the capabilities
assigned to docker containers when they are launched. While these may not
be case-sensitive from a docker perspective, it is best to keep these
- uppercase. To run without any capabilites, set this value to
+ uppercase. To run without any capabilities, set this value to
"none" or "NONE"</description>
<name>yarn.nodemanager.runtime.linux.docker.capabilities</name>
<value>CHOWN,DAC_OVERRIDE,FSETID,FOWNER,MKNOD,NET_RAW,SETGID,SETUID,SETFCAP,SETPCAP,NET_BIND_SERVICE,SYS_CHROOT,KILL,AUDIT_WRITE</value>
</property>
<property>
- <description>This configuration setting determines if
- privileged docker containers are allowed on this cluster.
- Use with extreme care.</description>
+ <description>This configuration setting determines if privileged docker
+ containers are allowed on this cluster. Privileged containers are granted
+ the complete set of capabilities and are not subject to the limitations
+ imposed by the device cgroup controller. In other words, privileged
+ containers can do almost everything that the host can do. Use with
+ extreme care.</description>
<name>yarn.nodemanager.runtime.linux.docker.privileged-containers.allowed</name>
<value>false</value>
</property>
@@ -2019,8 +2022,6 @@
<value>false</value>
</property>
- <!--Docker configuration-->
-
<property>
<description>
Adjustment to the container OS scheduling priority. In Linux, passed
http://git-wip-us.apache.org/repos/asf/hadoop/blob/bcfc9851/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md
index e35c906..0001489 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md
@@ -19,10 +19,8 @@ Launching Applications Using Docker Containers
Security Warning
---------------
-**IMPORTANT** This feature is experimental and is not complete. **IMPORTANT**
-Enabling this feature and running Docker containers in your cluster has security
-implications. With this feature enabled, it may be possible to gain root access
-to the YARN NodeManager hosts. Given Docker's integration with many powerful
+**IMPORTANT** Enabling this feature and running Docker containers in your
+cluster has security implications. Given Docker's integration with many powerful
kernel features, it is imperative that administrators understand
[Docker security](https://docs.docker.com/engine/security/security/) before
enabling this feature.
@@ -56,10 +54,6 @@ Docker for YARN provides both consistency (all YARN containers will have the
same software environment) and isolation (no interference with whatever is
installed on the physical machine).
-Docker support in the LCE is still evolving. To track progress, follow
-[YARN-3611](https://issues.apache.org/jira/browse/YARN-3611), the umbrella JIRA
-for Docker support improvements.
-
Cluster Configuration
---------------------
@@ -153,7 +147,30 @@ The following properties should be set in yarn-site.xml:
<value>false</value>
<description>
Optional. Whether applications are allowed to run in privileged
- containers.
+ containers. Privileged containers are granted the complete set of
+ capabilities and are not subject to the limitations imposed by the device
+ cgroup controller. In other words, privileged containers can do almost
+ everything that the host can do. Use with extreme care.
+ </description>
+ </property>
+
+ <property>
+ <name>yarn.nodemanager.runtime.linux.docker.delayed-removal.allowed</name>
+ <value>false</value>
+ <description>
+ Optional. Whether or not users are allowed to request that Docker
+ containers honor the debug deletion delay. This is useful for
+ troubleshooting Docker container related launch failures.
+ </description>
+ </property>
+
+ <property>
+ <name>yarn.nodemanager.runtime.linux.docker.stop.grace-period</name>
+ <value>10</value>
+ <description>
+ Optional. A configurable value to pass to the Docker Stop command. This
+ value defines the number of seconds between the docker stop command sending
+ a SIGTERM and a SIGKILL.
</description>
</property>
@@ -177,6 +194,36 @@ The following properties should be set in yarn-site.xml:
"none" or "NONE"
</description>
</property>
+
+ <property>
+ <name>yarn.nodemanager.runtime.linux.docker.enable-userremapping.allowed</name>
+ <value>true</value>
+ <description>
+ Optional. Whether docker containers are run with the UID and GID of the
+ calling user.
+ </description>
+ </property>
+
+ <property>
+ <name>yarn.nodemanager.runtime.linux.docker.userremapping-uid-threshold</name>
+ <value>1</value>
+ <description>
+ Optional. The minimum acceptable UID for a remapped user. Users with UIDs
+ lower than this value will not be allowed to launch containers when user
+ remapping is enabled.
+ </description>
+ </property>
+
+ <property>
+ <name>yarn.nodemanager.runtime.linux.docker.userremapping-gid-threshold</name>
+ <value>1</value>
+ <description>
+ Optional. The minimum acceptable GID for a remapped user. Users belonging
+ to any group with a GID lower than this value will not be allowed to
+ launch containers when user remapping is enabled.
+ </description>
+ </property>
+
</configuration>
```
@@ -204,6 +251,7 @@ are allowed. It contains the following properties:
| `docker.allowed.networks` | Comma separated networks that containers are allowed to use. If no network is specified when launching the container, the default Docker network will be used. |
| `docker.allowed.ro-mounts` | Comma separated directories that containers are allowed to mount in read-only mode. By default, no directories are allowed to mounted. |
| `docker.allowed.rw-mounts` | Comma separated directories that containers are allowed to mount in read-write mode. By default, no directories are allowed to mounted. |
+| `docker.allowed.volume-drivers` | Comma separated list of volume drivers which are allowed to be used. By default, no volume drivers are allowed. |
| `docker.host-pid-namespace.enabled` | Set to "true" or "false" to enable or disable using the host's PID namespace. Default value is "false". |
| `docker.privileged-containers.enabled` | Set to "true" or "false" to enable or disable launching privileged containers. Default value is "false". |
| `docker.trusted.registries` | Comma separated list of trusted docker registries for running trusted privileged docker containers. By default, no registries are defined. |
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org