[jira] [Commented] (YARN-8638) Allow linux container runtimes to be pluggable

["Allen Wittenauer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574191#comment-16574191 ] 

Allen Wittenauer commented on YARN-8638:
----------------------------------------

This sort of exemplifies the misuse of the word 'linux' when 'unix' or 'posix' would have been better.  With the exception of cgroups (which is relatively new), there is very little that doesn't work on a variety of platforms.

With this proposed change, it might be time to rename the class entirely and just add 'linux' as an alias. This would open the door up specifically for things like FreeBSD jails, Solaris Zones, quite a few other sandboxing technologies, and more. 

> Allow linux container runtimes to be pluggable
> ----------------------------------------------
>
>                 Key: YARN-8638
>                 URL: https://issues.apache.org/jira/browse/YARN-8638
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 3.2.0
>            Reporter: Craig Condit
>            Priority: Minor
>
> YARN currently supports three different Linux container runtimes (default, docker, and javasandbox). However, it would be relatively straightforward to support arbitrary runtime implementations. This would enable easier experimentation with new and emerging runtime technologies (runc, containerd, etc.) without requiring a rebuild and redeployment of Hadoop. 
> This could be accomplished via a simple configuration change:
> {code:xml}
> <property>
>  <name>yarn.nodemanager.runtime.linux.allowed-runtimes</name>
>  <value>default,docker,experimental</value>
> </property>
>  
> <property>
>  <name>yarn.nodemanager.runtime.linux.experimental.class</name>
>  <value>com.somecompany.yarn.runtime.ExperimentalLinuxContainerRuntime</value>
> </property>{code}
>  
> In this example, {{yarn.nodemanager.runtime.linux.allowed-runtimes}} would now allow arbitrary values. Additionally, {{yarn.nodemanager.runtime.linux.\{RUNTIME_KEY}.class}} would indicate the {{LinuxContainerRuntime}} implementation to instantiate. A no-argument constructor should be sufficient, as {{LinuxContainerRuntime}} already provides an {{initialize()}} method.
> {{DockerLinuxContainerRuntime.isDockerContainerRequested(Map<String, String> env)}} and {{JavaSandboxLinuxContainerRuntime.isSandboxContainerRequested()}} could be generalized to {{isRuntimeRequested(Map<String, String> env)}} and added to the {{LinuxContainerRuntime}} interface. This would allow {{DelegatingLinuxContainerRuntime}} to select an appropriate runtime based on whether that runtime claimed ownership of the current container execution.
> For backwards compatibility, the existing values (default,docker,javasandbox) would continue to be supported as-is. Under the current logic, the evaluation order is javasandbox, docker, default (with default being chosen if no other candidates are available). Under the new evaluation logic, pluggable runtimes would be evaluated after docker and before default, in the order in which they are defined in the allowed-runtimes list. This will change no behavior on current clusters (as there would be no pluggable runtimes defined), and preserves behavior with respect to ordering of existing runtimes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org