[jira] [Created] (YARN-9014) OCI/squashfs container runtime

["Jason Lowe (JIRA)" <ji...@apache.org>]

Jason Lowe created YARN-9014:
--------------------------------

             Summary: OCI/squashfs container runtime
                 Key: YARN-9014
                 URL: https://issues.apache.org/jira/browse/YARN-9014
             Project: Hadoop YARN
          Issue Type: New Feature
            Reporter: Jason Lowe
            Assignee: Jason Lowe


This JIRA tracks a YARN container runtime that supports running containers in images built by Docker but the runtime does not use Docker directly, and Docker does not have to be installed on the nodes.  The runtime leverages the [OCI runtime standard|https://github.com/opencontainers/runtime-spec] to launch containers, so an OCI-compliant runtime like {{runc}} is required.  {{runc}} has the benefit of not requiring a daemon like {{dockerd}} to be running in order to launch/control containers.

The layers comprising the Docker image are uploaded to HDFS as [squashfs|http://tldp.org/HOWTO/SquashFS-HOWTO/whatis.html] images, enabling the runtime to efficiently download and execute directly on the compressed layers.  This saves image unpack time and space on the local disk.  The image layers, like other entries in the YARN distributed cache, can be spread across the YARN local disks, increasing the available space for storing container images on each node.

A design document will be posted shortly.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org