[jira] [Created] (YARN-9669) YARN Container phase 3

["Eric Yang (JIRA)" <ji...@apache.org>]

Eric Yang created YARN-9669:
-------------------------------

             Summary: YARN Container phase 3
                 Key: YARN-9669
                 URL: https://issues.apache.org/jira/browse/YARN-9669
             Project: Hadoop YARN
          Issue Type: Improvement
            Reporter: Eric Yang


Docker container can run on Hadoop cluster fine today in Hadoop 3.1.1+ release and trunk code base.  We are getting feedback from potential customers for advance features that may take Docker support to the next level.  The frequently requested features are:

h3.  Improve cgroups support
h4. Support systemd based cgorup
YARN-9660 provides instruction to setup cgroups fs to control Docker container with cgroup.  The next step is to support Docker cgroups with systemd natively.  This will reduce some configuration steps on Docker side to make the user experience more seamless.

h4. Disk throttling
Docker supports IO throttling with several docker parameters:
{code}--device-read-bps=""
--device-write-bps=""
--device-read-iops=""
--device-write-iops=""{code}

This looks like a good feature for YARN to specify IO throughput for stress test simulation.

h3. Docker volume support

Docker can create data container of fixed size.  This is useful to limit disk usage for containers.  This feature provides resource management to data, and ability to create docker volume and anchor docker containers to colocate with data container.

h3. Protection against overlay network spike

Badly designed docker container can cause system to become non-responsive, if network spikes occurs or forking uncontrollably.  Some protection maybe useful to guard against network flood and fork bomb.  Active prevention can shutdown the crashing docker container before system hangs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org