Re: mesos git commit: Added documentation for posix/rlimit isolator.

[Alex R <al...@apache.org>]

Folks,

I saw this commit today and thought I would use this opportunity to remind
everyone the obvious: documentation is usually the first thing a
user—especially a new user—sees when they start evaluating Mesos. As with
everything, first impression is utterly important. Please strive to make
our docs *consistent* and *consumable*. ESLs please seek support of a
native speaker.

Alex.

On 6 December 2016 at 00:05, <ji...@apache.org> wrote:

> Repository: mesos
> Updated Branches:
>   refs/heads/master 814ed3f8e -> 134a6a5ce
>
>
> Added documentation for posix/rlimit isolator.
>
> Review: https://reviews.apache.org/r/53982/
>
>
> Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
> Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/134a6a5c
> Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/134a6a5c
> Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/134a6a5c
>
> Branch: refs/heads/master
> Commit: 134a6a5cea2af260fee870c8aea2667d60077946
> Parents: 814ed3f
> Author: Benjamin Bannier <be...@mesosphere.io>
> Authored: Mon Dec 5 15:04:59 2016 -0800
> Committer: Jie Yu <yu...@gmail.com>
> Committed: Mon Dec 5 15:04:59 2016 -0800
>
> ----------------------------------------------------------------------
>  docs/mesos-containerizer.md |  6 +++
>  docs/posix_rlimits.md       | 88 ++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 94 insertions(+)
> ----------------------------------------------------------------------
>
>
> http://git-wip-us.apache.org/repos/asf/mesos/blob/134a6a5c/
> docs/mesos-containerizer.md
> ----------------------------------------------------------------------
> diff --git a/docs/mesos-containerizer.md b/docs/mesos-containerizer.md
> index 6d13b2d..d0b9d76 100644
> --- a/docs/mesos-containerizer.md
> +++ b/docs/mesos-containerizer.md
> @@ -295,6 +295,12 @@ destroyed.
>
>  This is described in a [separate document](cni.md).
>
> +
>  ### The `linux/capabilities` Isolator
>
>  This is described in a [separate document](linux_capabilities.md).
> +
> +
> +### The `posix/rlimits` Isolator
> +
> +This is described in a [separate document](posix_rlimits.md).
>
> http://git-wip-us.apache.org/repos/asf/mesos/blob/134a6a5c/
> docs/posix_rlimits.md
> ----------------------------------------------------------------------
> diff --git a/docs/posix_rlimits.md b/docs/posix_rlimits.md
> new file mode 100644
> index 0000000..0a0e6a8
> --- /dev/null
> +++ b/docs/posix_rlimits.md
> @@ -0,0 +1,88 @@
> +# POSIX Resource Limits Support in Mesos Containerizer
> +
> +This document describes the `posix/rlimits` isolator. The isolator adds
> support
> +for setting POSIX resource limits (rlimits) for containers launched using
> the
> +[Mesos containerizer](mesos-containerizer.md).
> +
> +POSIX rlimits can be used control the resources a process can consume.
> Resource
> +limits are typically set at boot time and inherited when a child process
> is
> +forked from a parent process; resource limits can also be modified via
> +`setrlimit(2)`.  In many interactive shells, resource limits can be
> inspected or
> +modified with the `ulimit` shell built-in.
> +
> +A POSIX resource limit consist of a _soft_ and a _hard_ limit. The soft
> limit
> +specifies the effective resource limit for the current and forked
> process, while
> +the hard limit gives the value up to which processes may increase their
> +effective limit; increasing the hard limit is a privileged action. It is
> +required that the soft limit is less than or equal to the hard limit.
> +System administrators can use a hard resource limit to define the maximum
> amount
> +of resources that can be consumed by a user; users can employ soft
> resource
> +limits to ensure that one of their tasks only consumes a limited amount
> of the
> +global hard resource limit.
> +
> +This isolator permits setting per-task resource limits. This isolator
> interprets
> +rlimits specified as part of a task's `ContainerInfo` for the Mesos
> +containerizer, e.g.,
> +```{.json}
> +{
> +  "container": {
> +    "type": "MESOS",
> +    "rlimit_info": {
> +      "rlimits": [
> +        {
> +          "type": "RLMT_CORE"
> +        },
> +        {
> +          "type": "RLMT_STACK",
> +          "soft": 8192,
> +          "hard": 32768
> +        }
> +      ]
> +    }
> +  }
> +}
> +```
> +
> +To enable interpretation of rlimits, agents need to
> +be started with `posix/rlimits` in its `--isolation` flag, e.g.,
> +
> +```{.console}
> +mesos-agent --master=<master ip> --ip=<agent ip>
> +  --work_dir=/var/lib/mesos
> +  --isolation=posix/rlimits[,other isolation flags]
> +```
> +
> +To set a hard limit for a task larger than the current value of the hard
> limit,
> +the agent process needs to be under a privileged user (with the
> +`CAP_SYS_RESOURCE` capability), typically `root`.
> +
> +POSIX currently defines a base set of resources, see
> +[the documentation](http://pubs.opengroup.org/onlinepubs/
> 009695399/functions/getrlimit.html);
> +Linux defines additional resource limits, see e.g., the documentation of
> +`setrlimit(2)`.
> +
> +Resource          | Comment
> +----------------- | ------
> +`RLIMIT_CORE`     | _POSIX_: This is the maximum size of a core file, in
> bytes, that may be created by a process.
> +`RLIMIT_CPU`      | _POSIX_: This is the maximum amount of CPU time, in
> seconds, used by a process.
> +`RLIMIT_DATA`     | _POSIX_: This is the maximum size of a process' data
> segment, in bytes.
> +`RLIMIT_FSIZE`    | _POSIX_: This is the maximum size of a file, in
> bytes, that may be created by a process.
> +`RLIMIT_NOFILE`   | _POSIX_: This is a number one greater than the
> maximum value that the system may assign to a newly-created descriptor.
> +`RLIMIT_STACK`    | _POSIX_: This is the maximum size of the initial
> thread's stack, in bytes.
> +`RLIMIT_AS`       | _POSIX_: This is the maximum size of a process' total
> available memory, in bytes.
> +`RLMT_LOCKS`      | _Linux_: (Early Linux 2.4 only) A limit on the
> combined number of `flock(2)` locks and `fcntl(2)` leases that this process
> may establish.
> +`RLMT_MEMLOCK`    | _Linux_: The maximum number of bytes of memory that
> may be locked into RAM.
> +`RLMT_MSGQUEUE`   | _Linux_: Specifies the limit on the number of bytes
> that can be allocated for POSIX message queues for the real user ID of the
> calling process.
> +`RLMT_NICE`       | _Linux_: (Since Linux 2.6.12) Specifies a ceiling to
> which the process's nice value can be raised using `setpriority(2)` or
> `nice(2)`.
> +`RLMT_NPROC`      | _Linux_: The maximum number of processes (or, more
> precisely on Linux, threads) that can be created for the real user ID of
> the calling process.
> +`RLMT_RSS`        | _Linux_: Specifies the limit (in pages) of the
> process's resident set (the number of virtual pages resident in RAM).
> +`RLMT_RTPRIO`     | _Linux_: (Since Linux 2.6.12) Specifies a ceiling on
> the real-time priority that may be set for this process using
> sched_setscheduler(2) and sched_setparam(2).
> +`RLMT_RTTIME`     | _Linux_: (Since Linux 2.6.25) Specifies a limit (in
> microseconds) on the amount of CPU time that a process scheduled under a
> real-time scheduling policy may consume without making a blocking system
> call.
> +`RLMT_SIGPENDING` | _Linux_: (Since Linux 2.6.8) Specifies the limit on
> the number of signals that may be queued for the real user ID of the
> calling process.
> +
> +Mesos maps these resource types onto `RLimit` types, where by convention
> the
> +prefix `RLMT_` is used in place of `RLIMIT_` above. Not all limits types
> are
> +supported on all platforms.
> +
> +We require either both the soft and hard `RLimit` value, or none to be
> +set; the latter case is interpreted as the absence of an explicit limit.
>
>