You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by vi...@apache.org on 2016/04/25 23:36:19 UTC
[38/48] mesos git commit: Added documentation for multiple-disk
support.
Added documentation for multiple-disk support.
Review: https://reviews.apache.org/r/43583
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/efe8a2c6
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/efe8a2c6
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/efe8a2c6
Branch: refs/heads/0.27.x
Commit: efe8a2c6d25be863ae305d155ad6ca935d1a9d42
Parents: 8afbcc3
Author: Joris Van Remoortere <jo...@gmail.com>
Authored: Mon Feb 15 14:40:54 2016 -0500
Committer: Joris Van Remoortere <jo...@gmail.com>
Committed: Tue Feb 16 18:21:31 2016 -0500
----------------------------------------------------------------------
docs/home.md | 1 +
docs/multiple-disk.md | 141 +++++++++++++++++++++++++++++++++++++++++
docs/persistent-volume.md | 3 +
3 files changed, 145 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/efe8a2c6/docs/home.md
----------------------------------------------------------------------
diff --git a/docs/home.md b/docs/home.md
index dea6ec2..2780eea 100644
--- a/docs/home.md
+++ b/docs/home.md
@@ -41,6 +41,7 @@ layout: documentation
* [Networking for Mesos-managed Containers](networking-for-mesos-managed-containers.md)
* [Oversubscription](oversubscription.md) for how to configure Mesos to take advantage of unused resources to launch "best-effort" tasks.
* [Persistent Volume](persistent-volume.md) for how to allow tasks to access persistent storage resources.
+* [Multiple Disks](multiple-disk.md) for how to to allow tasks to use multiple isolated disk resources.
* [Quota](quota.md) for how to configure Mesos to provide guaranteed resource allocations for use by a role.
* [Reservation](reservation.md) for how operators and frameworks can reserve resources on individual agents for use by a role.
http://git-wip-us.apache.org/repos/asf/mesos/blob/efe8a2c6/docs/multiple-disk.md
----------------------------------------------------------------------
diff --git a/docs/multiple-disk.md b/docs/multiple-disk.md
new file mode 100644
index 0000000..4fc5327
--- /dev/null
+++ b/docs/multiple-disk.md
@@ -0,0 +1,141 @@
+---
+layout: documentation
+---
+
+# Multiple Disks
+
+Mesos provides a mechanism for operators to expose multiple disk resources. When
+creating [persistent volumes](persistent-volume.md) frameworks can decide
+whether to use specific disks by examining the `source` field on the disk
+resources offered.
+
+`Disk` resources come in three forms:
+
+* A `Root` disk is presented by not having the `source` set in `DiskInfo`.
+* A `Path` disk is presented by having the `PATH` enum set for `source` in
+ `DiskInfo`. It also has a `root` which the operator uses to specify the
+ directory to be used to store data.
+* A `Mount` disk is presented by having the `MOUNT` enum set for `source` in
+ `DiskInfo`. It also has a `root` which the operator uses to specify the
+ mount point used to store data.
+
+Operators can use the JSON-formated `--resources` option on the agent to provide
+these different kind of disk resources on agent start-up.
+
+#### `Root` disk
+
+A `Root` disk is the basic disk resource in Mesos. It usually maps to the
+storage on the main operating system drive that the operator has presented to
+the agent. Data is mapped into the `work_dir` of the agent.
+
+```
+{
+ "resources" : [
+ {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 2048 },
+ "role" : <framework_role>
+ }
+ ]
+}
+```
+
+#### `Path` disks
+
+A `Path` disk is an auxiliary disk resource provided by the operator. This can
+can be carved up into smaller chunks by accepting less than the total amount as
+frameworks see fit. Common uses for this kind of disk are extra logging space,
+file archives or caches, or other non performance-critical applications.
+Operators can present extra disks on their agents as `Path` disks simply by
+creating a directory and making that the `root` of the `Path` in `DiskInfo`'s
+`source`.
+
+`Path` disks are also useful for mocking up a multiple-disk environment by
+creating some directories on the operating system drive. This should only be
+done in a testing or staging environment.
+
+```
+{
+ "resources" : [
+ {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 2048 },
+ "role" : <framework_role>,
+ "disk" : {
+ "source" : {
+ "type" : "PATH",
+ "path" : { "root" : "/mnt/data" }
+ }
+ }
+ }
+ ]
+}
+```
+
+#### `Mount` disks
+
+A `Mount` disk is an auxiliary disk resource provided by the operator. This
+__cannot__ be carved up into smaller chunks by frameworks. This lack of
+flexibility allows operators to provide assurances to frameworks that they will
+have exclusive access to the disk device. Common uses for this kind of disk
+include database storage, write-ahead logs, or other performance-critical
+applications.
+
+Another requirement of `Mount` disks is that (on Linux) they map to a `mount`
+point in the `/proc/mounts` table. Operators should mount a physical disk with
+their preferred file system and provide the mount point as the `root` of the
+`Mount` in `DiskInfo`'s `source`.
+
+Aside from the performance advantages of `Mount` disks, applications running on
+them should be able to rely on disk errors when they attempt to exceed the
+capacity of the volume. This holds true as long as the file system in use
+correctly propagates these errors. Due to this expectation, the `posix/disk`
+quota enforcement is disabled for `Mount` disks.
+
+```
+{
+ "resources" : [
+ {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 2048 },
+ "role" : <framework_role>,
+ "disk" : {
+ "source" : {
+ "type" : "MOUNT",
+ "mount" : { "root" : "/mnt/data" }
+ }
+ }
+ }
+ ]
+}
+```
+
+#### `Block` disks
+
+Mesos currently does not allow operators to expose raw block devices. It may do
+so in the future, but there are security and flexibility concerns that need to
+be addressed in a design document first.
+
+### Storage Management
+
+Mesos currently does not clean up or destroy data when persistent volumes are
+destroyed. It may do so in the future; however, the expectation is currently
+upon the framework, executor, and application to delete their data before
+destroying their persistent volumes. This is strongly encouraged for both
+security and ensuring that future users of the underlying disk resource are not
+penalized for prior consumption of the disk capacity.
+
+### Implementation
+
+A `Path` disk will have sub-directories created within the `root` which will be
+used to differentiate the different volumes that are created on it.
+
+A `Mount` disk will __not__ have sub-directories created, allowing applications
+to use the full file system mounted on the device. This provides operators a
+construct through which to enable data ingestion.
+
+Operators should be aware of these distinctions when inspecting or cleaning up
+remnant data.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/mesos/blob/efe8a2c6/docs/persistent-volume.md
----------------------------------------------------------------------
diff --git a/docs/persistent-volume.md b/docs/persistent-volume.md
index 3a3e370..2c6308f 100644
--- a/docs/persistent-volume.md
+++ b/docs/persistent-volume.md
@@ -25,6 +25,9 @@ cluster.
Please refer to the [Reservation](reservation.md) documentation for details
regarding reservation mechanisms available in Mesos.
+Persistent volumes can also be created on isolated and auxiliary disks by
+reserving [Multiple Disk resources](multiple-disk.md).
+
Persistent volumes can be created by __operators__ and authorized
__frameworks__. We require a `principal` from the operator or framework in order
to authenticate/authorize the operations. Permissions are specified via the