You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by zh...@apache.org on 2018/05/04 23:32:08 UTC
mesos git commit: Added documentation on volume resize support.
Repository: mesos
Updated Branches:
refs/heads/master a779b2787 -> 110a176cc
Added documentation on volume resize support.
Review: https://reviews.apache.org/r/66923/
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/110a176c
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/110a176c
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/110a176c
Branch: refs/heads/master
Commit: 110a176cc8a8494ff235cbffe21e255a06ddfc64
Parents: a779b27
Author: Zhitao Li <zh...@gmail.com>
Authored: Fri May 4 14:05:39 2018 -0700
Committer: Zhitao Li <zh...@gmail.com>
Committed: Fri May 4 16:27:02 2018 -0700
----------------------------------------------------------------------
docs/authorization.md | 8 ++
docs/operator-http-api.md | 124 ++++++++++++++++--
docs/persistent-volume.md | 278 +++++++++++++++++++++++++++++++++++++++--
3 files changed, 392 insertions(+), 18 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/110a176c/docs/authorization.md
----------------------------------------------------------------------
diff --git a/docs/authorization.md b/docs/authorization.md
index fdbef77..cd8622b 100644
--- a/docs/authorization.md
+++ b/docs/authorization.md
@@ -209,6 +209,14 @@ entries, each representing an authorizable action:
</td>
</tr>
<tr>
+ <td><code>resize_volume</code></td>
+ <td>Framework principal or Operator username.</td>
+ <td>Resource role of the volume.</td>
+ <td>Growing or shrinking
+ <a href="persistent-volume.md">persistent volumes</a>.
+ </td>
+</tr>
+<tr>
<td><code>get_quotas</code></td>
<td>Operator username.</td>
<td>Resource role whose quota status will be queried.</td>
http://git-wip-us.apache.org/repos/asf/mesos/blob/110a176c/docs/operator-http-api.md
----------------------------------------------------------------------
diff --git a/docs/operator-http-api.md b/docs/operator-http-api.md
index 9be1e2d..c0f20c0 100644
--- a/docs/operator-http-api.md
+++ b/docs/operator-http-api.md
@@ -1883,11 +1883,11 @@ HTTP/1.1 202 Accepted
### CREATE_VOLUMES
-This call create persistent volumes on reserved resources. The request is
-forwarded asynchronously to the Mesos agent where the reserved resources are
-located. That asynchronous message may not be delivered or creating the volumes
-at the agent might fail. This call takes `agent_id` and `volumes` details like
-the following.
+This call create [persistent volumes](persistent-volume.md) on reserved
+resources. The request is forwarded asynchronously to the Mesos agent where the
+reserved resources are located. That asynchronous message may not be delivered
+or creating the volumes at the agent might fail. This call takes `agent_id`
+and `volumes` details like the following.
```
CREATE_VOLUMES HTTP Request (JSON):
@@ -1936,8 +1936,8 @@ HTTP/1.1 202 Accepted
### DESTROY_VOLUMES
-This call destroys persistent volumes. The request is forwarded asynchronously to the
-Mesos agent where the reserved resources are located.
+This call destroys [persistent volumes](persistent-volume.md). The request is
+forwarded asynchronously to the Mesos agent where the volumes are located.
```
DESTROY_VOLUMES HTTP Request (JSON):
@@ -1984,6 +1984,116 @@ HTTP/1.1 202 Accepted
```
+### GROW_VOLUME
+
+This call grows the size of a [persistent volume](persistent-volume.md). The
+request is forwarded asynchronously to the Mesos agent where the volume is
+located.
+
+```
+GROW_VOLUME HTTP Request (JSON):
+
+POST /api/v1 HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+Accept: application/json
+
+{
+ "type": "GROW_VOLUME",
+ "grow_volume": {
+ "agent_id": {
+ "value": "919141a8-b434-4946-86b9-e1b65c8171f6-S0"
+ },
+ "volume": {
+ "disk": {
+ "persistence": {
+ "id": "id1",
+ "principal": "my-principal"
+ },
+ "volume": {
+ "container_path": "path1",
+ "mode": "RW"
+ }
+ },
+ "name": "disk",
+ "role": "role1",
+ "scalar": {
+ "value": 64.0
+ },
+ "type": "SCALAR"
+ },
+ "addition": {
+ "name": "disk",
+ "role": "role1",
+ "scalar": {
+ "value": 64.0
+ },
+ "type": "SCALAR"
+ }
+ }
+}
+
+
+GROW_VOLUME HTTP Response:
+
+HTTP/1.1 202 Accepted
+
+```
+
+### SHRINK_VOLUME
+
+This call shrinks the size of a [persistent volume](persistent-volume.md).
+The request is forwarded asynchronously to the Mesos agent where the volume
+is located.
+
+```
+SHRINK_VOLUME HTTP Request (JSON):
+
+POST /api/v1 HTTP/1.1
+
+Host: masterhost:5050
+Content-Type: application/json
+Accept: application/json
+
+{
+ "type": "SHRINK_VOLUME",
+ "shrink_volume": {
+ "agent_id": {
+ "value": "919141a8-b434-4946-86b9-e1b65c8171f6-S0"
+ },
+ "volume": {
+ "disk": {
+ "persistence": {
+ "id": "id1",
+ "principal": "my-principal"
+ },
+ "volume": {
+ "container_path": "path1",
+ "mode": "RW"
+ }
+ },
+ "name": "disk",
+ "role": "role1",
+ "scalar": {
+ "value": 128.0
+ },
+ "type": "SCALAR"
+ },
+ "subtract": {
+ "value": 64.0
+ }
+ }
+}
+
+
+SHRINK_VOLUME HTTP Response:
+
+HTTP/1.1 202 Accepted
+
+```
+
+
### GET_MAINTENANCE_STATUS
This call retrieves the cluster's maintenance status.
http://git-wip-us.apache.org/repos/asf/mesos/blob/110a176c/docs/persistent-volume.md
----------------------------------------------------------------------
diff --git a/docs/persistent-volume.md b/docs/persistent-volume.md
index 1a5799b..cf3e575 100644
--- a/docs/persistent-volume.md
+++ b/docs/persistent-volume.md
@@ -53,9 +53,15 @@ create, and destroy operations, the Mesos master must be configured with the
appropriate ACLs. For more information, see the
[authorization documentation](authorization.md).
-* `Offer::Operation::Create` and `Offer::Operation::Destroy` messages are
- available for __frameworks__ to send back via the `acceptOffers` API as a
- response to a resource offer.
+* The following messages are available for __frameworks__ to send back via the
+ `acceptOffers` API as a response to a resource offer:
+ * `Offer::Operation::Create`
+ * `Offer::Operation::Destroy`
+ * `Offer::Operation::GrowVolume`
+ * `Offer::Operation::ShrinkVolume`
+* For each message in above list, a corresponding call in
+ [HTTP Operator API](operator-http-api.md) is available for operators or
+ administrative tools;
* `/create-volumes` and `/destroy-volumes` HTTP endpoints allow
__operators__ to manage persistent volumes through the master.
@@ -201,9 +207,9 @@ created. First, we receive a resource offer (copy/pasted from above):
]
}
-We can destroy the persistent volume by sending a `Offer::Operation` message via
-the `acceptOffers` API. `Offer::Operation::Destroy` has a `volumes` field which
-specifies the persistent volumes to be destroyed.
+We can destroy the persistent volume by sending an `Offer::Operation` message
+via the `acceptOffers` API. `Offer::Operation::Destroy` has a `volumes` field
+which specifies the persistent volumes to be destroyed.
{
"type" : Offer::Operation::DESTROY,
@@ -257,14 +263,263 @@ contain the following reserved disk resources:
Those reserved resources can then be used as normal: e.g., they can be used to
create another persistent volume or can be unreserved.
-## Operator HTTP Endpoints
+<a name="offer-operation-grow-volume"></a>
+### `Offer::Operation::GrowVolume`
+
+Sometimes, a framework or an operator may find that the size of an existing
+persistent volume may be too small (possibly due to increased usage). In
+[Offer::Operation::Create](#offer-operation-create), we created a persistent
+volume from 2048 MB of disk resources. Suppose we want to grow the size of
+the volume to 4096 MB, we first need resource offer(s) with at least 2048 MB of
+disk resources with the same reservation and disk information:
+
+ {
+ "id" : <offer_id>,
+ "framework_id" : <framework_id>,
+ "slave_id" : <slave_id>,
+ "hostname" : <hostname>,
+ "resources" : [
+ {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 2048 },
+ "role" : <offer's allocation role>,
+ "reservation" : {
+ "principal" : <framework_principal>
+ }
+ },
+ {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 2048 },
+ "role" : <offer's allocation role>,
+ "reservation" : {
+ "principal" : <framework_principal>
+ },
+ "disk": {
+ "persistence": {
+ "id" : <persistent_volume_id>
+ },
+ "volume" : {
+ "container_path" : <container_path>,
+ "mode" : <mode>
+ }
+ }
+ }
+ ]
+ }
+
+We can grow the persistent volume by sending an `Offer::Operation` message.
+`Offer::Operation::GrowVolume` has a `volume` field which specifies the
+persistent volume to grow, and an `addition` field which specifies he
+additional disk space resource.
+
+ {
+ "type" : Offer::Operation::GROW_VOLUME,
+ "grow_volume" : {
+ "volume" : {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 2048 },
+ "role" : <offer's allocation role>,
+ "reservation" : {
+ "principal" : <framework_principal>
+ },
+ "disk": {
+ "persistence": {
+ "id" : <persistent_volume_id>
+ },
+ "volume" : {
+ "container_path" : <container_path>,
+ "mode" : <mode>
+ }
+ }
+ },
+ "addition" : {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 2048 },
+ "role" : <offer's allocation role>,
+ "reservation" : {
+ "principal" : <framework_principal>
+ }
+ }
+ }
+ }
+
+If this request succeeds, the persistent volume will be grown to the new size,
+and all files and directories associated with the volume will not be touched.
+A subsequent resource offer will contain the grown volume:
+
+ {
+ "id" : <offer_id>,
+ "framework_id" : <framework_id>,
+ "slave_id" : <slave_id>,
+ "hostname" : <hostname>,
+ "resources" : [
+ {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 4096 },
+ "role" : <offer's allocation role>,
+ "reservation" : {
+ "principal" : <framework_principal>
+ },
+ "disk": {
+ "persistence": {
+ "id" : <persistent_volume_id>
+ },
+ "volume" : {
+ "container_path" : <container_path>,
+ "mode" : <mode>
+ }
+ }
+ }
+ ]
+ }
+
+<a name="offer-operation-shrink-volume"></a>
+### `Offer::Operation::ShrinkVolume`
+
+Similarly, a framework or an operator may find that the size of an existing
+persistent volume may be too large (possibly due to over provisioning), and want
+to free up unneeded disk space resources.
+In [Offer::Operation::Create](#offer-operation-create), we created a persistent
+volume from 2048 MB of disk resources. Suppose we want to shrink the size of
+the volume to 1024 MB, we first need a resource offer with the volume to shrink:
+
+ {
+ "id" : <offer_id>,
+ "framework_id" : <framework_id>,
+ "slave_id" : <slave_id>,
+ "hostname" : <hostname>,
+ "resources" : [
+ {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 2048 },
+ "role" : <offer's allocation role>,
+ "reservation" : {
+ "principal" : <framework_principal>
+ },
+ "disk": {
+ "persistence": {
+ "id" : <persistent_volume_id>
+ },
+ "volume" : {
+ "container_path" : <container_path>,
+ "mode" : <mode>
+ }
+ }
+ }
+ ]
+ }
+
+We can shrink the persistent volume by sending an `Offer::Operation` message via
+the `acceptOffers` API. `Offer::Operation::ShrinkVolume` has a `volume` field
+which specifies the persistent volume to grow, and a `subtract` field which
+specifies the scalar value of disk space to subtract from the volume:
+
+ {
+ "type" : Offer::Operation::SHRINK_VOLUME,
+ "shrink_volume" : {
+ "volume" : {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 2048 },
+ "role" : <offer's allocation role>,
+ "reservation" : {
+ "principal" : <framework_principal>
+ },
+ "disk": {
+ "persistence": {
+ "id" : <persistent_volume_id>
+ },
+ "volume" : {
+ "container_path" : <container_path>,
+ "mode" : <mode>
+ }
+ }
+ },
+ "subtract" : {
+ "value" : 1024
+ }
+ }
+ }
+
+If this request succeeds, the persistent volume will be shrunk to the new size,
+and all files and directories associated with the volume will not be touched.
+A subsequent resource offer will contain the shrunk volume as well as freed up
+disk resources with the same reservation information:
+
+ {
+ "id" : <offer_id>,
+ "framework_id" : <framework_id>,
+ "slave_id" : <slave_id>,
+ "hostname" : <hostname>,
+ "resources" : [
+ {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 1024 },
+ "role" : <offer's allocation role>,
+ "reservation" : {
+ "principal" : <framework_principal>
+ }
+ },
+ {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 1024 },
+ "role" : <offer's allocation role>,
+ "reservation" : {
+ "principal" : <framework_principal>
+ },
+ "disk": {
+ "persistence": {
+ "id" : <persistent_volume_id>
+ },
+ "volume" : {
+ "container_path" : <container_path>,
+ "mode" : <mode>
+ }
+ }
+ }
+ ]
+ }
+
+
+Some restrictions about resizing a volume (applicable to both
+[Offer::Operation::GrowVolume](#offer-operation-grow-volume) and
+[Offer::Operation::ShrinkVolume](#offer-operation-shrink-volume)):
+* Only persistent volumes created on an agent's local disk space with `ROOT` or
+ `PATH` type can be resized;
+* A persistent volume cannot be actively used by a task when being resized;
+* A persistent volume cannot be shared when being resized;
+* Volume resize operations cannot be included in an ACCEPT call with other
+ operations which make use of the resized volume.
+
+
+## Versioned HTTP Operator API
As described above, persistent volumes can be created by a framework scheduler
-as part of the resource offer cycle. Persistent volumes can also be created and
-destroyed using the [/create-volumes](endpoints/master/create-volumes.md) and
-[/destroy-volumes](endpoints/master/destroy-volumes.md) endpoints, respectively.
+as part of the resource offer cycle. Persistent volumes can also be managed
+using the [HTTP Operator API](operator-http-api.md).
+
This capability is intended for use by operators and administrative tools.
+For each offer operation which interacts with persistent volume, there is an
+equivalent call in master's [HTTP Operator API](operator-http-api.md).
+
+## Unversioned Operator HTTP Endpoints
+
+Several HTTP endpoints like
+[/create-volumes](endpoints/master/create-volumes.md) and
+[/destroy-volumes](endpoints/master/destroy-volumes.md) can still be used to
+manage persisent volumes, but we generally encourage operators to use
+versioned [HTTP Operator API](operator-http-api.md) instead, as new features
+like resize support may not be backported.
+
### `/create-volumes`
To use this endpoint, the operator should first ensure that a reservation for
@@ -504,4 +759,5 @@ master endpoint to include detailed information about persistent volumes and
dynamic reservations. Mesos 1.0 changed the semantics of destroying a volume:
in previous releases, destroying a volume would remove the Mesos-level metadata
but would not remove the volume's data from the agent's filesystem. Mesos 1.1
-introduced support for [shared persistent volumes](shared-resources.md).
+introduced support for [shared persistent volumes](shared-resources.md). Mesos
+1.6 introduced experimental support for resizing persistent volumes.