You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by bm...@apache.org on 2017/04/01 00:14:01 UTC
[1/2] mesos git commit: Documentation updates to reflect multi-role
framework support.
Repository: mesos
Updated Branches:
refs/heads/master 38736e85b -> 0d4e08ce3
Documentation updates to reflect multi-role framework support.
This also updates the roles documentation to reflect the vision
of roles representing the resource consumer.
Review: https://reviews.apache.org/r/58055
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/b1eb5711
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/b1eb5711
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/b1eb5711
Branch: refs/heads/master
Commit: b1eb5711cb9a3ac4fc53b0c49680565587aea993
Parents: 38736e8
Author: Benjamin Mahler <bm...@apache.org>
Authored: Wed Mar 29 18:05:59 2017 -0700
Committer: Benjamin Mahler <bm...@apache.org>
Committed: Fri Mar 31 17:11:26 2017 -0700
----------------------------------------------------------------------
docs/authentication.md | 4 +-
docs/authorization.md | 41 ++++++------
docs/persistent-volume.md | 41 ++++++------
docs/quota.md | 42 ++++++------
docs/reservation.md | 75 ++++++++++++---------
docs/roles.md | 140 +++++++++++++++++++++++-----------------
docs/scheduler-http-api.md | 63 ++++++++++--------
docs/shared-resources.md | 19 +++---
8 files changed, 239 insertions(+), 186 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/b1eb5711/docs/authentication.md
----------------------------------------------------------------------
diff --git a/docs/authentication.md b/docs/authentication.md
index 1574db9..279e00a 100644
--- a/docs/authentication.md
+++ b/docs/authentication.md
@@ -33,8 +33,8 @@ passwords.
Principals are used primarily for authentication and
[authorization](authorization.md); note that a principal is different from a
framework's *user*, which is the operating system account used by the agent to
-run executors, and a framework's *[role](roles.md)*, which is used to determine
-which resources a framework can use.
+run executors, and the framework's *[roles](roles.md)*, which are used to
+determine which resources a framework can use.
## Configuration
http://git-wip-us.apache.org/repos/asf/mesos/blob/b1eb5711/docs/authorization.md
----------------------------------------------------------------------
diff --git a/docs/authorization.md b/docs/authorization.md
index b019120..913765f 100644
--- a/docs/authorization.md
+++ b/docs/authorization.md
@@ -8,8 +8,8 @@ layout: documentation
In Mesos, the authorization subsystem allows the operator to configure the
actions that certain principals are allowed to perform. For example, the
operator can use authorization to ensure that principal `foo` can only register
-frameworks in role `bar`, and no other principals can register frameworks in
-any role.
+frameworks subscribed to role `bar`, and no other principals can register
+frameworks subscribed to any roles.
A reference implementation _local authorizer_ provides basic security for most
use cases. This authorizer is configured using Access Control Lists (ACLs).
@@ -50,19 +50,19 @@ launches a Mesos framework and then attempts to destroy a persistent volume:
be `payroll-framework`; this principal represents the trusted identity of the
framework.
* The framework now sends a registration message to the master. This message
- includes a `FrameworkInfo` object containing a `principal` and a `role`; in
- this case, it will use the role `accounting`. The principal in this message
- must be `payroll-framework`, to match the one used by the framework for
- authentication.
+ includes a `FrameworkInfo` object containing a `principal` and `roles`; in
+ this case, it will use a single role named `accounting`. The principal in
+ this message must be `payroll-framework`, to match the one used by the
+ framework for authentication.
* The master consults the local authorizer, which in turn looks through its ACLs
to see if it has a `RegisterFramework` ACL which authorizes the principal
`payroll-framework` to register with the `accounting` role. It does find such
- an ACL, the framework registers successfully. Now that the framework belongs
- to the `accounting` role, any [weights](roles.md),
+ an ACL, the framework registers successfully. Now that the framework is
+ subscribed to the `accounting` role, any [weights](weights.md),
[reservations](reservation.md), [persistent volumes](persistent-volume.md),
or [quota](quota.md) associated with the accounting department's role will
- apply. This allows operators to control the resource consumption of this
- department.
+ apply when allocating resources to this role within the framework. This
+ allows operators to control the resource consumption of this department.
* Suppose the framework has created a persistent volume on an agent which it
now wishes to destroy. The framework sends an `ACCEPT` call containing an
offer operation which will `DESTROY` the persistent volume.
@@ -269,9 +269,9 @@ The `get_endpoints` action covers:
### Examples
Consider for example the following ACL: Only principal `foo` can register
-frameworks within the `analytics` role. All principals can register to any
-other role (including the principal `foo` since permissive is the default
-behavior).
+frameworks subscribed to the `analytics` role. All principals can register
+frameworks subscribing to any other roles (including the principal `foo`
+since permissive is the default behavior).
```json
{
@@ -296,9 +296,9 @@ behavior).
}
```
-Principal `foo` can register frameworks with the `analytics` and `ads` roles
-and no other role. Any other principal (or framework without a principal) can
-register frameworks with any role.
+Principal `foo` can register frameworks subscribed to the `analytics` and
+`ads` roles and no other role. Any other principal (or framework without
+a principal) can register frameworks subscribed to any roles.
```json
{
@@ -323,9 +323,9 @@ register frameworks with any role.
}
```
-Only principal `foo` and no one else can register frameworks with the
+Only principal `foo` and no one else can register frameworks subscribed to the
`analytics` role. Any other principal (or framework without a principal) can
-register frameworks with any other role.
+register frameworks subscribed to any other roles.
```json
{
@@ -350,8 +350,9 @@ register frameworks with any other role.
}
```
-Principal `foo` can register frameworks with the `analytics` role and no other
-role. No other principal can register frameworks with any role, including `*`.
+Principal `foo` can register frameworks subscribed to the `analytics` role
+and no other roles. No other principal can register frameworks subscribed to
+any roles, including `*`.
```json
{
http://git-wip-us.apache.org/repos/asf/mesos/blob/b1eb5711/docs/persistent-volume.md
----------------------------------------------------------------------
diff --git a/docs/persistent-volume.md b/docs/persistent-volume.md
index 410993f..bd2f539 100644
--- a/docs/persistent-volume.md
+++ b/docs/persistent-volume.md
@@ -83,7 +83,7 @@ receive a resource offer with 2048 MB of dynamically reserved disk.
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 },
- "role" : <framework_role>,
+ "role" : <offer's allocation role>,
"reservation" : {
"principal" : <framework_principal>
}
@@ -115,7 +115,7 @@ volume information. We need to specify the following:
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 },
- "role" : <framework_role>,
+ "role" : <offer's allocation role>,
"reservation" : {
"principal" : <framework_principal>
},
@@ -147,7 +147,7 @@ persistent volume:
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 },
- "role" : <framework_role>,
+ "role" : <offer's allocation role>,
"reservation" : {
"principal" : <framework_principal>
},
@@ -182,7 +182,7 @@ created. First, we receive a resource offer (copy/pasted from above):
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 },
- "role" : <framework_role>,
+ "role" : <offer's allocation role>,
"reservation" : {
"principal" : <framework_principal>
},
@@ -211,7 +211,7 @@ specifies the persistent volumes to be destroyed.
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 },
- "role" : <framework_role>,
+ "role" : <offer's allocation role>,
"reservation" : {
"principal" : <framework_principal>
},
@@ -244,7 +244,7 @@ contain the following reserved disk resources:
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 },
- "role" : <framework_role>,
+ "role" : <offer's allocation role>,
"reservation" : {
"principal" : <framework_principal>
}
@@ -468,21 +468,22 @@ volumes:
next.
* Because persistent volumes are associated with roles, a volume might be
- offered to _any_ of the frameworks that are registered in that role. For
+ offered to _any_ of the frameworks that are subscribed to that role. For
example, a persistent volume might be created by one framework and then
- offered to a different framework in the same role. This can be used to pass
- large volumes of data between frameworks in a convenient way. However, this
- behavior might also allow sensitive data created by one framework to be read
- or modified by another framework in the same role. It can also make it more
- difficult for frameworks to determine whether a dynamic reservation has
- succeeded: as discussed above, frameworks need to wait for an offer that
- contains the "expected" reserved resources to determine when a reservation
- request has succeeded. Determining what a framework should "expect" to find in
- an offer is more difficult when multiple frameworks can make reservations
- for the same role concurrently. In general, whenever multiple frameworks are
- allowed to register in the same role, the operator should ensure that those
- frameworks are configured to collaborate with one another when using
- role-specific resources. For more information, see the discussion of
+ offered to a different framework subscribed to the same role. This can be
+ used to pass large volumes of data between frameworks in a convenient way.
+ However, this behavior might also allow sensitive data created by one
+ framework to be read or modified by another framework subscribed to the
+ same role. It can also make it more difficult for frameworks to determine
+ whether a dynamic reservation has succeeded: as discussed above, frameworks
+ need to wait for an offer that contains the "expected" reserved resources
+ to determine when a reservation request has succeeded. Determining what a
+ framework should "expect" to find in an offer is more difficult when
+ multiple frameworks can make reservations for the same role concurrently.
+ In general, whenever multiple frameworks are allowed to subscribe to the
+ same role, the operator should ensure that those frameworks are configured
+ to collaborate with one another when using role-specific resources. For
+ more information, see the discussion of
[multiple frameworks in the same role](roles.md#roles-multiple-frameworks).
## Version History
http://git-wip-us.apache.org/repos/asf/mesos/blob/b1eb5711/docs/quota.md
----------------------------------------------------------------------
diff --git a/docs/quota.md b/docs/quota.md
index 931542b..f36ab1a 100644
--- a/docs/quota.md
+++ b/docs/quota.md
@@ -60,21 +60,21 @@ use-cases are supported by quota, and which are not.
### Scenario 1: Greedy Framework
-There are two frameworks in a cluster, each running in a separate role with
-equal weights: framework fA in role rA and framework fB in role rB. There is a
-single resource available in the cluster: 100 CPUs. fA consumes 10 CPUs and is
-idle (declines resource offers), while fB is greedy and accepts all offers it gets,
-hogging the remaining 90 CPUs. Without quota, though fA's fair share is 50 CPUs
-it will not be able to make use of additional the 40 CPUs until some of fB's
-tasks terminate.
+There are two frameworks in a cluster, each are subscribed to their own distinct
+role, and both roles have equal weights: framework fA subscribed to role rA and
+framework fB subscribed to role rB. There is a single resource available in the
+cluster: 100 CPUs. fA consumes 10 CPUs and is idle (declines resource offers),
+while fB is greedy and accepts all offers it gets, hogging the remaining 90
+CPUs. Without quota, though fA's fair share is 50 CPUs it will not be able to
+make use of additional the 40 CPUs until some of fB's tasks terminate.
### Scenario 2: Resources for a new Framework
-A greedy framework fB in role rB is currently the only framework in the cluster
-and it uses all available resources---100 CPUs. If a new framework fA in role rA
-joins the cluster, it will not receive its fair share of the cluster resources
-(50 CPUs) until some of fB's tasks terminate.
+A greedy framework fB subscribed to role rB is currently the only framework in
+the cluster and it uses all available resources---100 CPUs. If a new framework
+fA subscribed to role rA joins the cluster, it will not receive its fair share
+of the cluster resources (50 CPUs) until some of fB's tasks terminate.
To deal with Scenario 2, quota by itself is not a sufficient solution as it
would be set after fB has started using all resources. Instead Scenario 2
@@ -300,17 +300,17 @@ outstanding offers with the following rules:
agent are rescinded. This is done in order to make the potential offer bigger,
which increases the chances that a quota'ed framework will be able to use the
offer.
-* Rescind offers from at least as many agents as there are frameworks in the
- role for which quota is being set. This enables (but does not guarantee, due
- to fair sharing) each framework in the role to receive an offer.
+* Rescind offers from at least as many agents as there are frameworks subscribed
+ to the role for which quota is being set. This enables (but does not guarantee,
+ due to fair sharing) each framework subscribed to the role to receive an offer.
<a name="allocatorEnforcement"></a>
## Enforcement by wDRF Allocator
The wDRF allocator first allocates (or lays away if offers are declined)
-resources to framework in roles with quota set. Once all quotas are
-satisfied, it proceeds with the standard wDRF for all frameworks.
+resources to roles with quota set. Once all quotas are satisfied, it proceeds
+with the standard wDRF for all remaining roles.
**NOTE:** A quota'ed role may not be allocated any unreserved non-revocable
resources beyond its quota guarantee. If frameworks in the quota'ed role have
@@ -319,12 +319,12 @@ the role is satisfied. In this case setting quota to any value that is less than
the role's fair share may reduce the amount of resources offered to this role.
**NOTE:** Currently quota guarantee also serves as quota limit, i.e. once quota
-for the role is satisfied, frameworks in this role will not be offered any
-resources except those reserved for the role. This behavior aims to mitigate the
-absence of quota limit and will be changed in future releases.
+for the role is satisfied, no further resources will be offered to the role
+except those reserved for the role. This behavior aims to mitigate the absence
+of quota limit and will be changed in future releases.
-If there are multiple frameworks in a role with quota set, the standard wDRF
-algorithm determines framework priority inside this role.
+If there are multiple frameworks subscribed to a role with quota set, the
+standard wDRF algorithm determines offer precedence amongst these frameworks.
The default wDRF allocator considers only non-revocable resources as applicable
towards quota.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b1eb5711/docs/reservation.md
----------------------------------------------------------------------
diff --git a/docs/reservation.md b/docs/reservation.md
index ace5cef..3a42783 100644
--- a/docs/reservation.md
+++ b/docs/reservation.md
@@ -47,13 +47,14 @@ That is, statically reserved resources cannot be reserved for another role nor
be unreserved. Dynamic reservation enables operators and authorized frameworks
to reserve and unreserve resources after slave-startup.
-By default, frameworks and operators can reserve resources for any role, and can
-unreserve any dynamically reserved resources. [Authorization](authorization.md)
-allows this behavior to be limited so that only particular roles can be reserved
-for, and only particular resources can be unreserved. For these operations to be
-authorized, the framework or operator should provide a `principal` to identify
-itself. To use authorization with reserve/unreserve operations, the Mesos master
-must be configured with the appropriate ACLs. For more information, see the
+By default, frameworks and operators are authorized to reserve resources for
+any role and to unreserve dynamically reserved resources.
+[Authorization](authorization.md) allows this behavior to be limited so that
+only particular roles can be reserved for, and only particular resources can
+be unreserved. For these operations to be authorized, the framework or operator
+should provide a `principal` to identify itself. To use authorization with
+reserve/unreserve operations, the Mesos master must be configured with the
+appropriate ACLs. For more information, see the
[authorization documentation](authorization.md).
* `Offer::Operation::Reserve` and `Offer::Operation::Unreserve` messages are
@@ -95,22 +96,27 @@ same slave and use the same role.
<a name="offer-operation-reserve"></a>
#### `Offer::Operation::Reserve`
-A framework can reserve resources through the resource offer cycle. Suppose we
-receive a resource offer with 12 CPUs and 6144 MB of RAM unreserved.
+A framework can reserve resources through the resource offer cycle. The
+reservation role must match the offer's allocation role. Suppose we
+receive a resource offer with 12 CPUs and 6144 MB of RAM unreserved, allocated
+to role `"engineering"`.
{
+ "allocation_info": { "role": "engineering" },
"id": <offer_id>,
"framework_id": <framework_id>,
"slave_id": <slave_id>,
"hostname": <hostname>,
"resources": [
{
+ "allocation_info": { "role": "engineering" },
"name": "cpus",
"type": "SCALAR",
"scalar": { "value": 12 },
"role": "*",
},
{
+ "allocation_info": { "role": "engineering" },
"name": "mem",
"type": "SCALAR",
"scalar": { "value": 6144 },
@@ -122,33 +128,36 @@ receive a resource offer with 12 CPUs and 6144 MB of RAM unreserved.
We can reserve 8 CPUs and 4096 MB of RAM by sending the following
`Offer::Operation` message. `Offer::Operation::Reserve` has a `resources` field
which we specify with the resources to be reserved. We must explicitly set the
-resources' `role` field with the framework's role. The required value of the
-`principal` field depends on whether or not the framework provided a principal
-when it registered with the master. If a principal was provided, then the
-resources' `principal` field must be equal to the framework's principal. If no
-principal was provided during registration, then the resources' `principal`
-field can take any value, or can be left unset. Note that the `principal` field
-determines the "reserver principal" when [authorization](authorization.md) is
-enabled, even if authentication is disabled.
+resources' `role` field to the offer's allocation role. The required value of
+the `principal` field depends on whether or not the framework provided a
+principal when it registered with the master. If a principal was provided, then
+the resources' `principal` field must be equal to the framework's principal.
+If no principal was provided during registration, then the resources'
+`principal` field can take any value, or can be left unset. Note that the
+`principal` field determines the "reserver principal" when
+[authorization](authorization.md) is enabled, even if authentication is
+disabled.
{
"type": Offer::Operation::RESERVE,
"reserve": {
"resources": [
{
+ "allocation_info": { "role": "engineering" },
"name": "cpus",
"type": "SCALAR",
"scalar": { "value": 8 },
- "role": <framework_role>,
+ "role": "engineering",
"reservation": {
"principal": <framework_principal>
}
},
{
+ "allocation_info": { "role": "engineering" },
"name": "mem",
"type": "SCALAR",
"scalar": { "value": 4096 },
- "role": <framework_role>,
+ "role": "engineering",
"reservation": {
"principal": <framework_principal>
}
@@ -161,25 +170,28 @@ If the reservation is successful, a subsequent resource offer will contain the
following reserved resources:
{
+ "allocation_info": { "role": "engineering" },
"id": <offer_id>,
"framework_id": <framework_id>,
"slave_id": <slave_id>,
"hostname": <hostname>,
"resources": [
{
+ "allocation_info": { "role": "engineering" },
"name": "cpus",
"type": "SCALAR",
"scalar": { "value": 8 },
- "role": <framework_role>,
+ "role": "engineering",
"reservation": {
"principal": <framework_principal>
}
},
{
+ "allocation_info": { "role": "engineering" },
"name": "mem",
"type": "SCALAR",
"scalar": { "value": 4096 },
- "role": <framework_role>,
+ "role": "engineering",
"reservation": {
"principal": <framework_principal>
}
@@ -192,31 +204,34 @@ following reserved resources:
A framework can unreserve resources through the resource offer cycle.
In [Offer::Operation::Reserve](#offer-operation-reserve), we reserved 8 CPUs
-and 4096 MB of RAM on a particular slave for our `role`. The master will
-continue to only offer these resources to our `role`. Suppose we would like to
-unreserve these resources. First, we receive a resource offer (copy/pasted
-from above):
+and 4096 MB of RAM on a particular slave for one of our subscribed roles
+(e.g. `"engineering"`). The master will continue to only offer these reserved
+resources to the reservation's `role`. Suppose we would like to unreserve
+these resources. First, we receive a resource offer (copy/pasted from above):
{
+ "allocation_info": { "role": "engineering" },
"id": <offer_id>,
"framework_id": <framework_id>,
"slave_id": <slave_id>,
"hostname": <hostname>,
"resources": [
{
+ "allocation_info": { "role": "engineering" },
"name": "cpus",
"type": "SCALAR",
"scalar": { "value": 8 },
- "role": <framework_role>,
+ "role": "engineering",
"reservation": {
"principal": <framework_principal>
}
},
{
+ "allocation_info": { "role": "engineering" },
"name": "mem",
"type": "SCALAR",
"scalar": { "value": 4096 },
- "role": <framework_role>,
+ "role": "engineering",
"reservation": {
"principal": <framework_principal>
}
@@ -233,19 +248,21 @@ which we can use to specify the resources to be unreserved.
"unreserve": {
"resources": [
{
+ "allocation_info": { "role": "engineering" },
"name": "cpus",
"type": "SCALAR",
"scalar": { "value": 8 },
- "role": <framework_role>,
+ "role": "engineering",
"reservation": {
"principal": <framework_principal>
}
},
{
+ "allocation_info": { "role": "engineering" },
"name": "mem",
"type": "SCALAR",
"scalar": { "value": 4096 },
- "role": <framework_role>,
+ "role": "engineering",
"reservation": {
"principal": <framework_principal>
}
http://git-wip-us.apache.org/repos/asf/mesos/blob/b1eb5711/docs/roles.md
----------------------------------------------------------------------
diff --git a/docs/roles.md b/docs/roles.md
index 344a0e3..33452ec 100644
--- a/docs/roles.md
+++ b/docs/roles.md
@@ -5,30 +5,44 @@ layout: documentation
# Roles
-In Mesos, __roles__ can be used to specify that certain
-[resources](attributes-resources.md) are reserved for the use of one or more
-frameworks. Roles can be used to enable a variety of restrictions on how
-resources are offered to frameworks. Some use-cases for roles include:
-
-* arranging for all the resources on a particular agent to only be offered to a
- particular framework.
-* dividing a cluster between two organizations: resources reserved for use by
- organization _A_ will only be offered to frameworks that have registered
- using organization _A_'s role (see the
- [reservation documentation](reservation.md)).
-* ensuring that [persistent volumes](persistent-volume.md) created by one
- framework are not offered to frameworks registered with a different role.
-* expressing that one group of frameworks should be considered "higher priority"
- (and offered more resources) than another group of frameworks.
-* setting a guaranteed resource allocation for one or more frameworks belonging
- to a role (see the [quota documentation](quota.md)).
-
+Many modern host-level operating systems (e.g. Linux, BSDs, etc) support
+multiple users. Similarly, Mesos is a multi-user cluster management system,
+with the expectation of a single Mesos cluster managing an organization's
+resources and servicing the organization's users.
+
+As such, Mesos has to address a number of requirements related to resource
+management:
+
+* Fair sharing of the resources amongst users
+* Providing resource guarantees to users (e.g. quota, priorities, isolation)
+* Providing accurate resource accounting
+ * How many resources are allocated / utilized / etc?
+ * Per-user accounting
+
+In Mesos, we refer to these "users" as __roles__. More precisely, a __role__
+within Mesos refers to a resource consumer within the cluster. This resource
+consumer could represent a user within an organization, but it could also
+represent a team, a group, a service, a framework, etc.
+
+Schedulers subscribe to one or more roles in order to receive resources and
+schedule work on behalf of the resource consumer(s) they are servicing.
+
+Some examples of resource allocation guarantees that Mesos provides:
+
+* Guaranteeing that a role is allocated a specified amount of resources
+ (via [quota](quota.md)).
+* Ensuring that some (or all) of the resources on a particular agent
+ are allocated to a particular role (via [reservations](reservation.md)).
+* Ensuring that resources are fairly shared between roles
+ (via [DRF](https://www.cs.berkeley.edu/~alig/papers/drf.pdf)).
+* Expressing that some roles should receive a higher relative share of the
+ cluster (via [weights](weights.md)).
## Roles and access control
-There are two ways to control which roles a framework is allowed to register
-as. First, ACLs can be used to specify which framework principals can register
-as which roles. For more information, see the [authorization](authorization.md)
+There are two ways to control which roles a framework is allowed to subscribe
+to. First, ACLs can be used to specify which framework principals can subscribe
+to which roles. For more information, see the [authorization](authorization.md)
documentation.
Second, a _role whitelist_ can be configured by passing the `--roles` flag to
@@ -49,36 +63,39 @@ recommended practice is to only use ACLs to define which roles can be used; the
## Associating frameworks with roles
-A framework can optionally specify the role it would like to use when it
-registers with the master.
+A framework specifies which roles it would like to subscribe to when it
+subscribes with the master.
-As a developer, you can specify the role your framework will use via the `role`
-field of the `FrameworkInfo` message.
+As a framework developer, you must specify the roles you would like to
+subscribe to via the `FrameworkInfo.roles` field.
-As a user, you can typically specify which role a framework will use when you
-start the framework. How to do this depends on the user interface of the
-framework you're using; for example, Marathon takes a `--mesos_role`
-command-line flag.
+As a user, you can typically specify which role(s) a framework will
+subscribe to when you start the framework. How to do this depends on the
+user interface of the framework you're using. For example, a single user
+scheduler might take a `--mesos_role` command-line flag, and a multi-user
+scheduler might take a `--mesos-roles` command-line flag or sync with
+the organization's LDAP system to automatically adjust which roles it
+is subscribed to as the organization's structure changes.
<a id="roles-multiple-frameworks"></a>
### Multiple frameworks in the same role
-Multiple frameworks can use the same role. This can be useful: for example, one
-framework can create a persistent volume and write data to it. Once the task
-that writes data to the persistent volume has finished, the volume will be
-offered to other frameworks in the same role; this might give a second
-("consumer") framework the opportunity to launch a task that reads the data
-produced by the first ("producer") framework.
+Multiple frameworks can be subscribed to the same role. This can be useful:
+for example, one framework can create a persistent volume and write data to
+it. Once the task that writes data to the persistent volume has finished,
+the volume will be offered to other frameworks subscribed to the same role;
+this might give a second ("consumer") framework the opportunity to launch a
+task that reads the data produced by the first ("producer") framework.
However, configuring multiple frameworks to use the same role should be done
with caution, because all the frameworks will have access to any resources that
have been reserved for that role. For example, if a framework stores sensitive
information on a persistent volume, that volume might be offered to a different
-framework in the same role. Similarly, if one framework creates a persistent
-volume, another framework in the same role might "steal" the volume and use it
-to launch a task of its own. In general, multiple frameworks sharing the same
-role should be prepared to collaborate with one another to ensure that
-role-specific resources are used appropriately.
+framework subscribed to the same role. Similarly, if one framework creates a
+persistent volume, another framework subscribed to the same role might "steal"
+the volume and use it to launch a task of its own. In general, multiple
+frameworks sharing the same role should be prepared to collaborate with one
+another to ensure that role-specific resources are used appropriately.
## Associating resources with roles
@@ -88,20 +105,18 @@ _dynamically_: frameworks and operators can specify that a certain resource
should subsequently be reserved for use by a given role. For more information,
see the [reservation](reservation.md) documentation.
-## The default role
+## Default role
-The role named `*` is special. Resources that are assigned to the `*` role are
-considered "unreserved"; similarly, when a framework registers without providing
-a role, it is assigned to the `*` role. By default, all the resources at an
-agent node are initially assigned to the `*` role (this can be changed via the
-`--default_role` command-line flag when starting the agent).
+The role named `*` is special. Unreserved resources are currently represented
+as having the special `*` role (the idea being that `*` matches any role). By
+default, all the resources at an agent node are unreserved (this can be changed
+via the `--default_role` command-line flag when starting the agent).
-The `*` role behaves differently from non-default roles. For example, dynamic
-reservations can be used to reassign resources from the `*` role to a specific
-role, but not from one specific role to another specific role (without first
-unreserving the resource, e.g., using the [/unreserve](endpoints/master/unreserve.md)
-operator HTTP endpoint). Similarly, persistent volumes cannot be created on
-unreserved resources.
+In addition, when a framework registers without providing a
+`FrameworkInfo.role`, it is assigned to the `*` role. In Mesos 1.3, frameworks
+should use the `FrameworkInfo.roles` field, which does not assign a default of
+`*`, but frameworks can still specify `*` explicitly if desired. Frameworks
+and operators cannot make reservations to the `*` role.
## Invalid role
@@ -114,10 +129,11 @@ A role name must be a valid directory name, so it cannot:
## Roles and resource allocation
-By default, the Mesos master uses Dominant Resource Fairness (DRF) to allocate
-resources. In particular, this implementation of DRF first identifies which
-_role_ is furthest below its fair share of the role's dominant resource. Each of
-the frameworks in that role are then offered additional resources in turn.
+By default, the Mesos master uses weighted Dominant Resource Fairness (wDRF) to
+allocate resources. In particular, this implementation of wDRF first identifies
+which _role_ is furthest below its fair share of the role's dominant resource.
+Each of the frameworks subscribed to that role are then offered additional
+resources in turn.
The resource allocation process can be customized by assigning
_[weights](weights.md)_ to roles: a role with a weight of 2 will be allocated
@@ -126,6 +142,14 @@ weight of 1. Weights can be configured using the
[/weights](endpoints/master/weights.md) operator endpoint, or else using the
deprecated `--weights` command-line flag when starting the Mesos master.
+## Roles and quota
+
+In order to guarantee that a role is allocated a specific amount of resources,
+quota can be specified via the [/quota](endpoints/master/quota.md) endpoint.
+
+The resource allocator will first attempt to satisfy the quota requirements,
+before fairly sharing the remaining resources. For more information, see the
+[quota](quota.md) documentation.
## Role vs. Principal
@@ -138,5 +162,5 @@ identity, and the principal may be used to [authorize](authorization.md) actions
performed by an entity, such as [resource reservation](reservation.md) and
[persistent volume](persistent-volume.md) creation/destruction.
-Roles, on the other hand, are used exclusively to associate resources with
-frameworks in various ways, as covered above.
+Roles, on the other hand, are used exclusively for resource allocation, as
+covered above.
http://git-wip-us.apache.org/repos/asf/mesos/blob/b1eb5711/docs/scheduler-http-api.md
----------------------------------------------------------------------
diff --git a/docs/scheduler-http-api.md b/docs/scheduler-http-api.md
index 7f808f1..4a5d77b 100644
--- a/docs/scheduler-http-api.md
+++ b/docs/scheduler-http-api.md
@@ -109,6 +109,8 @@ Connection: close
"framework_info" : {
"user" : "foo",
"name" : "Example HTTP Framework"
+ "roles": ["test"],
+ "capabilities" : [{"type": "MULTI_ROLE"}]
}
}
}
@@ -163,7 +165,7 @@ HTTP/1.1 202 Accepted
```
### ACCEPT
-Sent by the scheduler when it accepts offer(s) sent by the master. The `ACCEPT` request includes the type of operations (e.g., launch task, launch task group, reserve resources, create volumes) that the scheduler wants to perform on the offers. Note that until the scheduler replies (accepts or declines) to an offer, the offer's resources are considered allocated to the framework. Also, any of the offer's resources not used in the `ACCEPT` call (e.g., to launch a task or task group) are considered declined and might be reoffered to other frameworks. In other words, the same `OfferID` cannot be used in more than one `ACCEPT` call. These semantics might change when we add new features to Mesos (e.g., persistence, reservations, optimistic offers, resizeTask, etc.).
+Sent by the scheduler when it accepts offer(s) sent by the master. The `ACCEPT` request includes the type of operations (e.g., launch task, launch task group, reserve resources, create volumes) that the scheduler wants to perform on the offers. Note that until the scheduler replies (accepts or declines) to an offer, the offer's resources are considered allocated to the offer's role and to the framework. Also, any of the offer's resources not used in the `ACCEPT` call (e.g., to launch a task or task group) are considered declined and might be reoffered to other frameworks. In other words, the same `OfferID` cannot be used in more than one `ACCEPT` call. These semantics might change when we add new features to Mesos (e.g., persistence, reservations, optimistic offers, resizeTask, etc.).
```
ACCEPT Request (JSON):
@@ -198,12 +200,14 @@ Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
},
"resources" : [
{
- "name" : "cpus",
+ "allocation_info": {"role": "engineering"},
+ "name" : "cpus",
"role" : "*",
"type" : "SCALAR",
"scalar": {"value": 1.0}
},
{
+ "allocation_info": {"role": "engineering"},
"name" : "mem",
"role" : "*",
"type" : "SCALAR",
@@ -264,8 +268,9 @@ Content-Type: application/json
Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
- "framework_id" : {"value" : "12220-3440-12532-2345"},
- "type" : "REVIVE"
+ "framework_id" : {"value" : "12220-3440-12532-2345"},
+ "type" : "REVIVE",
+ "revive" : {"role": <one-of-the-subscribed-roles>}
}
REVIVE Response:
@@ -441,7 +446,7 @@ The first event sent by the master when the scheduler sends a `SUBSCRIBE` reques
### OFFERS
-Sent by the master whenever there are new resources that can be offered to the framework. Each offer corresponds to a set of resources on an agent. Until the scheduler 'Accept's or 'Decline's an offer the resources are considered allocated to the scheduler, unless the offer is otherwise rescinded, e.g., due to a lost agent or `--offer_timeout`.
+Sent by the master whenever there are new resources that can be offered to the framework. Each offer corresponds to a set of resources on an agent and is allocated to one of roles the framework is subscribed to. Until the scheduler 'Accept's or 'Decline's an offer the resources are considered allocated to the scheduler, unless the offer is otherwise rescinded, e.g., due to a lost agent or `--offer_timeout`.
```
OFFERS Event (JSON)
@@ -451,28 +456,30 @@ OFFERS Event (JSON)
"type" : "OFFERS",
"offers" : [
{
- "offer_id" : {"value": "12214-23523-O235235"},
- "framework_id" : {"value": "12124-235325-32425"},
- "agent_id" : {"value": "12325-23523-S23523"},
- "hostname" : "agent.host",
- "resources" : [
- {
- "name" : "cpus",
- "type" : "SCALAR",
- "scalar" : {"value" : 2},
- "role" : "*"
- }
- ],
- "attributes" : [
- {
- "name" : "os",
- "type" : "TEXT",
- "text" : {"value" : "ubuntu16.04"}
- }
- ],
- "executor_ids" : [
- {"value" : "12214-23523-my-executor"}
- ]
+ "allocation_info": { "role": "engineering" },
+ "id" : {"value": "12214-23523-O235235"},
+ "framework_id" : {"value": "12124-235325-32425"},
+ "agent_id" : {"value": "12325-23523-S23523"},
+ "hostname" : "agent.host",
+ "resources" : [
+ {
+ "allocation_info": { "role": "engineering" },
+ "name" : "cpus",
+ "type" : "SCALAR",
+ "scalar" : {"value" : 2},
+ "role" : "*"
+ }
+ ],
+ "attributes" : [
+ {
+ "name" : "os",
+ "type" : "TEXT",
+ "text" : {"value" : "ubuntu16.04"}
+ }
+ ],
+ "executor_ids" : [
+ {"value" : "12214-23523-my-executor"}
+ ]
}
]
}
@@ -551,7 +558,7 @@ FAILURE Event (JSON)
```
### ERROR
-Sent by the master when an asynchronous error event is generated (e.g., a framework is not authorized to subscribe with the given role). It is recommended that the framework abort when it receives an error and retry subscription as necessary.
+Sent by the master when an asynchronous error event is generated (e.g., a framework is not authorized to subscribe with one of the given roles). It is recommended that the framework abort when it receives an error and retry subscription as necessary.
```
ERROR Event (JSON)
http://git-wip-us.apache.org/repos/asf/mesos/blob/b1eb5711/docs/shared-resources.md
----------------------------------------------------------------------
diff --git a/docs/shared-resources.md b/docs/shared-resources.md
index 1b490e2..2643c28 100644
--- a/docs/shared-resources.md
+++ b/docs/shared-resources.md
@@ -25,21 +25,23 @@ either via the framework scheduler API or the
[/create-volumes](endpoints/master/create-volumes.md) HTTP endpoint. To
create a shared volume, set the `shared` field during volume creation.
-For example, suppose a framework receives a resource offer containing
-2048MB of dynamically reserved disk:
+For example, suppose a framework subscribed to the `"engineering"` role
+receives a resource offer containing 2048MB of dynamically reserved disk:
```
{
+ "allocation_info": { "role": "engineering" },
"id" : <offer_id>,
"framework_id" : <framework_id>,
"slave_id" : <slave_id>,
"hostname" : <hostname>,
"resources" : [
{
+ "allocation_info": { "role": "engineering" },
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 },
- "role" : <framework_role>,
+ "role" : "engineering",
"reservation" : {
"principal" : <framework_principal>
}
@@ -57,10 +59,11 @@ resource via the following offer operation:
"create": {
"volumes" : [
{
+ "allocation_info": { "role": "engineering" },
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 },
- "role" : <framework_role>,
+ "role" : "engineering",
"reservation" : {
"principal" : <framework_principal>
},
@@ -95,10 +98,10 @@ shared resources.
When a framework receives a resource offer, it can determine whether a
volume is shared by checking if the `shared` field has been set. Unlike
normal persistent volumes, a shared volume that is in use by a task will
-continue to be offered to the frameworks in the volume's role; this
-gives those frameworks the opportunity to launch additional tasks that
-can access the volume. A framework can also launch multiple tasks that
-access the volume using a single `ACCEPT` call.
+continue to be offered to the frameworks subscribed to the volume's role;
+this gives those frameworks the opportunity to launch additional tasks
+that can access the volume. A framework can also launch multiple tasks
+that access the volume using a single `ACCEPT` call.
Note that Mesos does not provide any isolation or concurrency control
between the tasks that are sharing a volume. Framework developers should
[2/2] mesos git commit: Added MULTI_ROLE support to the upgrades
documentation.
Posted by bm...@apache.org.
Added MULTI_ROLE support to the upgrades documentation.
Review: https://reviews.apache.org/r/58061
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/0d4e08ce
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/0d4e08ce
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/0d4e08ce
Branch: refs/heads/master
Commit: 0d4e08ce3725cf3c0b91acbe27870bc41c430429
Parents: b1eb571
Author: Benjamin Mahler <bm...@apache.org>
Authored: Wed Mar 29 18:36:40 2017 -0700
Committer: Benjamin Mahler <bm...@apache.org>
Committed: Fri Mar 31 17:13:38 2017 -0700
----------------------------------------------------------------------
docs/upgrades.md | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/0d4e08ce/docs/upgrades.md
----------------------------------------------------------------------
diff --git a/docs/upgrades.md b/docs/upgrades.md
index 249e844..a12e6ea 100644
--- a/docs/upgrades.md
+++ b/docs/upgrades.md
@@ -53,10 +53,20 @@ We categorize the changes as follows:
</ul>
</td>
<td style="word-wrap: break-word; overflow-wrap: break-word;"><!--Framework API-->
+ <ul style="padding-left:10px;">
+ <li>A <a href="#1-3-x-multi-role-support">MULTI_ROLE support</a></li>
+ <li>D <a href="#1-3-x-framework-info-role">FrameworkInfo.roles deprecates FrameworkInfo.role</a></li>
+ </ul>
</td>
<td style="word-wrap: break-word; overflow-wrap: break-word;"><!--Module API-->
+ <ul style="padding-left:10px;">
+ <li>C <a href="#1-3-x-allocator-interface-change">Allocator MULTI_ROLE interface changes</a></li>
+ </ul>
</td>
<td style="word-wrap: break-word; overflow-wrap: break-word;"><!--Endpoints-->
+ <ul style="padding-left:10px;">
+ <li>D <a href="#1-3-x-endpoints-roles">MULTI_ROLE deprecates 'role' field in endpoints</a></li>
+ </ul>
</td>
</tr>
<tr>
@@ -261,6 +271,15 @@ We categorize the changes as follows:
<a name="1-3-x-setquota-removequota-acl"></a>
* Support for deprecated ACLs `set_quotas` and `remove_quotas` has been removed from the local authorizer. Before upgrading the Mesos binaries, consolidate the ACLs used under `set_quotas` and `remove_quotes` under their replacemente ACL `update_quotas`. After consolidation of the ACLs, the binaries could be safely replaced.
+<a name="1-3-x-framework-info-role"></a>
+* Support for multi-role frameworks deprecates the `FrameworkInfo.role` field in favor of `FrameworkInfo.roles` and the `MULTI_ROLE` capability. Frameworks using the new field can continue to use a single role.
+
+<a name="1-3-x-endpoints-roles"></a>
+* Support for multi-role frameworks means that the framework `role` field in the master and agent endpoints is deprecated in favor of `roles`. Any tooling parsing endpoint information and relying on the role field needs to be updated before multi-role frameworks can be safely run in the cluster.
+
+<a name="1-3-x-allocator-interface-change"></a>
+* Implementors of allocator modules have to provide new implementation functionality to satisfy the `MULTI_ROLE` framework capability. Also, the interface has changed.
+
## Upgrading from 1.1.x to 1.2.x ##
<a name="1-2-x-heartbeat-flag"></a>