You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by as...@apache.org on 2020/11/05 12:46:40 UTC
[mesos] branch master updated: Documented setting offer constraints
via the scheduler API.
This is an automated email from the ASF dual-hosted git repository.
asekretenko pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
The following commit(s) were added to refs/heads/master by this push:
new ab272be Documented setting offer constraints via the scheduler API.
ab272be is described below
commit ab272be96edc40b69db5503e56b994198939db40
Author: Andrei Sekretenko <as...@apache.org>
AuthorDate: Tue Nov 3 21:19:54 2020 +0100
Documented setting offer constraints via the scheduler API.
Review: https://reviews.apache.org/r/73004
---
docs/app-framework-development-guide.md | 6 ++++
docs/scheduler-http-api.md | 60 ++++++++++++++++++++++++++++-----
2 files changed, 57 insertions(+), 9 deletions(-)
diff --git a/docs/app-framework-development-guide.md b/docs/app-framework-development-guide.md
index 8f30045..4653e1a 100644
--- a/docs/app-framework-development-guide.md
+++ b/docs/app-framework-development-guide.md
@@ -66,6 +66,12 @@ schedulers in the same Mesos cluster:
up-to-date with the minimum desired offer shape for each role will ensure that
the sccheduler gets a better chance to receive offers sized with sufficient
resources.
+6. Consider specifying **offer constraints** via `SUBSCRIBE`/`UPDATE_FRAMEWORK`
+ calls so that the framework role's quota is not consumed by offers that the
+ scheduler will have to decline anyway based on agent attributes.
+ See [MESOS-10161](https://issues.apache.org/jira/browse/MESOS-10161])
+ and [scheduler.proto](https://github.com/apache/mesos/blob/master/include/mesos/v1/scheduler/scheduler.proto)
+ for more details.
Operationally, the following can be done to ensure that schedulers get the resources
they need when co-existing with other schedulers:
diff --git a/docs/scheduler-http-api.md b/docs/scheduler-http-api.md
index e447ffb..6f4a804 100644
--- a/docs/scheduler-http-api.md
+++ b/docs/scheduler-http-api.md
@@ -55,7 +55,9 @@ The response returned from the `SUBSCRIBE` call (see [below](#subscribe)) is enc
This is the first step in the communication process between the scheduler and the master. This is also to be considered as subscription to the "/scheduler" event stream.
-To subscribe with the master, the scheduler sends an HTTP POST with a `SUBSCRIBE` message including the required FrameworkInfo and the list of initially suppressed roles (which must be a subset of roles in FrameworkInfo, see the section for `SUPPRESS` call). Note that if "subscribe.framework_info.id" and "FrameworkID" are not set, the master considers the scheduler as a new one and subscribes it by assigning it a FrameworkID. The HTTP response is a stream in RecordIO format; the event st [...]
+To subscribe with the master, the scheduler sends an HTTP POST with a `SUBSCRIBE` message including the required FrameworkInfo, the list of initially suppressed roles and the initial offer constraints. The initially suppressed roles, as well as roles for which offer constraints are specified, must be contained in the set of roles in FrameworkInfo. Note that Mesos 1.11.0 simply ignores constraints for invalid roles, but this might change in the future.
+
+Note that if "subscribe.framework_info.id" and "FrameworkID" are not set, the master considers the scheduler as a new one and subscribes it by assigning it a FrameworkID. The HTTP response is a stream in RecordIO format; the event stream begins with either a `SUBSCRIBED` event or an `ERROR` event (see details in **Events** section). The response also includes the `Mesos-Stream-Id` header, which is used by the master to uniquely identify the subscribed scheduler instance. This stream ID h [...]
```
SUBSCRIBE Request (JSON):
@@ -76,8 +78,20 @@ Connection: close
"roles": ["test1", "test2"],
"capabilities" : [{"type": "MULTI_ROLE"}]
},
- "suppressed_roles" : ["test2"]
- }
+ "suppressed_roles" : ["test2"],
+ "offer_constraints" : {
+ "role_constraints": {
+ "test1": {
+ "groups": [{
+ "attribute_constraints": [{
+ "selector": {"attribute_name": "foo"},
+ "predicate": {"exists": {}}
+ }]
+ }]
+ }
+ }
+ }
+ }
}
SUBSCRIBE Response Event (JSON):
@@ -103,10 +117,11 @@ with a `SUBSCRIBED` event. For further details, see the **Disconnections** secti
NOTE: In the old version of the API, (re-)registered callbacks also included MasterInfo, which contained information about the master the driver currently connected to. With the new API, since schedulers explicitly subscribe with the leading master (see details below in **Master Detection** section), it's not relevant anymore.
-NOTE: By providing a different FrameworkInfo and/or set of suppressed roles,
-re-subscribing scheduler can change some of the fields of FrameworkInfo and the
-set of suppressed roles. Allowed changes and their effects are consistent with
-those that can be performed via `UPDATE_FRAMEWORK` call (see below).
+NOTE: By providing a different FrameworkInfo and/or set of suppressed roles
+and/or offer constraints, a re-subscribing scheduler can change some of the
+fields of FrameworkInfo, the set of suppressed roles and/or offer constraints.
+Allowed changes and their effects are consistent with those that can be
+performed via `UPDATE_FRAMEWORK` call (see below).
If subscription fails for whatever reason (e.g., invalid request), an HTTP 4xx response is returned with the error message as part of the body and the connection is closed.
@@ -508,8 +523,8 @@ HTTP/1.1 202 Accepted
### UPDATE_FRAMEWORK
Sent by the scheduler to change fields of its `FrameworkInfo` and/or the set of
-suppressed roles. Allowed changes and their effects are consistent with changing
-FrameworkInfo and/or set of suppressed roles when re-subscribing.
+suppressed roles and/or offer constraints. Allowed changes and their effects
+are consistent with changing the same fields via re-subscribing.
#### Disallowed updates
Updating the following `FrameworkInfo` fields is not allowed:
@@ -539,6 +554,21 @@ Updating roles has the following effects:
* Other framework objects that use roles removed by this call (for example,
tasks) are not affected.
+#### Updating offer constraints
+For the `UPDATE_FRAMEWORK` call to be successfull, the `offer_constraints`
+field, if present, must be internally valid (for the constraints validity
+criteria, please refer to comments in
+[scheduler.proto](https://github.com/apache/mesos/blob/master/include/mesos/v1/scheduler/scheduler.proto))
+
+As of 1.11.0, Mesos ignores offer constraints for roles other than valid roles
+in `framework_info.roles`; future versions of Mesos are going to treat such
+offer constraints as invalid.
+
+Updated offer constraints have an immediate effect on offer generation after
+update, but have no effect on already outstanding offers. Frameworks should not
+expect that offers they receive right after the `UPDATE_FRAMEWORK` call
+will satisfy the new constraints.
+
#### Updating other fields
* Updating `name`, `hostname`, `webui_url` and `labels` is fully supported
by Mesos; these updates are simply propagated to Mesos API endpoints.
@@ -579,6 +609,18 @@ Connection: close
"capabilities" : [{"type": "MULTI_ROLE"}]
},
"suppressed_roles" : ["test2"]
+ "offer_constraints" : {
+ "role_constraints": {
+ "test1": {
+ "groups": [{
+ "attribute_constraints": [{
+ "selector": {"attribute_name": "foo"},
+ "predicate": {"exists": {}}
+ }]
+ }]
+ }
+ }
+ }
}
}