You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Benjamin Mahler <bm...@apache.org> on 2018/03/10 02:40:35 UTC
API Review: Quota Limits
As discussed during the API working group, we would like to ensure that
upcoming API changes are surfaced more broadly than just in Review Board.
The hope is that if we surface them on the dev@ list, it will raise
awareness and give a chance for more folks to give feedback on things like
naming, consistency, intuitiveness, etc of any API changes.
One upcoming change is the addition of quota "limits" to the quota API:
https://reviews.apache.org/r/65334/
JIRA / Design Doc:
https://issues.apache.org/jira/browse/MESOS-8068
https://docs.google.com/document/d/1EEpHIlJahYtjqv7zAEkW8U7Bi0CTY
FDqAhU0laM7Rzk/edit?usp=sharing (this is hierarchical, but we're tackling
non-hierarchical first).
Here's an overview:
(1) Currently, the default quota is: no guarantee and no limit. This will
remain unchanged.
(2) v0 /quota and v1 SET_QUOTA currently implicitly set quota limit equal
to quota guarantee.
(2)(a) For backwards compatibility, these APIs will continue to behave
the same.
(2)(b) We will also disallow users from explicitly setting limit in these
old APIs.
(3) A new UPDATE_QUOTA call will be added which allows users to set a limit.
(3)(a) This approach is more consistent with UPDATE_WEIGHTS: there is
always a quota, much like there is always a weight. If the quota is updated
to the default, it does not need to be persisted.
(3)(b) Unlike the old APIs, absence of a limit will mean no limit.
(3)(c) We'll validate that the limit does not exceed the cluster size and
the existing 'force' flag in QuotaRequest will bypass this validation.
Let me know if you have any feedback.
Ben
Re: API Review: Quota Limits
Posted by Benjamin Mahler <bm...@apache.org>.
The plan is to go with (1), so to check the "tree invariants" instead and
not bother checking the "satisfiability" of the guarantee and the
"reachability" of the limit. These latter concerns we would hope to tackle
in a more continuous manner (e.g. with monitoring and exposing information
like you're effective guarantee and limits at a given point in time).
On Mon, Mar 12, 2018 at 12:43 PM, Benjamin Mahler <bm...@apache.org>
wrote:
> Just to clarify up front, it's still a little unclear to me what the
> semantics of 'force' should be. The existing behavior with only guarantees
> is roughly equivalent to a check of whether the guarantee can be
> immediately attained. In other words, we'll let you set a guarantee without
> 'force' if it appears to be "guarantee-able".
>
> The first question is: should we maintain this existing behavior for
> guarantees in UPDATE_QUOTA? An alternative would be, to simplify the check
> to become an overcommit check. In other words, we'll let you set guarantees
> as long as the guarantees do not overcommit the cluster. This means if some
> roles are using resources above their guarantee (possibly having reserved
> them!), then not all of the guarantees will be satisfiable.
>
> When Meng and I discussed this, we were leaning more towards Mesos only
> checking the tree invariants in the new UPDATE_QUOTA API when 'force' isn't
> provided (i.e. parent.guarantee >= sum(child.guarantee) && limit <=
> child.limit, with root.guarantee == root.limit == total_cluster_resources).
> However, our discussion was mainly around whether we should try to check if
> the limit could be reached: limit <= parent.limit - sum(sibling.guarantees).
>
> There are a few options on the table for what to check when !force:
>
> (1) Check tree invariants: parent.guarantee >= sum(child.guarantee) &&
> limit <= child.limit, with root.guarantee == root.limit ==
> total_cluster_resources
>
> (2) Check (1) and also check that guarantee is guarantee-able: !overcommit
> && child.guarantee >= parent.guarantee - sum(sibling.allocation)
> (a) check role.limit >= role.allocation
> (b) check role.limit is "reachable": role.limit <= parent.limit -
> sum(sibling.guarantee)
>
> (1) is simple, but offers less help to users. (2)(b) Seems to try to offer
> the most help to users but is the most subtle. (2)(a) I'm not sure about
> since I think there will be a use case to set a limit on a greedy framework
> knowing that it will eventually release its resources.
>
> Do folks have any thoughts on these options?
>
> On Mon, Mar 12, 2018 at 1:22 AM, Qian Zhang <zh...@gmail.com> wrote:
>
>> For the new UPDATE_QUOTA call, do we allow setting limit lower than role's
>> existing allocation? If yes, does operator have to do it with the 'force'
>> flag?
>>
>>
>> Regards,
>> Qian Zhang
>>
>> On Sat, Mar 10, 2018 at 10:40 AM, Benjamin Mahler <bm...@apache.org>
>> wrote:
>>
>> > As discussed during the API working group, we would like to ensure that
>> > upcoming API changes are surfaced more broadly than just in Review
>> Board.
>> > The hope is that if we surface them on the dev@ list, it will raise
>> > awareness and give a chance for more folks to give feedback on things
>> like
>> > naming, consistency, intuitiveness, etc of any API changes.
>> >
>> > One upcoming change is the addition of quota "limits" to the quota API:
>> > https://reviews.apache.org/r/65334/
>> >
>> > JIRA / Design Doc:
>> > https://issues.apache.org/jira/browse/MESOS-8068
>> > https://docs.google.com/document/d/1EEpHIlJahYtjqv7zAEkW8U7Bi0CTY
>> > FDqAhU0laM7Rzk/edit?usp=sharing (this is hierarchical, but we're
>> tackling
>> > non-hierarchical first).
>> >
>> > Here's an overview:
>> >
>> > (1) Currently, the default quota is: no guarantee and no limit. This
>> will
>> > remain unchanged.
>> >
>> > (2) v0 /quota and v1 SET_QUOTA currently implicitly set quota limit
>> equal
>> > to quota guarantee.
>> > (2)(a) For backwards compatibility, these APIs will continue to behave
>> > the same.
>> > (2)(b) We will also disallow users from explicitly setting limit in
>> these
>> > old APIs.
>> >
>> > (3) A new UPDATE_QUOTA call will be added which allows users to set a
>> > limit.
>> > (3)(a) This approach is more consistent with UPDATE_WEIGHTS: there is
>> > always a quota, much like there is always a weight. If the quota is
>> updated
>> > to the default, it does not need to be persisted.
>> > (3)(b) Unlike the old APIs, absence of a limit will mean no limit.
>> > (3)(c) We'll validate that the limit does not exceed the cluster size
>> and
>> > the existing 'force' flag in QuotaRequest will bypass this validation.
>> >
>> > Let me know if you have any feedback.
>> >
>> > Ben
>> >
>>
>
>
Re: API Review: Quota Limits
Posted by Benjamin Mahler <bm...@apache.org>.
Just to clarify up front, it's still a little unclear to me what the
semantics of 'force' should be. The existing behavior with only guarantees
is roughly equivalent to a check of whether the guarantee can be
immediately attained. In other words, we'll let you set a guarantee without
'force' if it appears to be "guarantee-able".
The first question is: should we maintain this existing behavior for
guarantees in UPDATE_QUOTA? An alternative would be, to simplify the check
to become an overcommit check. In other words, we'll let you set guarantees
as long as the guarantees do not overcommit the cluster. This means if some
roles are using resources above their guarantee (possibly having reserved
them!), then not all of the guarantees will be satisfiable.
When Meng and I discussed this, we were leaning more towards Mesos only
checking the tree invariants in the new UPDATE_QUOTA API when 'force' isn't
provided (i.e. parent.guarantee >= sum(child.guarantee) && limit <=
child.limit, with root.guarantee == root.limit == total_cluster_resources).
However, our discussion was mainly around whether we should try to check if
the limit could be reached: limit <= parent.limit - sum(sibling.guarantees).
There are a few options on the table for what to check when !force:
(1) Check tree invariants: parent.guarantee >= sum(child.guarantee) &&
limit <= child.limit, with root.guarantee == root.limit ==
total_cluster_resources
(2) Check (1) and also check that guarantee is guarantee-able: !overcommit
&& child.guarantee >= parent.guarantee - sum(sibling.allocation)
(a) check role.limit >= role.allocation
(b) check role.limit is "reachable": role.limit <= parent.limit -
sum(sibling.guarantee)
(1) is simple, but offers less help to users. (2)(b) Seems to try to offer
the most help to users but is the most subtle. (2)(a) I'm not sure about
since I think there will be a use case to set a limit on a greedy framework
knowing that it will eventually release its resources.
Do folks have any thoughts on these options?
On Mon, Mar 12, 2018 at 1:22 AM, Qian Zhang <zh...@gmail.com> wrote:
> For the new UPDATE_QUOTA call, do we allow setting limit lower than role's
> existing allocation? If yes, does operator have to do it with the 'force'
> flag?
>
>
> Regards,
> Qian Zhang
>
> On Sat, Mar 10, 2018 at 10:40 AM, Benjamin Mahler <bm...@apache.org>
> wrote:
>
> > As discussed during the API working group, we would like to ensure that
> > upcoming API changes are surfaced more broadly than just in Review Board.
> > The hope is that if we surface them on the dev@ list, it will raise
> > awareness and give a chance for more folks to give feedback on things
> like
> > naming, consistency, intuitiveness, etc of any API changes.
> >
> > One upcoming change is the addition of quota "limits" to the quota API:
> > https://reviews.apache.org/r/65334/
> >
> > JIRA / Design Doc:
> > https://issues.apache.org/jira/browse/MESOS-8068
> > https://docs.google.com/document/d/1EEpHIlJahYtjqv7zAEkW8U7Bi0CTY
> > FDqAhU0laM7Rzk/edit?usp=sharing (this is hierarchical, but we're
> tackling
> > non-hierarchical first).
> >
> > Here's an overview:
> >
> > (1) Currently, the default quota is: no guarantee and no limit. This will
> > remain unchanged.
> >
> > (2) v0 /quota and v1 SET_QUOTA currently implicitly set quota limit equal
> > to quota guarantee.
> > (2)(a) For backwards compatibility, these APIs will continue to behave
> > the same.
> > (2)(b) We will also disallow users from explicitly setting limit in
> these
> > old APIs.
> >
> > (3) A new UPDATE_QUOTA call will be added which allows users to set a
> > limit.
> > (3)(a) This approach is more consistent with UPDATE_WEIGHTS: there is
> > always a quota, much like there is always a weight. If the quota is
> updated
> > to the default, it does not need to be persisted.
> > (3)(b) Unlike the old APIs, absence of a limit will mean no limit.
> > (3)(c) We'll validate that the limit does not exceed the cluster size
> and
> > the existing 'force' flag in QuotaRequest will bypass this validation.
> >
> > Let me know if you have any feedback.
> >
> > Ben
> >
>
Re: API Review: Quota Limits
Posted by Qian Zhang <zh...@gmail.com>.
For the new UPDATE_QUOTA call, do we allow setting limit lower than role's
existing allocation? If yes, does operator have to do it with the 'force'
flag?
Regards,
Qian Zhang
On Sat, Mar 10, 2018 at 10:40 AM, Benjamin Mahler <bm...@apache.org>
wrote:
> As discussed during the API working group, we would like to ensure that
> upcoming API changes are surfaced more broadly than just in Review Board.
> The hope is that if we surface them on the dev@ list, it will raise
> awareness and give a chance for more folks to give feedback on things like
> naming, consistency, intuitiveness, etc of any API changes.
>
> One upcoming change is the addition of quota "limits" to the quota API:
> https://reviews.apache.org/r/65334/
>
> JIRA / Design Doc:
> https://issues.apache.org/jira/browse/MESOS-8068
> https://docs.google.com/document/d/1EEpHIlJahYtjqv7zAEkW8U7Bi0CTY
> FDqAhU0laM7Rzk/edit?usp=sharing (this is hierarchical, but we're tackling
> non-hierarchical first).
>
> Here's an overview:
>
> (1) Currently, the default quota is: no guarantee and no limit. This will
> remain unchanged.
>
> (2) v0 /quota and v1 SET_QUOTA currently implicitly set quota limit equal
> to quota guarantee.
> (2)(a) For backwards compatibility, these APIs will continue to behave
> the same.
> (2)(b) We will also disallow users from explicitly setting limit in these
> old APIs.
>
> (3) A new UPDATE_QUOTA call will be added which allows users to set a
> limit.
> (3)(a) This approach is more consistent with UPDATE_WEIGHTS: there is
> always a quota, much like there is always a weight. If the quota is updated
> to the default, it does not need to be persisted.
> (3)(b) Unlike the old APIs, absence of a limit will mean no limit.
> (3)(c) We'll validate that the limit does not exceed the cluster size and
> the existing 'force' flag in QuotaRequest will bypass this validation.
>
> Let me know if you have any feedback.
>
> Ben
>