You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@mesos.apache.org by Benjamin Mahler <bm...@apache.org> on 2016/03/12 04:09:24 UTC

RFC: RevocableInfo Changes

Hey folks,

In the resource allocation working group we've been looking into a few
projects that will make the allocator able to offer out resources as
revocable. For example:

-We'll want to eventually allocate resources as revocable _by default_,
only allowing non-revocable when there are guarantees put in place (static
reservations or quota).

-On the path to revocable by default, we can incrementally start to offer
certain resources as revocable. Consider when quota is set but the role
isn't using all of the quota. The unallocated quota can be offered to other
roles, but it should be revocable because we may revoke them should the
quota'ed role want to use the resources. Unused reservations fall into a
similar category.

-Going revocable by default also allows us to enforce fairness in a
dynamically changing cluster by revoking resources as weights are changed,
frameworks are added or removed, etc.

In this context, "revocable" means that the resources may be taken away and
the container will be destroyed. The meaning of "revocable" in the context
of usage oversubscription includes this, but also the container may
experience a throttling (e.g. lower cpu shares, less network priority, etc).

For this reason, and because we internally need to distinguish revocable
resources between the those that are generated by usage oversubscription
and those that are generated by the allocator, we're thinking of the
following change to the API:



-  message RevocableInfo {}
+  message RevocableInfo {
+    message ThrottleInfo {}
+
+    // If set, indicates that the resources may be throttled at
+    // any time. Throttle-able resoruces can be used for tasks
+    // that do not have strict performance requirements and are
+    // capable of handling being throttled.
+    optional ThrottleInfo throttle_info;
+  }

   // If this is set, the resources are revocable, i.e., any tasks or
-  // executors launched using these resources could get preempted or
-  // throttled at any time. This could be used by frameworks to run
-  // best effort tasks that do not need strict uptime or performance
+  // executors launched using these resources could be terminated at
+  // any time. This could be used by frameworks to run
+  // best effort tasks that do not need strict uptime
   // guarantees. Note that if this is set, 'disk' or 'reservation'
   // cannot be set.
   optional RevocableInfo revocable = 9;



Essentially we want to distinguish between revocable and revocable +
throttle-able. This is because usage-oversubscription generates
throttle-able revocable resources, whereas the allocator does not. This
also solves our problem of distinguishing between these two kinds of
revocable resources internally.

Feedback welcome!

Ben

Re: RFC: RevocableInfo Changes

Posted by Klaus Ma <kl...@gmail.com>.

@ Niklas/Jie, just sent the invitation to you :)

----
Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
Platform OpenSource Technology, STG, IBM GCG
+86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me

On Tue, Mar 15, 2016 at 10:06 AM, Guangya Liu <gy...@gmail.com> wrote:

> This is the google doc that we used to trace all discussion topics:
>
> https://docs.google.com/document/d/1B_v52zCOFcwCpqCPhgYi9h630a0NE-QM9Br0nCOZUR4/edit?usp=sharing
>
> This is the link for google hang out
> video call Join video call
> <
> https://plus.google.com/hangouts/_/calendar/a2xhdXMxOTgyLmNuQGdtYWlsLmNvbQ.rgasirkilgmn1bm69kdcqjsb2s
> >The
> time is March 15th at 5pm PST
>
> Thanks,
>
> Guangya
>
> On Tue, Mar 15, 2016 at 9:47 AM, Benjamin Mahler <bm...@apache.org>
> wrote:
>
> > Sounds good, the next one is tomorrow March 15th at 5pm PST (they are at
> > 5pm PST to accommodate China time zone).
> >
> > Will that work?
> >
> > On Mon, Mar 14, 2016 at 10:53 AM, Niklas Nielsen <ni...@qni.dk> wrote:
> >
> >> Ben, when do you have your next mesos allocator sync? We don't have our
> >> next performance isolation sync lined up yet, so we could piggy back on
> >> yours if you have it scheduled already.
> >>
> >> Niklas
> >>
> >> On Mon, Mar 14, 2016 at 9:32 AM, Jie Yu <yu...@gmail.com> wrote:
> >>
> >> > >
> >> > > Just a quick note: Ian D. and the performance isolation working
> group
> >> are
> >> > > discussing similar annotations and we should meet and talk about the
> >> > > options.
> >> >
> >> >
> >> > +1
> >> >
> >> > Would love to understand the relationship between this and the
> >> > task/executor level annotations.
> >> >
> >> > - Jie
> >> >
> >> > On Mon, Mar 14, 2016 at 9:29 AM, Niklas Nielsen <ni...@qni.dk> wrote:
> >> >
> >> > > Hi Ben,
> >> > >
> >> > > Just a quick note: Ian D. and the performance isolation working
> group
> >> are
> >> > > discussing similar annotations and we should meet and talk about the
> >> > > options.
> >> > >
> >> > > Niklas
> >> > >
> >> > > On Sat, Mar 12, 2016 at 12:05 AM, Klaus Ma <kl...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Yes, I think that's true for now; so we define `ThrottleInfo` as
> >> > message
> >> > > to
> >> > > > be more flexible. In Optimistic Offer Phase 1, we only use it to
> >> > > > distinguish usage oversubscriptions and allocation
> oversubscription,
> >> > > > similar to bool :).
> >> > > >
> >> > > > Regarding the resources type, two questions after the discussion:
> >> > > >
> >> > > > 1. should we send different offer to the framework, so when
> >> > > > usage/allocation oversubscription updated, only one type of offer
> >> will
> >> > be
> >> > > > rescinded?
> >> > > > 2. should we define framework's capability against `ThrottleInfo`?
> >> > > >
> >> > > > ----
> >> > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> >> > > > Platform OpenSource Technology, STG, IBM GCG
> >> > > > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
> >> > > >
> >> > > > On Sat, Mar 12, 2016 at 12:03 PM, Guangya Liu <gyliu513@gmail.com
> >
> >> > > wrote:
> >> > > >
> >> > > > >
> >> > > > > Hi Ben,
> >> > > > >
> >> > > > > I think that currently and even in the near future, the
> >> > > __ThrottleInfo__
> >> > > > > will only be used by the usage oversubscriptions and the
> >> > > oversubscription
> >> > > > > for allocator (Both quota and reservations) will not use this
> >> value
> >> > but
> >> > > > > only using __RevocableInfo__ is enough.
> >> > > > >
> >> > > > > I can even think that the __ThrottleInfo__ as a boolean value in
> >> > > > > optimistic offer phase 1 as it is mainly used to distinguish
> >> > resources
> >> > > > > between usage oversubscriptions and allocation oversubscription
> >> > (Quota
> >> > > > and
> >> > > > > Reservations), comments?
> >> > > > >
> >> > > > > Thanks,
> >> > > > >
> >> > > > > Guangya
> >> > > > >
> >> > > > > 在 2016年3月12日星期六 UTC+8上午11:09:46，Benjamin Mahler写道：
> >> > > > >
> >> > > > >> Hey folks,
> >> > > > >>
> >> > > > >> In the resource allocation working group we've been looking
> into
> >> a
> >> > few
> >> > > > >> projects that will make the allocator able to offer out
> >> resources as
> >> > > > >> revocable. For example:
> >> > > > >>
> >> > > > >> -We'll want to eventually allocate resources as revocable _by
> >> > > default_,
> >> > > > >> only allowing non-revocable when there are guarantees put in
> >> place
> >> > > > (static
> >> > > > >> reservations or quota).
> >> > > > >>
> >> > > > >> -On the path to revocable by default, we can incrementally
> start
> >> to
> >> > > > offer
> >> > > > >> certain resources as revocable. Consider when quota is set but
> >> the
> >> > > role
> >> > > > >> isn't using all of the quota. The unallocated quota can be
> >> offered
> >> > to
> >> > > > other
> >> > > > >> roles, but it should be revocable because we may revoke them
> >> should
> >> > > the
> >> > > > >> quota'ed role want to use the resources. Unused reservations
> fall
> >> > > into a
> >> > > > >> similar category.
> >> > > > >>
> >> > > > >> -Going revocable by default also allows us to enforce fairness
> >> in a
> >> > > > >> dynamically changing cluster by revoking resources as weights
> are
> >> > > > changed,
> >> > > > >> frameworks are added or removed, etc.
> >> > > > >>
> >> > > > >> In this context, "revocable" means that the resources may be
> >> taken
> >> > > away
> >> > > > >> and the container will be destroyed. The meaning of "revocable"
> >> in
> >> > the
> >> > > > >> context of usage oversubscription includes this, but also the
> >> > > container
> >> > > > may
> >> > > > >> experience a throttling (e.g. lower cpu shares, less network
> >> > priority,
> >> > > > etc).
> >> > > > >>
> >> > > > >> For this reason, and because we internally need to distinguish
> >> > > revocable
> >> > > > >> resources between the those that are generated by usage
> >> > > oversubscription
> >> > > > >> and those that are generated by the allocator, we're thinking
> of
> >> the
> >> > > > >> following change to the API:
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >> -  message RevocableInfo {}
> >> > > > >> +  message RevocableInfo {
> >> > > > >> +    message ThrottleInfo {}
> >> > > > >> +
> >> > > > >> +    // If set, indicates that the resources may be throttled
> at
> >> > > > >> +    // any time. Throttle-able resoruces can be used for tasks
> >> > > > >> +    // that do not have strict performance requirements and
> are
> >> > > > >> +    // capable of handling being throttled.
> >> > > > >> +    optional ThrottleInfo throttle_info;
> >> > > > >> +  }
> >> > > > >>
> >> > > > >>    // If this is set, the resources are revocable, i.e., any
> >> tasks
> >> > or
> >> > > > >> -  // executors launched using these resources could get
> >> preempted
> >> > or
> >> > > > >> -  // throttled at any time. This could be used by frameworks
> to
> >> run
> >> > > > >> -  // best effort tasks that do not need strict uptime or
> >> > performance
> >> > > > >> +  // executors launched using these resources could be
> >> terminated
> >> > at
> >> > > > >> +  // any time. This could be used by frameworks to run
> >> > > > >> +  // best effort tasks that do not need strict uptime
> >> > > > >>    // guarantees. Note that if this is set, 'disk' or
> >> 'reservation'
> >> > > > >>    // cannot be set.
> >> > > > >>    optional RevocableInfo revocable = 9;
> >> > > > >>
> >> > > > >>
> >> > > > >>
> >> > > > >> Essentially we want to distinguish between revocable and
> >> revocable +
> >> > > > >> throttle-able. This is because usage-oversubscription generates
> >> > > > >> throttle-able revocable resources, whereas the allocator does
> >> not.
> >> > > This
> >> > > > >> also solves our problem of distinguishing between these two
> >> kinds of
> >> > > > >> revocable resources internally.
> >> > > > >>
> >> > > > >> Feedback welcome!
> >> > > > >>
> >> > > > >> Ben
> >> > > > >>
> >> > > > >> --
> >> > > > > You received this message because you are subscribed to the
> Google
> >> > > Groups
> >> > > > > "Mesos Resource Allocation Working Group" group.
> >> > > > > To unsubscribe from this group and stop receiving emails from
> it,
> >> > send
> >> > > an
> >> > > > > email to mesos-allocation+unsubscribe@googlegroups.com.
> >> > > > > To post to this group, send email to
> >> > mesos-allocation@googlegroups.com
> >> > > .
> >> > > > > To view this discussion on the web visit
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com
> >> > > > > <
> >> > > >
> >> > >
> >> >
> >>
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com?utm_medium=email&utm_source=footer
> >> > > > >
> >> > > > > .
> >> > > > > For more options, visit https://groups.google.com/d/optout.
> >> > > > >
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Niklas
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Niklas
> >>
> >
> >
>

Re: RFC: RevocableInfo Changes

Posted by Guangya Liu <gy...@gmail.com>.

This is the google doc that we used to trace all discussion topics:
https://docs.google.com/document/d/1B_v52zCOFcwCpqCPhgYi9h630a0NE-QM9Br0nCOZUR4/edit?usp=sharing

This is the link for google hang out
video call Join video call
<https://plus.google.com/hangouts/_/calendar/a2xhdXMxOTgyLmNuQGdtYWlsLmNvbQ.rgasirkilgmn1bm69kdcqjsb2s>The
time is March 15th at 5pm PST

Thanks,

Guangya

On Tue, Mar 15, 2016 at 9:47 AM, Benjamin Mahler <bm...@apache.org> wrote:

> Sounds good, the next one is tomorrow March 15th at 5pm PST (they are at
> 5pm PST to accommodate China time zone).
>
> Will that work?
>
> On Mon, Mar 14, 2016 at 10:53 AM, Niklas Nielsen <ni...@qni.dk> wrote:
>
>> Ben, when do you have your next mesos allocator sync? We don't have our
>> next performance isolation sync lined up yet, so we could piggy back on
>> yours if you have it scheduled already.
>>
>> Niklas
>>
>> On Mon, Mar 14, 2016 at 9:32 AM, Jie Yu <yu...@gmail.com> wrote:
>>
>> > >
>> > > Just a quick note: Ian D. and the performance isolation working group
>> are
>> > > discussing similar annotations and we should meet and talk about the
>> > > options.
>> >
>> >
>> > +1
>> >
>> > Would love to understand the relationship between this and the
>> > task/executor level annotations.
>> >
>> > - Jie
>> >
>> > On Mon, Mar 14, 2016 at 9:29 AM, Niklas Nielsen <ni...@qni.dk> wrote:
>> >
>> > > Hi Ben,
>> > >
>> > > Just a quick note: Ian D. and the performance isolation working group
>> are
>> > > discussing similar annotations and we should meet and talk about the
>> > > options.
>> > >
>> > > Niklas
>> > >
>> > > On Sat, Mar 12, 2016 at 12:05 AM, Klaus Ma <kl...@gmail.com>
>> > wrote:
>> > >
>> > > > Yes, I think that's true for now; so we define `ThrottleInfo` as
>> > message
>> > > to
>> > > > be more flexible. In Optimistic Offer Phase 1, we only use it to
>> > > > distinguish usage oversubscriptions and allocation oversubscription,
>> > > > similar to bool :).
>> > > >
>> > > > Regarding the resources type, two questions after the discussion:
>> > > >
>> > > > 1. should we send different offer to the framework, so when
>> > > > usage/allocation oversubscription updated, only one type of offer
>> will
>> > be
>> > > > rescinded?
>> > > > 2. should we define framework's capability against `ThrottleInfo`?
>> > > >
>> > > > ----
>> > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
>> > > > Platform OpenSource Technology, STG, IBM GCG
>> > > > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>> > > >
>> > > > On Sat, Mar 12, 2016 at 12:03 PM, Guangya Liu <gy...@gmail.com>
>> > > wrote:
>> > > >
>> > > > >
>> > > > > Hi Ben,
>> > > > >
>> > > > > I think that currently and even in the near future, the
>> > > __ThrottleInfo__
>> > > > > will only be used by the usage oversubscriptions and the
>> > > oversubscription
>> > > > > for allocator (Both quota and reservations) will not use this
>> value
>> > but
>> > > > > only using __RevocableInfo__ is enough.
>> > > > >
>> > > > > I can even think that the __ThrottleInfo__ as a boolean value in
>> > > > > optimistic offer phase 1 as it is mainly used to distinguish
>> > resources
>> > > > > between usage oversubscriptions and allocation oversubscription
>> > (Quota
>> > > > and
>> > > > > Reservations), comments?
>> > > > >
>> > > > > Thanks,
>> > > > >
>> > > > > Guangya
>> > > > >
>> > > > > 在 2016年3月12日星期六 UTC+8上午11:09:46，Benjamin Mahler写道：
>> > > > >
>> > > > >> Hey folks,
>> > > > >>
>> > > > >> In the resource allocation working group we've been looking into
>> a
>> > few
>> > > > >> projects that will make the allocator able to offer out
>> resources as
>> > > > >> revocable. For example:
>> > > > >>
>> > > > >> -We'll want to eventually allocate resources as revocable _by
>> > > default_,
>> > > > >> only allowing non-revocable when there are guarantees put in
>> place
>> > > > (static
>> > > > >> reservations or quota).
>> > > > >>
>> > > > >> -On the path to revocable by default, we can incrementally start
>> to
>> > > > offer
>> > > > >> certain resources as revocable. Consider when quota is set but
>> the
>> > > role
>> > > > >> isn't using all of the quota. The unallocated quota can be
>> offered
>> > to
>> > > > other
>> > > > >> roles, but it should be revocable because we may revoke them
>> should
>> > > the
>> > > > >> quota'ed role want to use the resources. Unused reservations fall
>> > > into a
>> > > > >> similar category.
>> > > > >>
>> > > > >> -Going revocable by default also allows us to enforce fairness
>> in a
>> > > > >> dynamically changing cluster by revoking resources as weights are
>> > > > changed,
>> > > > >> frameworks are added or removed, etc.
>> > > > >>
>> > > > >> In this context, "revocable" means that the resources may be
>> taken
>> > > away
>> > > > >> and the container will be destroyed. The meaning of "revocable"
>> in
>> > the
>> > > > >> context of usage oversubscription includes this, but also the
>> > > container
>> > > > may
>> > > > >> experience a throttling (e.g. lower cpu shares, less network
>> > priority,
>> > > > etc).
>> > > > >>
>> > > > >> For this reason, and because we internally need to distinguish
>> > > revocable
>> > > > >> resources between the those that are generated by usage
>> > > oversubscription
>> > > > >> and those that are generated by the allocator, we're thinking of
>> the
>> > > > >> following change to the API:
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >> -  message RevocableInfo {}
>> > > > >> +  message RevocableInfo {
>> > > > >> +    message ThrottleInfo {}
>> > > > >> +
>> > > > >> +    // If set, indicates that the resources may be throttled at
>> > > > >> +    // any time. Throttle-able resoruces can be used for tasks
>> > > > >> +    // that do not have strict performance requirements and are
>> > > > >> +    // capable of handling being throttled.
>> > > > >> +    optional ThrottleInfo throttle_info;
>> > > > >> +  }
>> > > > >>
>> > > > >>    // If this is set, the resources are revocable, i.e., any
>> tasks
>> > or
>> > > > >> -  // executors launched using these resources could get
>> preempted
>> > or
>> > > > >> -  // throttled at any time. This could be used by frameworks to
>> run
>> > > > >> -  // best effort tasks that do not need strict uptime or
>> > performance
>> > > > >> +  // executors launched using these resources could be
>> terminated
>> > at
>> > > > >> +  // any time. This could be used by frameworks to run
>> > > > >> +  // best effort tasks that do not need strict uptime
>> > > > >>    // guarantees. Note that if this is set, 'disk' or
>> 'reservation'
>> > > > >>    // cannot be set.
>> > > > >>    optional RevocableInfo revocable = 9;
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >> Essentially we want to distinguish between revocable and
>> revocable +
>> > > > >> throttle-able. This is because usage-oversubscription generates
>> > > > >> throttle-able revocable resources, whereas the allocator does
>> not.
>> > > This
>> > > > >> also solves our problem of distinguishing between these two
>> kinds of
>> > > > >> revocable resources internally.
>> > > > >>
>> > > > >> Feedback welcome!
>> > > > >>
>> > > > >> Ben
>> > > > >>
>> > > > >> --
>> > > > > You received this message because you are subscribed to the Google
>> > > Groups
>> > > > > "Mesos Resource Allocation Working Group" group.
>> > > > > To unsubscribe from this group and stop receiving emails from it,
>> > send
>> > > an
>> > > > > email to mesos-allocation+unsubscribe@googlegroups.com.
>> > > > > To post to this group, send email to
>> > mesos-allocation@googlegroups.com
>> > > .
>> > > > > To view this discussion on the web visit
>> > > > >
>> > > >
>> > >
>> >
>> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com
>> > > > > <
>> > > >
>> > >
>> >
>> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com?utm_medium=email&utm_source=footer
>> > > > >
>> > > > > .
>> > > > > For more options, visit https://groups.google.com/d/optout.
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Niklas
>> > >
>> >
>>
>>
>>
>> --
>> Niklas
>>
>
>

Re: RFC: RevocableInfo Changes

Posted by Benjamin Mahler <bm...@apache.org>.

Sounds good, the next one is tomorrow March 15th at 5pm PST (they are at
5pm PST to accommodate China time zone).

Will that work?

On Mon, Mar 14, 2016 at 10:53 AM, Niklas Nielsen <ni...@qni.dk> wrote:

> Ben, when do you have your next mesos allocator sync? We don't have our
> next performance isolation sync lined up yet, so we could piggy back on
> yours if you have it scheduled already.
>
> Niklas
>
> On Mon, Mar 14, 2016 at 9:32 AM, Jie Yu <yu...@gmail.com> wrote:
>
> > >
> > > Just a quick note: Ian D. and the performance isolation working group
> are
> > > discussing similar annotations and we should meet and talk about the
> > > options.
> >
> >
> > +1
> >
> > Would love to understand the relationship between this and the
> > task/executor level annotations.
> >
> > - Jie
> >
> > On Mon, Mar 14, 2016 at 9:29 AM, Niklas Nielsen <ni...@qni.dk> wrote:
> >
> > > Hi Ben,
> > >
> > > Just a quick note: Ian D. and the performance isolation working group
> are
> > > discussing similar annotations and we should meet and talk about the
> > > options.
> > >
> > > Niklas
> > >
> > > On Sat, Mar 12, 2016 at 12:05 AM, Klaus Ma <kl...@gmail.com>
> > wrote:
> > >
> > > > Yes, I think that's true for now; so we define `ThrottleInfo` as
> > message
> > > to
> > > > be more flexible. In Optimistic Offer Phase 1, we only use it to
> > > > distinguish usage oversubscriptions and allocation oversubscription,
> > > > similar to bool :).
> > > >
> > > > Regarding the resources type, two questions after the discussion:
> > > >
> > > > 1. should we send different offer to the framework, so when
> > > > usage/allocation oversubscription updated, only one type of offer
> will
> > be
> > > > rescinded?
> > > > 2. should we define framework's capability against `ThrottleInfo`?
> > > >
> > > > ----
> > > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > > Platform OpenSource Technology, STG, IBM GCG
> > > > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
> > > >
> > > > On Sat, Mar 12, 2016 at 12:03 PM, Guangya Liu <gy...@gmail.com>
> > > wrote:
> > > >
> > > > >
> > > > > Hi Ben,
> > > > >
> > > > > I think that currently and even in the near future, the
> > > __ThrottleInfo__
> > > > > will only be used by the usage oversubscriptions and the
> > > oversubscription
> > > > > for allocator (Both quota and reservations) will not use this value
> > but
> > > > > only using __RevocableInfo__ is enough.
> > > > >
> > > > > I can even think that the __ThrottleInfo__ as a boolean value in
> > > > > optimistic offer phase 1 as it is mainly used to distinguish
> > resources
> > > > > between usage oversubscriptions and allocation oversubscription
> > (Quota
> > > > and
> > > > > Reservations), comments?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Guangya
> > > > >
> > > > > 在 2016年3月12日星期六 UTC+8上午11:09:46，Benjamin Mahler写道：
> > > > >
> > > > >> Hey folks,
> > > > >>
> > > > >> In the resource allocation working group we've been looking into a
> > few
> > > > >> projects that will make the allocator able to offer out resources
> as
> > > > >> revocable. For example:
> > > > >>
> > > > >> -We'll want to eventually allocate resources as revocable _by
> > > default_,
> > > > >> only allowing non-revocable when there are guarantees put in place
> > > > (static
> > > > >> reservations or quota).
> > > > >>
> > > > >> -On the path to revocable by default, we can incrementally start
> to
> > > > offer
> > > > >> certain resources as revocable. Consider when quota is set but the
> > > role
> > > > >> isn't using all of the quota. The unallocated quota can be offered
> > to
> > > > other
> > > > >> roles, but it should be revocable because we may revoke them
> should
> > > the
> > > > >> quota'ed role want to use the resources. Unused reservations fall
> > > into a
> > > > >> similar category.
> > > > >>
> > > > >> -Going revocable by default also allows us to enforce fairness in
> a
> > > > >> dynamically changing cluster by revoking resources as weights are
> > > > changed,
> > > > >> frameworks are added or removed, etc.
> > > > >>
> > > > >> In this context, "revocable" means that the resources may be taken
> > > away
> > > > >> and the container will be destroyed. The meaning of "revocable" in
> > the
> > > > >> context of usage oversubscription includes this, but also the
> > > container
> > > > may
> > > > >> experience a throttling (e.g. lower cpu shares, less network
> > priority,
> > > > etc).
> > > > >>
> > > > >> For this reason, and because we internally need to distinguish
> > > revocable
> > > > >> resources between the those that are generated by usage
> > > oversubscription
> > > > >> and those that are generated by the allocator, we're thinking of
> the
> > > > >> following change to the API:
> > > > >>
> > > > >>
> > > > >>
> > > > >> -  message RevocableInfo {}
> > > > >> +  message RevocableInfo {
> > > > >> +    message ThrottleInfo {}
> > > > >> +
> > > > >> +    // If set, indicates that the resources may be throttled at
> > > > >> +    // any time. Throttle-able resoruces can be used for tasks
> > > > >> +    // that do not have strict performance requirements and are
> > > > >> +    // capable of handling being throttled.
> > > > >> +    optional ThrottleInfo throttle_info;
> > > > >> +  }
> > > > >>
> > > > >>    // If this is set, the resources are revocable, i.e., any tasks
> > or
> > > > >> -  // executors launched using these resources could get preempted
> > or
> > > > >> -  // throttled at any time. This could be used by frameworks to
> run
> > > > >> -  // best effort tasks that do not need strict uptime or
> > performance
> > > > >> +  // executors launched using these resources could be terminated
> > at
> > > > >> +  // any time. This could be used by frameworks to run
> > > > >> +  // best effort tasks that do not need strict uptime
> > > > >>    // guarantees. Note that if this is set, 'disk' or
> 'reservation'
> > > > >>    // cannot be set.
> > > > >>    optional RevocableInfo revocable = 9;
> > > > >>
> > > > >>
> > > > >>
> > > > >> Essentially we want to distinguish between revocable and
> revocable +
> > > > >> throttle-able. This is because usage-oversubscription generates
> > > > >> throttle-able revocable resources, whereas the allocator does not.
> > > This
> > > > >> also solves our problem of distinguishing between these two kinds
> of
> > > > >> revocable resources internally.
> > > > >>
> > > > >> Feedback welcome!
> > > > >>
> > > > >> Ben
> > > > >>
> > > > >> --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "Mesos Resource Allocation Working Group" group.
> > > > > To unsubscribe from this group and stop receiving emails from it,
> > send
> > > an
> > > > > email to mesos-allocation+unsubscribe@googlegroups.com.
> > > > > To post to this group, send email to
> > mesos-allocation@googlegroups.com
> > > .
> > > > > To view this discussion on the web visit
> > > > >
> > > >
> > >
> >
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com
> > > > > <
> > > >
> > >
> >
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com?utm_medium=email&utm_source=footer
> > > > >
> > > > > .
> > > > > For more options, visit https://groups.google.com/d/optout.
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Niklas
> > >
> >
>
>
>
> --
> Niklas
>

Re: RFC: RevocableInfo Changes

Posted by Niklas Nielsen <ni...@qni.dk>.

Ben, when do you have your next mesos allocator sync? We don't have our
next performance isolation sync lined up yet, so we could piggy back on
yours if you have it scheduled already.

Niklas

On Mon, Mar 14, 2016 at 9:32 AM, Jie Yu <yu...@gmail.com> wrote:

> >
> > Just a quick note: Ian D. and the performance isolation working group are
> > discussing similar annotations and we should meet and talk about the
> > options.
>
>
> +1
>
> Would love to understand the relationship between this and the
> task/executor level annotations.
>
> - Jie
>
> On Mon, Mar 14, 2016 at 9:29 AM, Niklas Nielsen <ni...@qni.dk> wrote:
>
> > Hi Ben,
> >
> > Just a quick note: Ian D. and the performance isolation working group are
> > discussing similar annotations and we should meet and talk about the
> > options.
> >
> > Niklas
> >
> > On Sat, Mar 12, 2016 at 12:05 AM, Klaus Ma <kl...@gmail.com>
> wrote:
> >
> > > Yes, I think that's true for now; so we define `ThrottleInfo` as
> message
> > to
> > > be more flexible. In Optimistic Offer Phase 1, we only use it to
> > > distinguish usage oversubscriptions and allocation oversubscription,
> > > similar to bool :).
> > >
> > > Regarding the resources type, two questions after the discussion:
> > >
> > > 1. should we send different offer to the framework, so when
> > > usage/allocation oversubscription updated, only one type of offer will
> be
> > > rescinded?
> > > 2. should we define framework's capability against `ThrottleInfo`?
> > >
> > > ----
> > > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > > Platform OpenSource Technology, STG, IBM GCG
> > > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
> > >
> > > On Sat, Mar 12, 2016 at 12:03 PM, Guangya Liu <gy...@gmail.com>
> > wrote:
> > >
> > > >
> > > > Hi Ben,
> > > >
> > > > I think that currently and even in the near future, the
> > __ThrottleInfo__
> > > > will only be used by the usage oversubscriptions and the
> > oversubscription
> > > > for allocator (Both quota and reservations) will not use this value
> but
> > > > only using __RevocableInfo__ is enough.
> > > >
> > > > I can even think that the __ThrottleInfo__ as a boolean value in
> > > > optimistic offer phase 1 as it is mainly used to distinguish
> resources
> > > > between usage oversubscriptions and allocation oversubscription
> (Quota
> > > and
> > > > Reservations), comments?
> > > >
> > > > Thanks,
> > > >
> > > > Guangya
> > > >
> > > > 在 2016年3月12日星期六 UTC+8上午11:09:46，Benjamin Mahler写道：
> > > >
> > > >> Hey folks,
> > > >>
> > > >> In the resource allocation working group we've been looking into a
> few
> > > >> projects that will make the allocator able to offer out resources as
> > > >> revocable. For example:
> > > >>
> > > >> -We'll want to eventually allocate resources as revocable _by
> > default_,
> > > >> only allowing non-revocable when there are guarantees put in place
> > > (static
> > > >> reservations or quota).
> > > >>
> > > >> -On the path to revocable by default, we can incrementally start to
> > > offer
> > > >> certain resources as revocable. Consider when quota is set but the
> > role
> > > >> isn't using all of the quota. The unallocated quota can be offered
> to
> > > other
> > > >> roles, but it should be revocable because we may revoke them should
> > the
> > > >> quota'ed role want to use the resources. Unused reservations fall
> > into a
> > > >> similar category.
> > > >>
> > > >> -Going revocable by default also allows us to enforce fairness in a
> > > >> dynamically changing cluster by revoking resources as weights are
> > > changed,
> > > >> frameworks are added or removed, etc.
> > > >>
> > > >> In this context, "revocable" means that the resources may be taken
> > away
> > > >> and the container will be destroyed. The meaning of "revocable" in
> the
> > > >> context of usage oversubscription includes this, but also the
> > container
> > > may
> > > >> experience a throttling (e.g. lower cpu shares, less network
> priority,
> > > etc).
> > > >>
> > > >> For this reason, and because we internally need to distinguish
> > revocable
> > > >> resources between the those that are generated by usage
> > oversubscription
> > > >> and those that are generated by the allocator, we're thinking of the
> > > >> following change to the API:
> > > >>
> > > >>
> > > >>
> > > >> -  message RevocableInfo {}
> > > >> +  message RevocableInfo {
> > > >> +    message ThrottleInfo {}
> > > >> +
> > > >> +    // If set, indicates that the resources may be throttled at
> > > >> +    // any time. Throttle-able resoruces can be used for tasks
> > > >> +    // that do not have strict performance requirements and are
> > > >> +    // capable of handling being throttled.
> > > >> +    optional ThrottleInfo throttle_info;
> > > >> +  }
> > > >>
> > > >>    // If this is set, the resources are revocable, i.e., any tasks
> or
> > > >> -  // executors launched using these resources could get preempted
> or
> > > >> -  // throttled at any time. This could be used by frameworks to run
> > > >> -  // best effort tasks that do not need strict uptime or
> performance
> > > >> +  // executors launched using these resources could be terminated
> at
> > > >> +  // any time. This could be used by frameworks to run
> > > >> +  // best effort tasks that do not need strict uptime
> > > >>    // guarantees. Note that if this is set, 'disk' or 'reservation'
> > > >>    // cannot be set.
> > > >>    optional RevocableInfo revocable = 9;
> > > >>
> > > >>
> > > >>
> > > >> Essentially we want to distinguish between revocable and revocable +
> > > >> throttle-able. This is because usage-oversubscription generates
> > > >> throttle-able revocable resources, whereas the allocator does not.
> > This
> > > >> also solves our problem of distinguishing between these two kinds of
> > > >> revocable resources internally.
> > > >>
> > > >> Feedback welcome!
> > > >>
> > > >> Ben
> > > >>
> > > >> --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Mesos Resource Allocation Working Group" group.
> > > > To unsubscribe from this group and stop receiving emails from it,
> send
> > an
> > > > email to mesos-allocation+unsubscribe@googlegroups.com.
> > > > To post to this group, send email to
> mesos-allocation@googlegroups.com
> > .
> > > > To view this discussion on the web visit
> > > >
> > >
> >
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com
> > > > <
> > >
> >
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com?utm_medium=email&utm_source=footer
> > > >
> > > > .
> > > > For more options, visit https://groups.google.com/d/optout.
> > > >
> > >
> >
> >
> >
> > --
> > Niklas
> >
>



-- 
Niklas

Re: RFC: RevocableInfo Changes

Posted by Jie Yu <yu...@gmail.com>.

>
> Just a quick note: Ian D. and the performance isolation working group are
> discussing similar annotations and we should meet and talk about the
> options.


+1

Would love to understand the relationship between this and the
task/executor level annotations.

- Jie

On Mon, Mar 14, 2016 at 9:29 AM, Niklas Nielsen <ni...@qni.dk> wrote:

> Hi Ben,
>
> Just a quick note: Ian D. and the performance isolation working group are
> discussing similar annotations and we should meet and talk about the
> options.
>
> Niklas
>
> On Sat, Mar 12, 2016 at 12:05 AM, Klaus Ma <kl...@gmail.com> wrote:
>
> > Yes, I think that's true for now; so we define `ThrottleInfo` as message
> to
> > be more flexible. In Optimistic Offer Phase 1, we only use it to
> > distinguish usage oversubscriptions and allocation oversubscription,
> > similar to bool :).
> >
> > Regarding the resources type, two questions after the discussion:
> >
> > 1. should we send different offer to the framework, so when
> > usage/allocation oversubscription updated, only one type of offer will be
> > rescinded?
> > 2. should we define framework's capability against `ThrottleInfo`?
> >
> > ----
> > Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> > Platform OpenSource Technology, STG, IBM GCG
> > +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
> >
> > On Sat, Mar 12, 2016 at 12:03 PM, Guangya Liu <gy...@gmail.com>
> wrote:
> >
> > >
> > > Hi Ben,
> > >
> > > I think that currently and even in the near future, the
> __ThrottleInfo__
> > > will only be used by the usage oversubscriptions and the
> oversubscription
> > > for allocator (Both quota and reservations) will not use this value but
> > > only using __RevocableInfo__ is enough.
> > >
> > > I can even think that the __ThrottleInfo__ as a boolean value in
> > > optimistic offer phase 1 as it is mainly used to distinguish resources
> > > between usage oversubscriptions and allocation oversubscription (Quota
> > and
> > > Reservations), comments?
> > >
> > > Thanks,
> > >
> > > Guangya
> > >
> > > 在 2016年3月12日星期六 UTC+8上午11:09:46，Benjamin Mahler写道：
> > >
> > >> Hey folks,
> > >>
> > >> In the resource allocation working group we've been looking into a few
> > >> projects that will make the allocator able to offer out resources as
> > >> revocable. For example:
> > >>
> > >> -We'll want to eventually allocate resources as revocable _by
> default_,
> > >> only allowing non-revocable when there are guarantees put in place
> > (static
> > >> reservations or quota).
> > >>
> > >> -On the path to revocable by default, we can incrementally start to
> > offer
> > >> certain resources as revocable. Consider when quota is set but the
> role
> > >> isn't using all of the quota. The unallocated quota can be offered to
> > other
> > >> roles, but it should be revocable because we may revoke them should
> the
> > >> quota'ed role want to use the resources. Unused reservations fall
> into a
> > >> similar category.
> > >>
> > >> -Going revocable by default also allows us to enforce fairness in a
> > >> dynamically changing cluster by revoking resources as weights are
> > changed,
> > >> frameworks are added or removed, etc.
> > >>
> > >> In this context, "revocable" means that the resources may be taken
> away
> > >> and the container will be destroyed. The meaning of "revocable" in the
> > >> context of usage oversubscription includes this, but also the
> container
> > may
> > >> experience a throttling (e.g. lower cpu shares, less network priority,
> > etc).
> > >>
> > >> For this reason, and because we internally need to distinguish
> revocable
> > >> resources between the those that are generated by usage
> oversubscription
> > >> and those that are generated by the allocator, we're thinking of the
> > >> following change to the API:
> > >>
> > >>
> > >>
> > >> -  message RevocableInfo {}
> > >> +  message RevocableInfo {
> > >> +    message ThrottleInfo {}
> > >> +
> > >> +    // If set, indicates that the resources may be throttled at
> > >> +    // any time. Throttle-able resoruces can be used for tasks
> > >> +    // that do not have strict performance requirements and are
> > >> +    // capable of handling being throttled.
> > >> +    optional ThrottleInfo throttle_info;
> > >> +  }
> > >>
> > >>    // If this is set, the resources are revocable, i.e., any tasks or
> > >> -  // executors launched using these resources could get preempted or
> > >> -  // throttled at any time. This could be used by frameworks to run
> > >> -  // best effort tasks that do not need strict uptime or performance
> > >> +  // executors launched using these resources could be terminated at
> > >> +  // any time. This could be used by frameworks to run
> > >> +  // best effort tasks that do not need strict uptime
> > >>    // guarantees. Note that if this is set, 'disk' or 'reservation'
> > >>    // cannot be set.
> > >>    optional RevocableInfo revocable = 9;
> > >>
> > >>
> > >>
> > >> Essentially we want to distinguish between revocable and revocable +
> > >> throttle-able. This is because usage-oversubscription generates
> > >> throttle-able revocable resources, whereas the allocator does not.
> This
> > >> also solves our problem of distinguishing between these two kinds of
> > >> revocable resources internally.
> > >>
> > >> Feedback welcome!
> > >>
> > >> Ben
> > >>
> > >> --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Mesos Resource Allocation Working Group" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> an
> > > email to mesos-allocation+unsubscribe@googlegroups.com.
> > > To post to this group, send email to mesos-allocation@googlegroups.com
> .
> > > To view this discussion on the web visit
> > >
> >
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com
> > > <
> >
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com?utm_medium=email&utm_source=footer
> > >
> > > .
> > > For more options, visit https://groups.google.com/d/optout.
> > >
> >
>
>
>
> --
> Niklas
>

Re: RFC: RevocableInfo Changes

Posted by Niklas Nielsen <ni...@qni.dk>.

Hi Ben,

Just a quick note: Ian D. and the performance isolation working group are
discussing similar annotations and we should meet and talk about the
options.

Niklas

On Sat, Mar 12, 2016 at 12:05 AM, Klaus Ma <kl...@gmail.com> wrote:

> Yes, I think that's true for now; so we define `ThrottleInfo` as message to
> be more flexible. In Optimistic Offer Phase 1, we only use it to
> distinguish usage oversubscriptions and allocation oversubscription,
> similar to bool :).
>
> Regarding the resources type, two questions after the discussion:
>
> 1. should we send different offer to the framework, so when
> usage/allocation oversubscription updated, only one type of offer will be
> rescinded?
> 2. should we define framework's capability against `ThrottleInfo`?
>
> ----
> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> Platform OpenSource Technology, STG, IBM GCG
> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>
> On Sat, Mar 12, 2016 at 12:03 PM, Guangya Liu <gy...@gmail.com> wrote:
>
> >
> > Hi Ben,
> >
> > I think that currently and even in the near future, the __ThrottleInfo__
> > will only be used by the usage oversubscriptions and the oversubscription
> > for allocator (Both quota and reservations) will not use this value but
> > only using __RevocableInfo__ is enough.
> >
> > I can even think that the __ThrottleInfo__ as a boolean value in
> > optimistic offer phase 1 as it is mainly used to distinguish resources
> > between usage oversubscriptions and allocation oversubscription (Quota
> and
> > Reservations), comments?
> >
> > Thanks,
> >
> > Guangya
> >
> > 在 2016年3月12日星期六 UTC+8上午11:09:46，Benjamin Mahler写道：
> >
> >> Hey folks,
> >>
> >> In the resource allocation working group we've been looking into a few
> >> projects that will make the allocator able to offer out resources as
> >> revocable. For example:
> >>
> >> -We'll want to eventually allocate resources as revocable _by default_,
> >> only allowing non-revocable when there are guarantees put in place
> (static
> >> reservations or quota).
> >>
> >> -On the path to revocable by default, we can incrementally start to
> offer
> >> certain resources as revocable. Consider when quota is set but the role
> >> isn't using all of the quota. The unallocated quota can be offered to
> other
> >> roles, but it should be revocable because we may revoke them should the
> >> quota'ed role want to use the resources. Unused reservations fall into a
> >> similar category.
> >>
> >> -Going revocable by default also allows us to enforce fairness in a
> >> dynamically changing cluster by revoking resources as weights are
> changed,
> >> frameworks are added or removed, etc.
> >>
> >> In this context, "revocable" means that the resources may be taken away
> >> and the container will be destroyed. The meaning of "revocable" in the
> >> context of usage oversubscription includes this, but also the container
> may
> >> experience a throttling (e.g. lower cpu shares, less network priority,
> etc).
> >>
> >> For this reason, and because we internally need to distinguish revocable
> >> resources between the those that are generated by usage oversubscription
> >> and those that are generated by the allocator, we're thinking of the
> >> following change to the API:
> >>
> >>
> >>
> >> -  message RevocableInfo {}
> >> +  message RevocableInfo {
> >> +    message ThrottleInfo {}
> >> +
> >> +    // If set, indicates that the resources may be throttled at
> >> +    // any time. Throttle-able resoruces can be used for tasks
> >> +    // that do not have strict performance requirements and are
> >> +    // capable of handling being throttled.
> >> +    optional ThrottleInfo throttle_info;
> >> +  }
> >>
> >>    // If this is set, the resources are revocable, i.e., any tasks or
> >> -  // executors launched using these resources could get preempted or
> >> -  // throttled at any time. This could be used by frameworks to run
> >> -  // best effort tasks that do not need strict uptime or performance
> >> +  // executors launched using these resources could be terminated at
> >> +  // any time. This could be used by frameworks to run
> >> +  // best effort tasks that do not need strict uptime
> >>    // guarantees. Note that if this is set, 'disk' or 'reservation'
> >>    // cannot be set.
> >>    optional RevocableInfo revocable = 9;
> >>
> >>
> >>
> >> Essentially we want to distinguish between revocable and revocable +
> >> throttle-able. This is because usage-oversubscription generates
> >> throttle-able revocable resources, whereas the allocator does not. This
> >> also solves our problem of distinguishing between these two kinds of
> >> revocable resources internally.
> >>
> >> Feedback welcome!
> >>
> >> Ben
> >>
> >> --
> > You received this message because you are subscribed to the Google Groups
> > "Mesos Resource Allocation Working Group" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to mesos-allocation+unsubscribe@googlegroups.com.
> > To post to this group, send email to mesos-allocation@googlegroups.com.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com
> > <
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com?utm_medium=email&utm_source=footer
> >
> > .
> > For more options, visit https://groups.google.com/d/optout.
> >
>



-- 
Niklas

Re: RFC: RevocableInfo Changes

Posted by Klaus Ma <kl...@gmail.com>.

Yes, I think that's true for now; so we define `ThrottleInfo` as message to
be more flexible. In Optimistic Offer Phase 1, we only use it to
distinguish usage oversubscriptions and allocation oversubscription,
similar to bool :).

Regarding the resources type, two questions after the discussion:

1. should we send different offer to the framework, so when
usage/allocation oversubscription updated, only one type of offer will be
rescinded?
2. should we define framework's capability against `ThrottleInfo`?

----
Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
Platform OpenSource Technology, STG, IBM GCG
+86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me

On Sat, Mar 12, 2016 at 12:03 PM, Guangya Liu <gy...@gmail.com> wrote:

>
> Hi Ben,
>
> I think that currently and even in the near future, the __ThrottleInfo__
> will only be used by the usage oversubscriptions and the oversubscription
> for allocator (Both quota and reservations) will not use this value but
> only using __RevocableInfo__ is enough.
>
> I can even think that the __ThrottleInfo__ as a boolean value in
> optimistic offer phase 1 as it is mainly used to distinguish resources
> between usage oversubscriptions and allocation oversubscription (Quota and
> Reservations), comments?
>
> Thanks,
>
> Guangya
>
> 在 2016年3月12日星期六 UTC+8上午11:09:46，Benjamin Mahler写道：
>
>> Hey folks,
>>
>> In the resource allocation working group we've been looking into a few
>> projects that will make the allocator able to offer out resources as
>> revocable. For example:
>>
>> -We'll want to eventually allocate resources as revocable _by default_,
>> only allowing non-revocable when there are guarantees put in place (static
>> reservations or quota).
>>
>> -On the path to revocable by default, we can incrementally start to offer
>> certain resources as revocable. Consider when quota is set but the role
>> isn't using all of the quota. The unallocated quota can be offered to other
>> roles, but it should be revocable because we may revoke them should the
>> quota'ed role want to use the resources. Unused reservations fall into a
>> similar category.
>>
>> -Going revocable by default also allows us to enforce fairness in a
>> dynamically changing cluster by revoking resources as weights are changed,
>> frameworks are added or removed, etc.
>>
>> In this context, "revocable" means that the resources may be taken away
>> and the container will be destroyed. The meaning of "revocable" in the
>> context of usage oversubscription includes this, but also the container may
>> experience a throttling (e.g. lower cpu shares, less network priority, etc).
>>
>> For this reason, and because we internally need to distinguish revocable
>> resources between the those that are generated by usage oversubscription
>> and those that are generated by the allocator, we're thinking of the
>> following change to the API:
>>
>>
>>
>> -  message RevocableInfo {}
>> +  message RevocableInfo {
>> +    message ThrottleInfo {}
>> +
>> +    // If set, indicates that the resources may be throttled at
>> +    // any time. Throttle-able resoruces can be used for tasks
>> +    // that do not have strict performance requirements and are
>> +    // capable of handling being throttled.
>> +    optional ThrottleInfo throttle_info;
>> +  }
>>
>>    // If this is set, the resources are revocable, i.e., any tasks or
>> -  // executors launched using these resources could get preempted or
>> -  // throttled at any time. This could be used by frameworks to run
>> -  // best effort tasks that do not need strict uptime or performance
>> +  // executors launched using these resources could be terminated at
>> +  // any time. This could be used by frameworks to run
>> +  // best effort tasks that do not need strict uptime
>>    // guarantees. Note that if this is set, 'disk' or 'reservation'
>>    // cannot be set.
>>    optional RevocableInfo revocable = 9;
>>
>>
>>
>> Essentially we want to distinguish between revocable and revocable +
>> throttle-able. This is because usage-oversubscription generates
>> throttle-able revocable resources, whereas the allocator does not. This
>> also solves our problem of distinguishing between these two kinds of
>> revocable resources internally.
>>
>> Feedback welcome!
>>
>> Ben
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Mesos Resource Allocation Working Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mesos-allocation+unsubscribe@googlegroups.com.
> To post to this group, send email to mesos-allocation@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com
> <https://groups.google.com/d/msgid/mesos-allocation/a68b9e92-f22e-4cf7-9499-b982c9c07613%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

Re: RFC: RevocableInfo Changes

Posted by Guangya Liu <gy...@gmail.com>.

Hi Ben,

I think that currently and even in the near future, the __ThrottleInfo__ 
will only be used by the usage oversubscriptions and the oversubscription 
for allocator (Both quota and reservations) will not use this value but 
only using __RevocableInfo__ is enough.

I can even think that the __ThrottleInfo__ as a boolean value in optimistic 
offer phase 1 as it is mainly used to distinguish resources between usage 
oversubscriptions and allocation oversubscription (Quota and Reservations), 
comments?

Thanks,

Guangya

在 2016年3月12日星期六 UTC+8上午11:09:46，Benjamin Mahler写道：
>
> Hey folks,
>
> In the resource allocation working group we've been looking into a few 
> projects that will make the allocator able to offer out resources as 
> revocable. For example:
>
> -We'll want to eventually allocate resources as revocable _by default_, 
> only allowing non-revocable when there are guarantees put in place (static 
> reservations or quota).
>
> -On the path to revocable by default, we can incrementally start to offer 
> certain resources as revocable. Consider when quota is set but the role 
> isn't using all of the quota. The unallocated quota can be offered to other 
> roles, but it should be revocable because we may revoke them should the 
> quota'ed role want to use the resources. Unused reservations fall into a 
> similar category.
>
> -Going revocable by default also allows us to enforce fairness in a 
> dynamically changing cluster by revoking resources as weights are changed, 
> frameworks are added or removed, etc.
>
> In this context, "revocable" means that the resources may be taken away 
> and the container will be destroyed. The meaning of "revocable" in the 
> context of usage oversubscription includes this, but also the container may 
> experience a throttling (e.g. lower cpu shares, less network priority, etc).
>
> For this reason, and because we internally need to distinguish revocable 
> resources between the those that are generated by usage oversubscription 
> and those that are generated by the allocator, we're thinking of the 
> following change to the API:
>
>
>
> -  message RevocableInfo {}
> +  message RevocableInfo {
> +    message ThrottleInfo {}
> +
> +    // If set, indicates that the resources may be throttled at
> +    // any time. Throttle-able resoruces can be used for tasks
> +    // that do not have strict performance requirements and are
> +    // capable of handling being throttled.
> +    optional ThrottleInfo throttle_info;
> +  }
>
>    // If this is set, the resources are revocable, i.e., any tasks or
> -  // executors launched using these resources could get preempted or
> -  // throttled at any time. This could be used by frameworks to run
> -  // best effort tasks that do not need strict uptime or performance
> +  // executors launched using these resources could be terminated at
> +  // any time. This could be used by frameworks to run
> +  // best effort tasks that do not need strict uptime
>    // guarantees. Note that if this is set, 'disk' or 'reservation'
>    // cannot be set.
>    optional RevocableInfo revocable = 9;
>
>
>
> Essentially we want to distinguish between revocable and revocable + 
> throttle-able. This is because usage-oversubscription generates 
> throttle-able revocable resources, whereas the allocator does not. This 
> also solves our problem of distinguishing between these two kinds of 
> revocable resources internally.
>
> Feedback welcome!
>
> Ben
>
>

Re: RFC: RevocableInfo Changes

Posted by Klaus Ma <kl...@hotmail.com>.

Answer inline :).

> On Mar 29, 2016, at 05:25, Niklas Nielsen <ni...@qni.dk> wrote:
> 
> Echoing Ben Mahler's comment. I still don't find the ThrottleInfo very intuitive.
[Klaus]: Not sure which one is better: "ScavengeInfo"/ "BestEffortInfo"/“ThrottleableInfo” :).

> Did you discuss the general notion of resource quality further?
[Klaus]: Yes, I’m thinking how user/developer will use this feature and its behaviour; that’ll be helpful to the design.

> 
> On Mon, Mar 21, 2016 at 11:50 PM, Klaus Ma <klaus1982.cn@gmail.com <ma...@gmail.com>> wrote:
> @benm/joris,
> 
> here's the user scenario in my mind:
> 
> 1. master offers resources to the framework, e.g. 2 cpu
> 2. framework launch a task (2 cpu) and mark the task/executors as throttleable
> 3. in ResourceEstimator, it should only consider the throttleable task/executors:
>   - keep enough resources for the tasks/executors without throttleable flag/attribute
>   - report allocated but not used resources by task/executor with throttleable flag/attribute; for example, report 1 cpu as "Revocable.Throttleable" resources to framework in this case
> 4. it's up to framework to use which resources; "Revocable.Throttleable" means it'll share compress resources with resources owner, "Revocable" (without ThrottleableInfo) means it'll be evicted when the resources owner reclaimed it back
> 5. QoS Controller makes sure:
>   - enough resources for the tasks/executors without throttleable flag/attribute
>   - if used resources exceed allocated resources with throttleable flag/attribute, evict the task/executor on revocable resource
> 
> So to @connor's question, maybe a flag/attribute to task/executor when launching it. Regarding the name, both "ScavengeInfo"/ "BestEffortInfo"/"ThrottleableInfo" are OK for me, maybe "ScavengeInfo" is better.
> 
> Any comments?
> 
> For this scenario, I think there're still open questions:
> 1. Can framework launch task with throttleable flag/attribute on revocable resources?
> 2. For ResourceEstimator/QoS Controllor, should Agent double check it report?
> 3. What's the behaviour between the two container: the container on original resouces & the container on revocable resource?
> 4. Who handle compressible/in-compressible resources? Maybe ResourceEstimator/QoSController, it should not report in-compressible resources as Revocable.Throttleable.
> 
> Thanks
> Klaus
> 
> On Tuesday, March 22, 2016 at 4:13:10 AM UTC+8, Benjamin Mahler wrote:
> Yeah that's definitely a question I've been asking myself, and we synced on that with Niklas during the last meeting. The thought currently is that we should choose a better name than ThrottleInfo. ThrottleInfo seems to carry too strong of an implication about what the resources will experience. Rather, we could pick a name like "ScavengeInfo" / "BestEffortInfo" / etc that indicates that these resources are running within the un-utilized portion of the machine and _may_ experience degradation.
> 
> On Mon, Mar 21, 2016 at 1:26 AM, Joris Van Remoortere <joris@mesosphere.io <ma...@mesosphere.io>> wrote:
> @klaus:
> I think @connor's question is whether we are absolutely sure we never want to support throttle-able but non-revocable resources.
> It's clear from the protos that this is not supported, the question is whether we are sure that is what we want. If so, can you elaborate as to *why* we would never want that concept in Mesos.
> 
> — 
> Joris Van Remoortere
> Mesosphere
> 
> On Sun, Mar 20, 2016 at 8:33 PM, Klaus Ma <klaus1982.cn@gmail.com <ma...@gmail.com>> wrote:
> Here's some input :).
> 
> If throttling is tolerable but preemption is not, how would that be expressed? (Is that supported?)
> [Klaus]: It's not supported; only revocable resources has this attribute: non-throttleable or throttleable. The throttleable revocable resources is reported by ResourceEstimator which means the resources maybe throttled by its original owner.
> 
> How does this work with the QoS controller? Will there be a new correction type to indicate throttling, or does throttling happen "behind the agent's back"?
> [Klaus]: The QoSController/ResourceEstimator only manages throttleable revocable resources; the others resources (regular resources and non-throttleable revocable resources) are managed by allocator. The "manage" means generation and destroy/eviction. Regarding "throttling happen", good question. I think the throttling will dependent on containers, let me double check it :).
> 
> If any comments, please let me know.
> 
> ----
> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer 
> Platform OpenSource Technology, STG, IBM GCG 
> +86-10-8245 4084 <> | klaus1982.cn@gmail.com <ma...@gmail.com> | http://k82.me <http://k82.me/>
> On Sat, Mar 19, 2016 at 11:15 PM, <connor.p.d@gmail.com <ma...@gmail.com>> wrote:
> Thanks for the good explanations so far Ben and Klaus.  Apologies if you guys already covered these questions in the meeting:
> 
> If throttling is tolerable but preemption is not, how would that be expressed? (Is that supported?)
> 
> How does this work with the QoS controller? Will there be a new correction type to indicate throttling, or does throttling happen "behind the agent's back"?
> 
> Thanks,
> --
> Connor
> 
> > On Mar 19, 2016, at 04:01, Klaus Ma <klaus1982.cn@gmail.com <ma...@gmail.com>> wrote:
> >
> > @team, in the latest meeting, we agree to keep current name ThrottleInfo.
> >
> > If any more comments, please let me know.
> >
> >> On Wednesday, March 16, 2016 at 9:32:37 PM UTC+8, Guangya Liu wrote:
> >> Also please show your comments if any for the name here, the current name is ThrottleInfo, in Kubernetes resources qos design document, they are using scavenging as the key work for such behaviour, so a possible name here could be ScavengeInfo , please show your comments if any for those two names or even if you want to propose a new name here.
> >>
> >> message RevocableInfo {
> >>     message ThrottleInfo {}
> >>
> >>     // If set, indicates that the resources may be throttled at
> >>     // any time. Throttle-able resoruces can be used for tasks
> >>     // that do not have strict performance requirements and are
> >>     // capable of handling being throttled.
> >>     optional ThrottleInfo throttle_info = 1;
> >>   }
> >>
> >> 在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道：
> >>>
> >>> The patches are updated accordingly; JIRA: MESOS-3888 , RR: https://reviews.apache.org/r/40375/ <https://reviews.apache.org/r/40375/> .
> >>>
> >>> Thanks
> >>> klaus
> >>>
> >>>> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler wrote:
> >>>> Hey folks,
> >>>>
> >>>> In the resource allocation working group we've been looking into a few projects that will make the allocator able to offer out resources as revocable. For example:
> >>>>
> >>>> -We'll want to eventually allocate resources as revocable _by default_, only allowing non-revocable when there are guarantees put in place (static reservations or quota).
> >>>>
> >>>> -On the path to revocable by default, we can incrementally start to offer certain resources as revocable. Consider when quota is set but the role isn't using all of the quota. The unallocated quota can be offered to other roles, but it should be revocable because we may revoke them should the quota'ed role want to use the resources. Unused reservations fall into a similar category.
> >>>>
> >>>> -Going revocable by default also allows us to enforce fairness in a dynamically changing cluster by revoking resources as weights are changed, frameworks are added or removed, etc.
> >>>>
> >>>> In this context, "revocable" means that the resources may be taken away and the container will be destroyed. The meaning of "revocable" in the context of usage oversubscription includes this, but also the container may experience a throttling (e.g. lower cpu shares, less network priority, etc).
> >>>>
> >>>> For this reason, and because we internally need to distinguish revocable resources between the those that are generated by usage oversubscription and those that are generated by the allocator, we're thinking of the following change to the API:
> >>>>
> >>>>
> >>>>
> >>>> -  message RevocableInfo {}
> >>>> +  message RevocableInfo {
> >>>> +    message ThrottleInfo {}
> >>>> +
> >>>> +    // If set, indicates that the resources may be throttled at
> >>>> +    // any time. Throttle-able resoruces can be used for tasks
> >>>> +    // that do not have strict performance requirements and are
> >>>> +    // capable of handling being throttled.
> >>>> +    optional ThrottleInfo throttle_info;
> >>>> +  }
> >>>>
> >>>>    // If this is set, the resources are revocable, i.e., any tasks or
> >>>> -  // executors launched using these resources could get preempted or
> >>>> -  // throttled at any time. This could be used by frameworks to run
> >>>> -  // best effort tasks that do not need strict uptime or performance
> >>>> +  // executors launched using these resources could be terminated at
> >>>> +  // any time. This could be used by frameworks to run
> >>>> +  // best effort tasks that do not need strict uptime
> >>>>    // guarantees. Note that if this is set, 'disk' or 'reservation'
> >>>>    // cannot be set.
> >>>>    optional RevocableInfo revocable = 9;
> >>>>
> >>>>
> >>>>
> >>>> Essentially we want to distinguish between revocable and revocable + throttle-able. This is because usage-oversubscription generates throttle-able revocable resources, whereas the allocator does not. This also solves our problem of distinguishing between these two kinds of revocable resources internally.
> >>>>
> >>>> Feedback welcome!
> >>>>
> >>>> Ben
> >>>>
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups "Mesos Resource Allocation Working Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to mesos-allocation+unsubscribe@googlegroups.com <ma...@googlegroups.com>.
> To post to this group, send email to mesos-allocation@googlegroups.com <ma...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com <https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> 
> For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups "Mesos Resource Allocation Working Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to mesos-allocation+unsubscribe@googlegroups.com <ma...@googlegroups.com>.
> To post to this group, send email to mesos-allocation@googlegroups.com <ma...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/mesos-allocation/CAKgKkU6EoOVvFVw0ppptE%2BA%2BHXuTB5UwETkHXz8CPsdqaDeh%2BQ%40mail.gmail.com <https://groups.google.com/d/msgid/mesos-allocation/CAKgKkU6EoOVvFVw0ppptE%2BA%2BHXuTB5UwETkHXz8CPsdqaDeh%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> 
> For more options, visit https://groups.google.com/d/optout <https://groups.google.com/d/optout>.
> 
> 
> 
> 
> -- 
> Niklas

Re: RFC: RevocableInfo Changes

Posted by Niklas Nielsen <ni...@qni.dk>.

Echoing Ben Mahler's comment. I still don't find the ThrottleInfo very
intuitive. Did you discuss the general notion of resource quality further?

On Mon, Mar 21, 2016 at 11:50 PM, Klaus Ma <kl...@gmail.com> wrote:

> @benm/joris,
>
> here's the user scenario in my mind:
>
> 1. master offers resources to the framework, e.g. 2 cpu
> 2. framework launch a task (2 cpu) and *mark the task/executors as
> throttleable*
> 3. in ResourceEstimator, it should only consider the throttleable
> task/executors:
>   - keep enough resources for the tasks/executors *without* throttleable
> flag/attribute
>   - report allocated but not used resources by task/executor *with*
> throttleable flag/attribute; for example, report 1 cpu as "
> *Revocable.Throttleable"* resources to framework in this case
> 4. it's up to framework to use which resources; "*Revocable.Throttleable*"
> means it'll share compress resources with resources owner, "*Revocable*"
> (without ThrottleableInfo) means it'll be evicted when the resources owner
> reclaimed it back
> 5. QoS Controller makes sure:
>   - enough resources for the tasks/executors *without* throttleable
> flag/attribute
>   - if used resources exceed allocated resources *with* throttleable
> flag/attribute, evict the task/executor on revocable resource
>
> So to @connor's question, maybe a flag/attribute to task/executor when
> launching it. Regarding the name, both "ScavengeInfo"/
> "BestEffortInfo"/"ThrottleableInfo" are OK for me, maybe "ScavengeInfo" is
> better.
>
> Any comments?
>
> For this scenario, I think there're still open questions:
> 1. Can framework launch task with throttleable flag/attribute on revocable
> resources?
> 2. For ResourceEstimator/QoS Controllor, should Agent double check it
> report?
> 3. What's the behaviour between the two container: the container on
> original resouces & the container on revocable resource?
> 4. Who handle compressible/in-compressible resources? Maybe
> ResourceEstimator/QoSController, it should not report in-compressible
> resources as Revocable.Throttleable.
>
> Thanks
> Klaus
>
> On Tuesday, March 22, 2016 at 4:13:10 AM UTC+8, Benjamin Mahler wrote:
>>
>> Yeah that's definitely a question I've been asking myself, and we synced
>> on that with Niklas during the last meeting. The thought currently is that
>> we should choose a better name than ThrottleInfo. ThrottleInfo seems to
>> carry too strong of an implication about what the resources will
>> experience. Rather, we could pick a name like "ScavengeInfo" /
>> "BestEffortInfo" / etc that indicates that these resources are running
>> within the un-utilized portion of the machine and _may_ experience
>> degradation.
>>
>> On Mon, Mar 21, 2016 at 1:26 AM, Joris Van Remoortere <
>> joris@mesosphere.io> wrote:
>>
>>> @klaus:
>>> I think @connor's question is whether we are absolutely sure we never
>>> want to support throttle-able but non-revocable resources.
>>> It's clear from the protos that this is not supported, the question is
>>> whether we are sure that is what we want. If so, can you elaborate as to
>>> *why* we would never want that concept in Mesos.
>>>
>>> —
>>> *Joris Van Remoortere*
>>> Mesosphere
>>>
>>> On Sun, Mar 20, 2016 at 8:33 PM, Klaus Ma <kl...@gmail.com>
>>> wrote:
>>>
>>>> Here's some input :).
>>>>
>>>> If throttling is tolerable but preemption is not, how would that be
>>>> expressed? (Is that supported?)
>>>> [Klaus]: It's not supported; only revocable resources has this
>>>> attribute: non-throttleable or throttleable. The throttleable revocable
>>>> resources is reported by ResourceEstimator which means the resources maybe
>>>> throttled by its original owner.
>>>>
>>>> How does this work with the QoS controller? Will there be a new
>>>> correction type to indicate throttling, or does throttling happen "behind
>>>> the agent's back"?
>>>> [Klaus]: The QoSController/ResourceEstimator only manages throttleable
>>>> revocable resources; the others resources (regular resources and
>>>> non-throttleable revocable resources) are managed by allocator. The
>>>> "manage" means generation and destroy/eviction. Regarding "throttling
>>>> happen", good question. I think the throttling will dependent on
>>>> containers, let me double check it :).
>>>>
>>>> If any comments, please let me know.
>>>>
>>>> ----
>>>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
>>>> Platform OpenSource Technology, STG, IBM GCG
>>>> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>>>>
>>>> On Sat, Mar 19, 2016 at 11:15 PM, <co...@gmail.com> wrote:
>>>>
>>>>> Thanks for the good explanations so far Ben and Klaus.  Apologies if
>>>>> you guys already covered these questions in the meeting:
>>>>>
>>>>> If throttling is tolerable but preemption is not, how would that be
>>>>> expressed? (Is that supported?)
>>>>>
>>>>> How does this work with the QoS controller? Will there be a new
>>>>> correction type to indicate throttling, or does throttling happen "behind
>>>>> the agent's back"?
>>>>>
>>>>> Thanks,
>>>>> --
>>>>> Connor
>>>>>
>>>>> > On Mar 19, 2016, at 04:01, Klaus Ma <kl...@gmail.com> wrote:
>>>>> >
>>>>> > @team, in the latest meeting, we agree to keep current name
>>>>> ThrottleInfo.
>>>>> >
>>>>> > If any more comments, please let me know.
>>>>> >
>>>>> >> On Wednesday, March 16, 2016 at 9:32:37 PM UTC+8, Guangya Liu wrote:
>>>>> >> Also please show your comments if any for the name here, the
>>>>> current name is ThrottleInfo, in Kubernetes resources qos design document,
>>>>> they are using scavenging as the key work for such behaviour, so a possible
>>>>> name here could be ScavengeInfo , please show your comments if any for
>>>>> those two names or even if you want to propose a new name here.
>>>>> >>
>>>>> >> message RevocableInfo {
>>>>> >>     message ThrottleInfo {}
>>>>> >>
>>>>> >>     // If set, indicates that the resources may be throttled at
>>>>> >>     // any time. Throttle-able resoruces can be used for tasks
>>>>> >>     // that do not have strict performance requirements and are
>>>>> >>     // capable of handling being throttled.
>>>>> >>     optional ThrottleInfo throttle_info = 1;
>>>>> >>   }
>>>>> >>
>>>>> >> 在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道：
>>>>> >>>
>>>>> >>> The patches are updated accordingly; JIRA: MESOS-3888 , RR:
>>>>> https://reviews.apache.org/r/40375/ .
>>>>> >>>
>>>>> >>> Thanks
>>>>> >>> klaus
>>>>> >>>
>>>>> >>>> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler
>>>>> wrote:
>>>>> >>>> Hey folks,
>>>>> >>>>
>>>>> >>>> In the resource allocation working group we've been looking into
>>>>> a few projects that will make the allocator able to offer out resources as
>>>>> revocable. For example:
>>>>> >>>>
>>>>> >>>> -We'll want to eventually allocate resources as revocable _by
>>>>> default_, only allowing non-revocable when there are guarantees put in
>>>>> place (static reservations or quota).
>>>>> >>>>
>>>>> >>>> -On the path to revocable by default, we can incrementally start
>>>>> to offer certain resources as revocable. Consider when quota is set but the
>>>>> role isn't using all of the quota. The unallocated quota can be offered to
>>>>> other roles, but it should be revocable because we may revoke them should
>>>>> the quota'ed role want to use the resources. Unused reservations fall into
>>>>> a similar category.
>>>>> >>>>
>>>>> >>>> -Going revocable by default also allows us to enforce fairness in
>>>>> a dynamically changing cluster by revoking resources as weights are
>>>>> changed, frameworks are added or removed, etc.
>>>>> >>>>
>>>>> >>>> In this context, "revocable" means that the resources may be
>>>>> taken away and the container will be destroyed. The meaning of "revocable"
>>>>> in the context of usage oversubscription includes this, but also the
>>>>> container may experience a throttling (e.g. lower cpu shares, less network
>>>>> priority, etc).
>>>>> >>>>
>>>>> >>>> For this reason, and because we internally need to distinguish
>>>>> revocable resources between the those that are generated by usage
>>>>> oversubscription and those that are generated by the allocator, we're
>>>>> thinking of the following change to the API:
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> -  message RevocableInfo {}
>>>>> >>>> +  message RevocableInfo {
>>>>> >>>> +    message ThrottleInfo {}
>>>>> >>>> +
>>>>> >>>> +    // If set, indicates that the resources may be throttled at
>>>>> >>>> +    // any time. Throttle-able resoruces can be used for tasks
>>>>> >>>> +    // that do not have strict performance requirements and are
>>>>> >>>> +    // capable of handling being throttled.
>>>>> >>>> +    optional ThrottleInfo throttle_info;
>>>>> >>>> +  }
>>>>> >>>>
>>>>> >>>>    // If this is set, the resources are revocable, i.e., any
>>>>> tasks or
>>>>> >>>> -  // executors launched using these resources could get
>>>>> preempted or
>>>>> >>>> -  // throttled at any time. This could be used by frameworks to
>>>>> run
>>>>> >>>> -  // best effort tasks that do not need strict uptime or
>>>>> performance
>>>>> >>>> +  // executors launched using these resources could be
>>>>> terminated at
>>>>> >>>> +  // any time. This could be used by frameworks to run
>>>>> >>>> +  // best effort tasks that do not need strict uptime
>>>>> >>>>    // guarantees. Note that if this is set, 'disk' or
>>>>> 'reservation'
>>>>> >>>>    // cannot be set.
>>>>> >>>>    optional RevocableInfo revocable = 9;
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> Essentially we want to distinguish between revocable and
>>>>> revocable + throttle-able. This is because usage-oversubscription generates
>>>>> throttle-able revocable resources, whereas the allocator does not. This
>>>>> also solves our problem of distinguishing between these two kinds of
>>>>> revocable resources internally.
>>>>> >>>>
>>>>> >>>> Feedback welcome!
>>>>> >>>>
>>>>> >>>> Ben
>>>>> >>>>
>>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Mesos Resource Allocation Working Group" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to mesos-allocation+unsubscribe@googlegroups.com.
>>>> To post to this group, send email to mesos-allocation@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Mesos Resource Allocation Working Group" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to mesos-allocation+unsubscribe@googlegroups.com.
>>> To post to this group, send email to mesos-allocation@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/mesos-allocation/CAKgKkU6EoOVvFVw0ppptE%2BA%2BHXuTB5UwETkHXz8CPsdqaDeh%2BQ%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/mesos-allocation/CAKgKkU6EoOVvFVw0ppptE%2BA%2BHXuTB5UwETkHXz8CPsdqaDeh%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>


-- 
Niklas

Re: RFC: RevocableInfo Changes

Posted by Klaus Ma <kl...@gmail.com>.

@benm/joris,

here's the user scenario in my mind:

1. master offers resources to the framework, e.g. 2 cpu
2. framework launch a task (2 cpu) and *mark the task/executors as 
throttleable*
3. in ResourceEstimator, it should only consider the throttleable 
task/executors:
  - keep enough resources for the tasks/executors *without* throttleable 
flag/attribute
  - report allocated but not used resources by task/executor *with* 
throttleable flag/attribute; for example, report 1 cpu as "
*Revocable.Throttleable"* resources to framework in this case
4. it's up to framework to use which resources; "*Revocable.Throttleable*" 
means it'll share compress resources with resources owner, "*Revocable*" 
(without ThrottleableInfo) means it'll be evicted when the resources owner 
reclaimed it back
5. QoS Controller makes sure:
  - enough resources for the tasks/executors *without* throttleable 
flag/attribute
  - if used resources exceed allocated resources *with* throttleable 
flag/attribute, evict the task/executor on revocable resource

So to @connor's question, maybe a flag/attribute to task/executor when 
launching it. Regarding the name, both "ScavengeInfo"/ 
"BestEffortInfo"/"ThrottleableInfo" are OK for me, maybe "ScavengeInfo" is 
better.

Any comments?

For this scenario, I think there're still open questions:
1. Can framework launch task with throttleable flag/attribute on revocable 
resources?
2. For ResourceEstimator/QoS Controllor, should Agent double check it 
report?
3. What's the behaviour between the two container: the container on 
original resouces & the container on revocable resource?
4. Who handle compressible/in-compressible resources? Maybe 
ResourceEstimator/QoSController, it should not report in-compressible 
resources as Revocable.Throttleable.

Thanks
Klaus

On Tuesday, March 22, 2016 at 4:13:10 AM UTC+8, Benjamin Mahler wrote:
>
> Yeah that's definitely a question I've been asking myself, and we synced 
> on that with Niklas during the last meeting. The thought currently is that 
> we should choose a better name than ThrottleInfo. ThrottleInfo seems to 
> carry too strong of an implication about what the resources will 
> experience. Rather, we could pick a name like "ScavengeInfo" / 
> "BestEffortInfo" / etc that indicates that these resources are running 
> within the un-utilized portion of the machine and _may_ experience 
> degradation.
>
> On Mon, Mar 21, 2016 at 1:26 AM, Joris Van Remoortere <joris@mesosphere.io
> > wrote:
>
>> @klaus:
>> I think @connor's question is whether we are absolutely sure we never 
>> want to support throttle-able but non-revocable resources.
>> It's clear from the protos that this is not supported, the question is 
>> whether we are sure that is what we want. If so, can you elaborate as to 
>> *why* we would never want that concept in Mesos.
>>
>> — 
>> *Joris Van Remoortere*
>> Mesosphere
>>
>> On Sun, Mar 20, 2016 at 8:33 PM, Klaus Ma <kl...@gmail.com> wrote:
>>
>>> Here's some input :).
>>>
>>> If throttling is tolerable but preemption is not, how would that be 
>>> expressed? (Is that supported?)
>>> [Klaus]: It's not supported; only revocable resources has this 
>>> attribute: non-throttleable or throttleable. The throttleable revocable 
>>> resources is reported by ResourceEstimator which means the resources maybe 
>>> throttled by its original owner.
>>>
>>> How does this work with the QoS controller? Will there be a new 
>>> correction type to indicate throttling, or does throttling happen "behind 
>>> the agent's back"?
>>> [Klaus]: The QoSController/ResourceEstimator only manages throttleable 
>>> revocable resources; the others resources (regular resources and 
>>> non-throttleable revocable resources) are managed by allocator. The 
>>> "manage" means generation and destroy/eviction. Regarding "throttling 
>>> happen", good question. I think the throttling will dependent on 
>>> containers, let me double check it :).
>>>
>>> If any comments, please let me know.
>>>
>>> ----
>>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer 
>>> Platform OpenSource Technology, STG, IBM GCG 
>>> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>>>
>>> On Sat, Mar 19, 2016 at 11:15 PM, <co...@gmail.com> wrote:
>>>
>>>> Thanks for the good explanations so far Ben and Klaus.  Apologies if 
>>>> you guys already covered these questions in the meeting:
>>>>
>>>> If throttling is tolerable but preemption is not, how would that be 
>>>> expressed? (Is that supported?)
>>>>
>>>> How does this work with the QoS controller? Will there be a new 
>>>> correction type to indicate throttling, or does throttling happen "behind 
>>>> the agent's back"?
>>>>
>>>> Thanks,
>>>> --
>>>> Connor
>>>>
>>>> > On Mar 19, 2016, at 04:01, Klaus Ma <kl...@gmail.com> wrote:
>>>> >
>>>> > @team, in the latest meeting, we agree to keep current name 
>>>> ThrottleInfo.
>>>> >
>>>> > If any more comments, please let me know.
>>>> >
>>>> >> On Wednesday, March 16, 2016 at 9:32:37 PM UTC+8, Guangya Liu wrote:
>>>> >> Also please show your comments if any for the name here, the current 
>>>> name is ThrottleInfo, in Kubernetes resources qos design document, they are 
>>>> using scavenging as the key work for such behaviour, so a possible name 
>>>> here could be ScavengeInfo , please show your comments if any for those two 
>>>> names or even if you want to propose a new name here.
>>>> >>
>>>> >> message RevocableInfo {
>>>> >>     message ThrottleInfo {}
>>>> >>
>>>> >>     // If set, indicates that the resources may be throttled at
>>>> >>     // any time. Throttle-able resoruces can be used for tasks
>>>> >>     // that do not have strict performance requirements and are
>>>> >>     // capable of handling being throttled.
>>>> >>     optional ThrottleInfo throttle_info = 1;
>>>> >>   }
>>>> >>
>>>> >> 在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道：
>>>> >>>
>>>> >>> The patches are updated accordingly; JIRA: MESOS-3888 , RR: 
>>>> https://reviews.apache.org/r/40375/ .
>>>> >>>
>>>> >>> Thanks
>>>> >>> klaus
>>>> >>>
>>>> >>>> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler 
>>>> wrote:
>>>> >>>> Hey folks,
>>>> >>>>
>>>> >>>> In the resource allocation working group we've been looking into a 
>>>> few projects that will make the allocator able to offer out resources as 
>>>> revocable. For example:
>>>> >>>>
>>>> >>>> -We'll want to eventually allocate resources as revocable _by 
>>>> default_, only allowing non-revocable when there are guarantees put in 
>>>> place (static reservations or quota).
>>>> >>>>
>>>> >>>> -On the path to revocable by default, we can incrementally start 
>>>> to offer certain resources as revocable. Consider when quota is set but the 
>>>> role isn't using all of the quota. The unallocated quota can be offered to 
>>>> other roles, but it should be revocable because we may revoke them should 
>>>> the quota'ed role want to use the resources. Unused reservations fall into 
>>>> a similar category.
>>>> >>>>
>>>> >>>> -Going revocable by default also allows us to enforce fairness in 
>>>> a dynamically changing cluster by revoking resources as weights are 
>>>> changed, frameworks are added or removed, etc.
>>>> >>>>
>>>> >>>> In this context, "revocable" means that the resources may be taken 
>>>> away and the container will be destroyed. The meaning of "revocable" in the 
>>>> context of usage oversubscription includes this, but also the container may 
>>>> experience a throttling (e.g. lower cpu shares, less network priority, etc).
>>>> >>>>
>>>> >>>> For this reason, and because we internally need to distinguish 
>>>> revocable resources between the those that are generated by usage 
>>>> oversubscription and those that are generated by the allocator, we're 
>>>> thinking of the following change to the API:
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> -  message RevocableInfo {}
>>>> >>>> +  message RevocableInfo {
>>>> >>>> +    message ThrottleInfo {}
>>>> >>>> +
>>>> >>>> +    // If set, indicates that the resources may be throttled at
>>>> >>>> +    // any time. Throttle-able resoruces can be used for tasks
>>>> >>>> +    // that do not have strict performance requirements and are
>>>> >>>> +    // capable of handling being throttled.
>>>> >>>> +    optional ThrottleInfo throttle_info;
>>>> >>>> +  }
>>>> >>>>
>>>> >>>>    // If this is set, the resources are revocable, i.e., any tasks 
>>>> or
>>>> >>>> -  // executors launched using these resources could get preempted 
>>>> or
>>>> >>>> -  // throttled at any time. This could be used by frameworks to 
>>>> run
>>>> >>>> -  // best effort tasks that do not need strict uptime or 
>>>> performance
>>>> >>>> +  // executors launched using these resources could be terminated 
>>>> at
>>>> >>>> +  // any time. This could be used by frameworks to run
>>>> >>>> +  // best effort tasks that do not need strict uptime
>>>> >>>>    // guarantees. Note that if this is set, 'disk' or 'reservation'
>>>> >>>>    // cannot be set.
>>>> >>>>    optional RevocableInfo revocable = 9;
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> Essentially we want to distinguish between revocable and revocable 
>>>> + throttle-able. This is because usage-oversubscription generates 
>>>> throttle-able revocable resources, whereas the allocator does not. This 
>>>> also solves our problem of distinguishing between these two kinds of 
>>>> revocable resources internally.
>>>> >>>>
>>>> >>>> Feedback welcome!
>>>> >>>>
>>>> >>>> Ben
>>>> >>>>
>>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Mesos Resource Allocation Working Group" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to mesos-allocation+unsubscribe@googlegroups.com.
>>> To post to this group, send email to mesos-allocation@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com 
>>> <https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Mesos Resource Allocation Working Group" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to mesos-allocation+unsubscribe@googlegroups.com.
>> To post to this group, send email to mesos-allocation@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/mesos-allocation/CAKgKkU6EoOVvFVw0ppptE%2BA%2BHXuTB5UwETkHXz8CPsdqaDeh%2BQ%40mail.gmail.com 
>> <https://groups.google.com/d/msgid/mesos-allocation/CAKgKkU6EoOVvFVw0ppptE%2BA%2BHXuTB5UwETkHXz8CPsdqaDeh%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

Re: RFC: RevocableInfo Changes

Posted by Guangya Liu <gy...@gmail.com>.

Some of my thinking here:

1) The ThrottleInfo may need to belong to "Resources" but not limited to 
"RevocableInfo". The cpus resources can be throttled even if it is not 
revocable resources.
2) There need to be a flag to indicate if the resources is Scavenge-able or 
Best effort, I did not have inclination on which one is better as both 
seems clear enough to describe the resources type. The Kubernetes document 
here 
<https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/resource-qos.md#motivation> 
is also using scavenge and best effort concept here.

*message ThrottleInfo {}*
message RevocableInfo {
    *message ScavengeInfo {}*

*    // If set, indicates that the resources may be revoked at*
*    // any time. Scavenge-able resources can be used for tasks*
*    // that do not have strict performance requirements and are*
*    // capable of handling being revoked.*
*    optional **ScavengeInfo** scavenge_info = 1;*
  }

Thanks,

Guangya

在 2016年3月22日星期二 UTC+8上午4:13:10，Benjamin Mahler写道：
>
> Yeah that's definitely a question I've been asking myself, and we synced 
> on that with Niklas during the last meeting. The thought currently is that 
> we should choose a better name than ThrottleInfo. ThrottleInfo seems to 
> carry too strong of an implication about what the resources will 
> experience. Rather, we could pick a name like "ScavengeInfo" / 
> "BestEffortInfo" / etc that indicates that these resources are running 
> within the un-utilized portion of the machine and _may_ experience 
> degradation.
>
> On Mon, Mar 21, 2016 at 1:26 AM, Joris Van Remoortere <joris@mesosphere.io
> > wrote:
>
>> @klaus:
>> I think @connor's question is whether we are absolutely sure we never 
>> want to support throttle-able but non-revocable resources.
>> It's clear from the protos that this is not supported, the question is 
>> whether we are sure that is what we want. If so, can you elaborate as to 
>> *why* we would never want that concept in Mesos.
>>
>> — 
>> *Joris Van Remoortere*
>> Mesosphere
>>
>> On Sun, Mar 20, 2016 at 8:33 PM, Klaus Ma <kl...@gmail.com> wrote:
>>
>>> Here's some input :).
>>>
>>> If throttling is tolerable but preemption is not, how would that be 
>>> expressed? (Is that supported?)
>>> [Klaus]: It's not supported; only revocable resources has this 
>>> attribute: non-throttleable or throttleable. The throttleable revocable 
>>> resources is reported by ResourceEstimator which means the resources maybe 
>>> throttled by its original owner.
>>>
>>> How does this work with the QoS controller? Will there be a new 
>>> correction type to indicate throttling, or does throttling happen "behind 
>>> the agent's back"?
>>> [Klaus]: The QoSController/ResourceEstimator only manages throttleable 
>>> revocable resources; the others resources (regular resources and 
>>> non-throttleable revocable resources) are managed by allocator. The 
>>> "manage" means generation and destroy/eviction. Regarding "throttling 
>>> happen", good question. I think the throttling will dependent on 
>>> containers, let me double check it :).
>>>
>>> If any comments, please let me know.
>>>
>>> ----
>>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer 
>>> Platform OpenSource Technology, STG, IBM GCG 
>>> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>>>
>>> On Sat, Mar 19, 2016 at 11:15 PM, <co...@gmail.com> wrote:
>>>
>>>> Thanks for the good explanations so far Ben and Klaus.  Apologies if 
>>>> you guys already covered these questions in the meeting:
>>>>
>>>> If throttling is tolerable but preemption is not, how would that be 
>>>> expressed? (Is that supported?)
>>>>
>>>> How does this work with the QoS controller? Will there be a new 
>>>> correction type to indicate throttling, or does throttling happen "behind 
>>>> the agent's back"?
>>>>
>>>> Thanks,
>>>> --
>>>> Connor
>>>>
>>>> > On Mar 19, 2016, at 04:01, Klaus Ma <kl...@gmail.com> wrote:
>>>> >
>>>> > @team, in the latest meeting, we agree to keep current name 
>>>> ThrottleInfo.
>>>> >
>>>> > If any more comments, please let me know.
>>>> >
>>>> >> On Wednesday, March 16, 2016 at 9:32:37 PM UTC+8, Guangya Liu wrote:
>>>> >> Also please show your comments if any for the name here, the current 
>>>> name is ThrottleInfo, in Kubernetes resources qos design document, they are 
>>>> using scavenging as the key work for such behaviour, so a possible name 
>>>> here could be ScavengeInfo , please show your comments if any for those two 
>>>> names or even if you want to propose a new name here.
>>>> >>
>>>> >> message RevocableInfo {
>>>> >>     message ThrottleInfo {}
>>>> >>
>>>> >>     // If set, indicates that the resources may be throttled at
>>>> >>     // any time. Throttle-able resoruces can be used for tasks
>>>> >>     // that do not have strict performance requirements and are
>>>> >>     // capable of handling being throttled.
>>>> >>     optional ThrottleInfo throttle_info = 1;
>>>> >>   }
>>>> >>
>>>> >> 在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道：
>>>> >>>
>>>> >>> The patches are updated accordingly; JIRA: MESOS-3888 , RR: 
>>>> https://reviews.apache.org/r/40375/ .
>>>> >>>
>>>> >>> Thanks
>>>> >>> klaus
>>>> >>>
>>>> >>>> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler 
>>>> wrote:
>>>> >>>> Hey folks,
>>>> >>>>
>>>> >>>> In the resource allocation working group we've been looking into a 
>>>> few projects that will make the allocator able to offer out resources as 
>>>> revocable. For example:
>>>> >>>>
>>>> >>>> -We'll want to eventually allocate resources as revocable _by 
>>>> default_, only allowing non-revocable when there are guarantees put in 
>>>> place (static reservations or quota).
>>>> >>>>
>>>> >>>> -On the path to revocable by default, we can incrementally start 
>>>> to offer certain resources as revocable. Consider when quota is set but the 
>>>> role isn't using all of the quota. The unallocated quota can be offered to 
>>>> other roles, but it should be revocable because we may revoke them should 
>>>> the quota'ed role want to use the resources. Unused reservations fall into 
>>>> a similar category.
>>>> >>>>
>>>> >>>> -Going revocable by default also allows us to enforce fairness in 
>>>> a dynamically changing cluster by revoking resources as weights are 
>>>> changed, frameworks are added or removed, etc.
>>>> >>>>
>>>> >>>> In this context, "revocable" means that the resources may be taken 
>>>> away and the container will be destroyed. The meaning of "revocable" in the 
>>>> context of usage oversubscription includes this, but also the container may 
>>>> experience a throttling (e.g. lower cpu shares, less network priority, etc).
>>>> >>>>
>>>> >>>> For this reason, and because we internally need to distinguish 
>>>> revocable resources between the those that are generated by usage 
>>>> oversubscription and those that are generated by the allocator, we're 
>>>> thinking of the following change to the API:
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> -  message RevocableInfo {}
>>>> >>>> +  message RevocableInfo {
>>>> >>>> +    message ThrottleInfo {}
>>>> >>>> +
>>>> >>>> +    // If set, indicates that the resources may be throttled at
>>>> >>>> +    // any time. Throttle-able resoruces can be used for tasks
>>>> >>>> +    // that do not have strict performance requirements and are
>>>> >>>> +    // capable of handling being throttled.
>>>> >>>> +    optional ThrottleInfo throttle_info;
>>>> >>>> +  }
>>>> >>>>
>>>> >>>>    // If this is set, the resources are revocable, i.e., any tasks 
>>>> or
>>>> >>>> -  // executors launched using these resources could get preempted 
>>>> or
>>>> >>>> -  // throttled at any time. This could be used by frameworks to 
>>>> run
>>>> >>>> -  // best effort tasks that do not need strict uptime or 
>>>> performance
>>>> >>>> +  // executors launched using these resources could be terminated 
>>>> at
>>>> >>>> +  // any time. This could be used by frameworks to run
>>>> >>>> +  // best effort tasks that do not need strict uptime
>>>> >>>>    // guarantees. Note that if this is set, 'disk' or 'reservation'
>>>> >>>>    // cannot be set.
>>>> >>>>    optional RevocableInfo revocable = 9;
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> Essentially we want to distinguish between revocable and revocable 
>>>> + throttle-able. This is because usage-oversubscription generates 
>>>> throttle-able revocable resources, whereas the allocator does not. This 
>>>> also solves our problem of distinguishing between these two kinds of 
>>>> revocable resources internally.
>>>> >>>>
>>>> >>>> Feedback welcome!
>>>> >>>>
>>>> >>>> Ben
>>>> >>>>
>>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Mesos Resource Allocation Working Group" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to mesos-allocation+unsubscribe@googlegroups.com.
>>> To post to this group, send email to mesos-allocation@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com 
>>> <https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Mesos Resource Allocation Working Group" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to mesos-allocation+unsubscribe@googlegroups.com.
>> To post to this group, send email to mesos-allocation@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/mesos-allocation/CAKgKkU6EoOVvFVw0ppptE%2BA%2BHXuTB5UwETkHXz8CPsdqaDeh%2BQ%40mail.gmail.com 
>> <https://groups.google.com/d/msgid/mesos-allocation/CAKgKkU6EoOVvFVw0ppptE%2BA%2BHXuTB5UwETkHXz8CPsdqaDeh%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

Re: RFC: RevocableInfo Changes

Posted by Benjamin Mahler <bm...@apache.org>.

Yeah that's definitely a question I've been asking myself, and we synced on
that with Niklas during the last meeting. The thought currently is that we
should choose a better name than ThrottleInfo. ThrottleInfo seems to carry
too strong of an implication about what the resources will experience.
Rather, we could pick a name like "ScavengeInfo" / "BestEffortInfo" / etc
that indicates that these resources are running within the un-utilized
portion of the machine and _may_ experience degradation.

On Mon, Mar 21, 2016 at 1:26 AM, Joris Van Remoortere <jo...@mesosphere.io>
wrote:

> @klaus:
> I think @connor's question is whether we are absolutely sure we never want
> to support throttle-able but non-revocable resources.
> It's clear from the protos that this is not supported, the question is
> whether we are sure that is what we want. If so, can you elaborate as to
> *why* we would never want that concept in Mesos.
>
> —
> *Joris Van Remoortere*
> Mesosphere
>
> On Sun, Mar 20, 2016 at 8:33 PM, Klaus Ma <kl...@gmail.com> wrote:
>
>> Here's some input :).
>>
>> If throttling is tolerable but preemption is not, how would that be
>> expressed? (Is that supported?)
>> [Klaus]: It's not supported; only revocable resources has this attribute:
>> non-throttleable or throttleable. The throttleable revocable resources is
>> reported by ResourceEstimator which means the resources maybe throttled by
>> its original owner.
>>
>> How does this work with the QoS controller? Will there be a new
>> correction type to indicate throttling, or does throttling happen "behind
>> the agent's back"?
>> [Klaus]: The QoSController/ResourceEstimator only manages throttleable
>> revocable resources; the others resources (regular resources and
>> non-throttleable revocable resources) are managed by allocator. The
>> "manage" means generation and destroy/eviction. Regarding "throttling
>> happen", good question. I think the throttling will dependent on
>> containers, let me double check it :).
>>
>> If any comments, please let me know.
>>
>> ----
>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
>> Platform OpenSource Technology, STG, IBM GCG
>> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>>
>> On Sat, Mar 19, 2016 at 11:15 PM, <co...@gmail.com> wrote:
>>
>>> Thanks for the good explanations so far Ben and Klaus.  Apologies if you
>>> guys already covered these questions in the meeting:
>>>
>>> If throttling is tolerable but preemption is not, how would that be
>>> expressed? (Is that supported?)
>>>
>>> How does this work with the QoS controller? Will there be a new
>>> correction type to indicate throttling, or does throttling happen "behind
>>> the agent's back"?
>>>
>>> Thanks,
>>> --
>>> Connor
>>>
>>> > On Mar 19, 2016, at 04:01, Klaus Ma <kl...@gmail.com> wrote:
>>> >
>>> > @team, in the latest meeting, we agree to keep current name
>>> ThrottleInfo.
>>> >
>>> > If any more comments, please let me know.
>>> >
>>> >> On Wednesday, March 16, 2016 at 9:32:37 PM UTC+8, Guangya Liu wrote:
>>> >> Also please show your comments if any for the name here, the current
>>> name is ThrottleInfo, in Kubernetes resources qos design document, they are
>>> using scavenging as the key work for such behaviour, so a possible name
>>> here could be ScavengeInfo , please show your comments if any for those two
>>> names or even if you want to propose a new name here.
>>> >>
>>> >> message RevocableInfo {
>>> >>     message ThrottleInfo {}
>>> >>
>>> >>     // If set, indicates that the resources may be throttled at
>>> >>     // any time. Throttle-able resoruces can be used for tasks
>>> >>     // that do not have strict performance requirements and are
>>> >>     // capable of handling being throttled.
>>> >>     optional ThrottleInfo throttle_info = 1;
>>> >>   }
>>> >>
>>> >> 在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道：
>>> >>>
>>> >>> The patches are updated accordingly; JIRA: MESOS-3888 , RR:
>>> https://reviews.apache.org/r/40375/ .
>>> >>>
>>> >>> Thanks
>>> >>> klaus
>>> >>>
>>> >>>> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler
>>> wrote:
>>> >>>> Hey folks,
>>> >>>>
>>> >>>> In the resource allocation working group we've been looking into a
>>> few projects that will make the allocator able to offer out resources as
>>> revocable. For example:
>>> >>>>
>>> >>>> -We'll want to eventually allocate resources as revocable _by
>>> default_, only allowing non-revocable when there are guarantees put in
>>> place (static reservations or quota).
>>> >>>>
>>> >>>> -On the path to revocable by default, we can incrementally start to
>>> offer certain resources as revocable. Consider when quota is set but the
>>> role isn't using all of the quota. The unallocated quota can be offered to
>>> other roles, but it should be revocable because we may revoke them should
>>> the quota'ed role want to use the resources. Unused reservations fall into
>>> a similar category.
>>> >>>>
>>> >>>> -Going revocable by default also allows us to enforce fairness in a
>>> dynamically changing cluster by revoking resources as weights are changed,
>>> frameworks are added or removed, etc.
>>> >>>>
>>> >>>> In this context, "revocable" means that the resources may be taken
>>> away and the container will be destroyed. The meaning of "revocable" in the
>>> context of usage oversubscription includes this, but also the container may
>>> experience a throttling (e.g. lower cpu shares, less network priority, etc).
>>> >>>>
>>> >>>> For this reason, and because we internally need to distinguish
>>> revocable resources between the those that are generated by usage
>>> oversubscription and those that are generated by the allocator, we're
>>> thinking of the following change to the API:
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> -  message RevocableInfo {}
>>> >>>> +  message RevocableInfo {
>>> >>>> +    message ThrottleInfo {}
>>> >>>> +
>>> >>>> +    // If set, indicates that the resources may be throttled at
>>> >>>> +    // any time. Throttle-able resoruces can be used for tasks
>>> >>>> +    // that do not have strict performance requirements and are
>>> >>>> +    // capable of handling being throttled.
>>> >>>> +    optional ThrottleInfo throttle_info;
>>> >>>> +  }
>>> >>>>
>>> >>>>    // If this is set, the resources are revocable, i.e., any tasks
>>> or
>>> >>>> -  // executors launched using these resources could get preempted
>>> or
>>> >>>> -  // throttled at any time. This could be used by frameworks to run
>>> >>>> -  // best effort tasks that do not need strict uptime or
>>> performance
>>> >>>> +  // executors launched using these resources could be terminated
>>> at
>>> >>>> +  // any time. This could be used by frameworks to run
>>> >>>> +  // best effort tasks that do not need strict uptime
>>> >>>>    // guarantees. Note that if this is set, 'disk' or 'reservation'
>>> >>>>    // cannot be set.
>>> >>>>    optional RevocableInfo revocable = 9;
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> Essentially we want to distinguish between revocable and revocable
>>> + throttle-able. This is because usage-oversubscription generates
>>> throttle-able revocable resources, whereas the allocator does not. This
>>> also solves our problem of distinguishing between these two kinds of
>>> revocable resources internally.
>>> >>>>
>>> >>>> Feedback welcome!
>>> >>>>
>>> >>>> Ben
>>> >>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Mesos Resource Allocation Working Group" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to mesos-allocation+unsubscribe@googlegroups.com.
>> To post to this group, send email to mesos-allocation@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com
>> <https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Mesos Resource Allocation Working Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mesos-allocation+unsubscribe@googlegroups.com.
> To post to this group, send email to mesos-allocation@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mesos-allocation/CAKgKkU6EoOVvFVw0ppptE%2BA%2BHXuTB5UwETkHXz8CPsdqaDeh%2BQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/mesos-allocation/CAKgKkU6EoOVvFVw0ppptE%2BA%2BHXuTB5UwETkHXz8CPsdqaDeh%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

Re: RFC: RevocableInfo Changes

Posted by Joris Van Remoortere <jo...@mesosphere.io>.

@klaus:
I think @connor's question is whether we are absolutely sure we never want
to support throttle-able but non-revocable resources.
It's clear from the protos that this is not supported, the question is
whether we are sure that is what we want. If so, can you elaborate as to
*why* we would never want that concept in Mesos.

—
*Joris Van Remoortere*
Mesosphere

On Sun, Mar 20, 2016 at 8:33 PM, Klaus Ma <kl...@gmail.com> wrote:

> Here's some input :).
>
> If throttling is tolerable but preemption is not, how would that be
> expressed? (Is that supported?)
> [Klaus]: It's not supported; only revocable resources has this attribute:
> non-throttleable or throttleable. The throttleable revocable resources is
> reported by ResourceEstimator which means the resources maybe throttled by
> its original owner.
>
> How does this work with the QoS controller? Will there be a new correction
> type to indicate throttling, or does throttling happen "behind the agent's
> back"?
> [Klaus]: The QoSController/ResourceEstimator only manages throttleable
> revocable resources; the others resources (regular resources and
> non-throttleable revocable resources) are managed by allocator. The
> "manage" means generation and destroy/eviction. Regarding "throttling
> happen", good question. I think the throttling will dependent on
> containers, let me double check it :).
>
> If any comments, please let me know.
>
> ----
> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> Platform OpenSource Technology, STG, IBM GCG
> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>
> On Sat, Mar 19, 2016 at 11:15 PM, <co...@gmail.com> wrote:
>
>> Thanks for the good explanations so far Ben and Klaus.  Apologies if you
>> guys already covered these questions in the meeting:
>>
>> If throttling is tolerable but preemption is not, how would that be
>> expressed? (Is that supported?)
>>
>> How does this work with the QoS controller? Will there be a new
>> correction type to indicate throttling, or does throttling happen "behind
>> the agent's back"?
>>
>> Thanks,
>> --
>> Connor
>>
>> > On Mar 19, 2016, at 04:01, Klaus Ma <kl...@gmail.com> wrote:
>> >
>> > @team, in the latest meeting, we agree to keep current name
>> ThrottleInfo.
>> >
>> > If any more comments, please let me know.
>> >
>> >> On Wednesday, March 16, 2016 at 9:32:37 PM UTC+8, Guangya Liu wrote:
>> >> Also please show your comments if any for the name here, the current
>> name is ThrottleInfo, in Kubernetes resources qos design document, they are
>> using scavenging as the key work for such behaviour, so a possible name
>> here could be ScavengeInfo , please show your comments if any for those two
>> names or even if you want to propose a new name here.
>> >>
>> >> message RevocableInfo {
>> >>     message ThrottleInfo {}
>> >>
>> >>     // If set, indicates that the resources may be throttled at
>> >>     // any time. Throttle-able resoruces can be used for tasks
>> >>     // that do not have strict performance requirements and are
>> >>     // capable of handling being throttled.
>> >>     optional ThrottleInfo throttle_info = 1;
>> >>   }
>> >>
>> >> 在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道：
>> >>>
>> >>> The patches are updated accordingly; JIRA: MESOS-3888 , RR:
>> https://reviews.apache.org/r/40375/ .
>> >>>
>> >>> Thanks
>> >>> klaus
>> >>>
>> >>>> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler
>> wrote:
>> >>>> Hey folks,
>> >>>>
>> >>>> In the resource allocation working group we've been looking into a
>> few projects that will make the allocator able to offer out resources as
>> revocable. For example:
>> >>>>
>> >>>> -We'll want to eventually allocate resources as revocable _by
>> default_, only allowing non-revocable when there are guarantees put in
>> place (static reservations or quota).
>> >>>>
>> >>>> -On the path to revocable by default, we can incrementally start to
>> offer certain resources as revocable. Consider when quota is set but the
>> role isn't using all of the quota. The unallocated quota can be offered to
>> other roles, but it should be revocable because we may revoke them should
>> the quota'ed role want to use the resources. Unused reservations fall into
>> a similar category.
>> >>>>
>> >>>> -Going revocable by default also allows us to enforce fairness in a
>> dynamically changing cluster by revoking resources as weights are changed,
>> frameworks are added or removed, etc.
>> >>>>
>> >>>> In this context, "revocable" means that the resources may be taken
>> away and the container will be destroyed. The meaning of "revocable" in the
>> context of usage oversubscription includes this, but also the container may
>> experience a throttling (e.g. lower cpu shares, less network priority, etc).
>> >>>>
>> >>>> For this reason, and because we internally need to distinguish
>> revocable resources between the those that are generated by usage
>> oversubscription and those that are generated by the allocator, we're
>> thinking of the following change to the API:
>> >>>>
>> >>>>
>> >>>>
>> >>>> -  message RevocableInfo {}
>> >>>> +  message RevocableInfo {
>> >>>> +    message ThrottleInfo {}
>> >>>> +
>> >>>> +    // If set, indicates that the resources may be throttled at
>> >>>> +    // any time. Throttle-able resoruces can be used for tasks
>> >>>> +    // that do not have strict performance requirements and are
>> >>>> +    // capable of handling being throttled.
>> >>>> +    optional ThrottleInfo throttle_info;
>> >>>> +  }
>> >>>>
>> >>>>    // If this is set, the resources are revocable, i.e., any tasks or
>> >>>> -  // executors launched using these resources could get preempted or
>> >>>> -  // throttled at any time. This could be used by frameworks to run
>> >>>> -  // best effort tasks that do not need strict uptime or performance
>> >>>> +  // executors launched using these resources could be terminated at
>> >>>> +  // any time. This could be used by frameworks to run
>> >>>> +  // best effort tasks that do not need strict uptime
>> >>>>    // guarantees. Note that if this is set, 'disk' or 'reservation'
>> >>>>    // cannot be set.
>> >>>>    optional RevocableInfo revocable = 9;
>> >>>>
>> >>>>
>> >>>>
>> >>>> Essentially we want to distinguish between revocable and revocable +
>> throttle-able. This is because usage-oversubscription generates
>> throttle-able revocable resources, whereas the allocator does not. This
>> also solves our problem of distinguishing between these two kinds of
>> revocable resources internally.
>> >>>>
>> >>>> Feedback welcome!
>> >>>>
>> >>>> Ben
>> >>>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Mesos Resource Allocation Working Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mesos-allocation+unsubscribe@googlegroups.com.
> To post to this group, send email to mesos-allocation@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com
> <https://groups.google.com/d/msgid/mesos-allocation/CAGmKMfU4NJpY8NP6PVu9g-ebfi7Q9UseEdPUc0XkL1qqqqm75g%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

Re: RFC: RevocableInfo Changes

Posted by Klaus Ma <kl...@gmail.com>.

Here's some input :).

If throttling is tolerable but preemption is not, how would that be
expressed? (Is that supported?)
[Klaus]: It's not supported; only revocable resources has this attribute:
non-throttleable or throttleable. The throttleable revocable resources is
reported by ResourceEstimator which means the resources maybe throttled by
its original owner.

How does this work with the QoS controller? Will there be a new correction
type to indicate throttling, or does throttling happen "behind the agent's
back"?
[Klaus]: The QoSController/ResourceEstimator only manages throttleable
revocable resources; the others resources (regular resources and
non-throttleable revocable resources) are managed by allocator. The
"manage" means generation and destroy/eviction. Regarding "throttling
happen", good question. I think the throttling will dependent on
containers, let me double check it :).

If any comments, please let me know.

----
Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
Platform OpenSource Technology, STG, IBM GCG
+86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me

On Sat, Mar 19, 2016 at 11:15 PM, <co...@gmail.com> wrote:

> Thanks for the good explanations so far Ben and Klaus.  Apologies if you
> guys already covered these questions in the meeting:
>
> If throttling is tolerable but preemption is not, how would that be
> expressed? (Is that supported?)
>
> How does this work with the QoS controller? Will there be a new correction
> type to indicate throttling, or does throttling happen "behind the agent's
> back"?
>
> Thanks,
> --
> Connor
>
> > On Mar 19, 2016, at 04:01, Klaus Ma <kl...@gmail.com> wrote:
> >
> > @team, in the latest meeting, we agree to keep current name ThrottleInfo.
> >
> > If any more comments, please let me know.
> >
> >> On Wednesday, March 16, 2016 at 9:32:37 PM UTC+8, Guangya Liu wrote:
> >> Also please show your comments if any for the name here, the current
> name is ThrottleInfo, in Kubernetes resources qos design document, they are
> using scavenging as the key work for such behaviour, so a possible name
> here could be ScavengeInfo , please show your comments if any for those two
> names or even if you want to propose a new name here.
> >>
> >> message RevocableInfo {
> >>     message ThrottleInfo {}
> >>
> >>     // If set, indicates that the resources may be throttled at
> >>     // any time. Throttle-able resoruces can be used for tasks
> >>     // that do not have strict performance requirements and are
> >>     // capable of handling being throttled.
> >>     optional ThrottleInfo throttle_info = 1;
> >>   }
> >>
> >> 在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道：
> >>>
> >>> The patches are updated accordingly; JIRA: MESOS-3888 , RR:
> https://reviews.apache.org/r/40375/ .
> >>>
> >>> Thanks
> >>> klaus
> >>>
> >>>> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler
> wrote:
> >>>> Hey folks,
> >>>>
> >>>> In the resource allocation working group we've been looking into a
> few projects that will make the allocator able to offer out resources as
> revocable. For example:
> >>>>
> >>>> -We'll want to eventually allocate resources as revocable _by
> default_, only allowing non-revocable when there are guarantees put in
> place (static reservations or quota).
> >>>>
> >>>> -On the path to revocable by default, we can incrementally start to
> offer certain resources as revocable. Consider when quota is set but the
> role isn't using all of the quota. The unallocated quota can be offered to
> other roles, but it should be revocable because we may revoke them should
> the quota'ed role want to use the resources. Unused reservations fall into
> a similar category.
> >>>>
> >>>> -Going revocable by default also allows us to enforce fairness in a
> dynamically changing cluster by revoking resources as weights are changed,
> frameworks are added or removed, etc.
> >>>>
> >>>> In this context, "revocable" means that the resources may be taken
> away and the container will be destroyed. The meaning of "revocable" in the
> context of usage oversubscription includes this, but also the container may
> experience a throttling (e.g. lower cpu shares, less network priority, etc).
> >>>>
> >>>> For this reason, and because we internally need to distinguish
> revocable resources between the those that are generated by usage
> oversubscription and those that are generated by the allocator, we're
> thinking of the following change to the API:
> >>>>
> >>>>
> >>>>
> >>>> -  message RevocableInfo {}
> >>>> +  message RevocableInfo {
> >>>> +    message ThrottleInfo {}
> >>>> +
> >>>> +    // If set, indicates that the resources may be throttled at
> >>>> +    // any time. Throttle-able resoruces can be used for tasks
> >>>> +    // that do not have strict performance requirements and are
> >>>> +    // capable of handling being throttled.
> >>>> +    optional ThrottleInfo throttle_info;
> >>>> +  }
> >>>>
> >>>>    // If this is set, the resources are revocable, i.e., any tasks or
> >>>> -  // executors launched using these resources could get preempted or
> >>>> -  // throttled at any time. This could be used by frameworks to run
> >>>> -  // best effort tasks that do not need strict uptime or performance
> >>>> +  // executors launched using these resources could be terminated at
> >>>> +  // any time. This could be used by frameworks to run
> >>>> +  // best effort tasks that do not need strict uptime
> >>>>    // guarantees. Note that if this is set, 'disk' or 'reservation'
> >>>>    // cannot be set.
> >>>>    optional RevocableInfo revocable = 9;
> >>>>
> >>>>
> >>>>
> >>>> Essentially we want to distinguish between revocable and revocable +
> throttle-able. This is because usage-oversubscription generates
> throttle-able revocable resources, whereas the allocator does not. This
> also solves our problem of distinguishing between these two kinds of
> revocable resources internally.
> >>>>
> >>>> Feedback welcome!
> >>>>
> >>>> Ben
> >>>>
>

Re: RFC: RevocableInfo Changes

Posted by co...@gmail.com.

Thanks for the good explanations so far Ben and Klaus.  Apologies if you guys already covered these questions in the meeting:

If throttling is tolerable but preemption is not, how would that be expressed? (Is that supported?)

How does this work with the QoS controller? Will there be a new correction type to indicate throttling, or does throttling happen "behind the agent's back"?

Thanks,
--
Connor

> On Mar 19, 2016, at 04:01, Klaus Ma <kl...@gmail.com> wrote:
> 
> @team, in the latest meeting, we agree to keep current name ThrottleInfo.
> 
> If any more comments, please let me know.
> 
>> On Wednesday, March 16, 2016 at 9:32:37 PM UTC+8, Guangya Liu wrote:
>> Also please show your comments if any for the name here, the current name is ThrottleInfo, in Kubernetes resources qos design document, they are using scavenging as the key work for such behaviour, so a possible name here could be ScavengeInfo , please show your comments if any for those two names or even if you want to propose a new name here.
>> 
>> message RevocableInfo {
>>     message ThrottleInfo {}
>> 
>>     // If set, indicates that the resources may be throttled at
>>     // any time. Throttle-able resoruces can be used for tasks
>>     // that do not have strict performance requirements and are
>>     // capable of handling being throttled.
>>     optional ThrottleInfo throttle_info = 1;
>>   }
>> 
>> 在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道：
>>> 
>>> The patches are updated accordingly; JIRA: MESOS-3888 , RR: https://reviews.apache.org/r/40375/ .
>>> 
>>> Thanks
>>> klaus
>>> 
>>>> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler wrote:
>>>> Hey folks,
>>>> 
>>>> In the resource allocation working group we've been looking into a few projects that will make the allocator able to offer out resources as revocable. For example:
>>>> 
>>>> -We'll want to eventually allocate resources as revocable _by default_, only allowing non-revocable when there are guarantees put in place (static reservations or quota).
>>>> 
>>>> -On the path to revocable by default, we can incrementally start to offer certain resources as revocable. Consider when quota is set but the role isn't using all of the quota. The unallocated quota can be offered to other roles, but it should be revocable because we may revoke them should the quota'ed role want to use the resources. Unused reservations fall into a similar category.
>>>> 
>>>> -Going revocable by default also allows us to enforce fairness in a dynamically changing cluster by revoking resources as weights are changed, frameworks are added or removed, etc.
>>>> 
>>>> In this context, "revocable" means that the resources may be taken away and the container will be destroyed. The meaning of "revocable" in the context of usage oversubscription includes this, but also the container may experience a throttling (e.g. lower cpu shares, less network priority, etc).
>>>> 
>>>> For this reason, and because we internally need to distinguish revocable resources between the those that are generated by usage oversubscription and those that are generated by the allocator, we're thinking of the following change to the API:
>>>> 
>>>> 
>>>> 
>>>> -  message RevocableInfo {}
>>>> +  message RevocableInfo {
>>>> +    message ThrottleInfo {}
>>>> +
>>>> +    // If set, indicates that the resources may be throttled at
>>>> +    // any time. Throttle-able resoruces can be used for tasks
>>>> +    // that do not have strict performance requirements and are
>>>> +    // capable of handling being throttled.
>>>> +    optional ThrottleInfo throttle_info;
>>>> +  }
>>>> 
>>>>    // If this is set, the resources are revocable, i.e., any tasks or
>>>> -  // executors launched using these resources could get preempted or
>>>> -  // throttled at any time. This could be used by frameworks to run
>>>> -  // best effort tasks that do not need strict uptime or performance
>>>> +  // executors launched using these resources could be terminated at
>>>> +  // any time. This could be used by frameworks to run
>>>> +  // best effort tasks that do not need strict uptime
>>>>    // guarantees. Note that if this is set, 'disk' or 'reservation'
>>>>    // cannot be set.
>>>>    optional RevocableInfo revocable = 9;
>>>> 
>>>> 
>>>> 
>>>> Essentially we want to distinguish between revocable and revocable + throttle-able. This is because usage-oversubscription generates throttle-able revocable resources, whereas the allocator does not. This also solves our problem of distinguishing between these two kinds of revocable resources internally.
>>>> 
>>>> Feedback welcome!
>>>> 
>>>> Ben
>>>>

Re: RFC: RevocableInfo Changes

Posted by Klaus Ma <kl...@gmail.com>.

@team, in the latest meeting, we agree to keep current name *ThrottleInfo.*

If any more comments, please let me know.

On Wednesday, March 16, 2016 at 9:32:37 PM UTC+8, Guangya Liu wrote:
>
> Also please show your comments if any for the name here, the current name 
> is *ThrottleInfo*, in Kubernetes resources qos design document, they are 
> using scavenging as the key work for such behaviour, so a possible name 
> here could be *ScavengeInfo , *please show your comments if any for those 
> two names or even if you want to propose a new name here.
>
> message RevocableInfo {
>     *message ThrottleInfo {}*
>
> *    // If set, indicates that the resources may be throttled at*
> *    // any time. Throttle-able resoruces can be used for tasks*
> *    // that do not have strict performance requirements and are*
> *    // capable of handling being throttled.*
> *    optional ThrottleInfo throttle_info = 1;*
>   }
>
> 在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道：
>>
>> The patches are updated accordingly; JIRA: MESOS-3888 
>> <https://issues.apache.org/jira/browse/MESOS-3888> , RR: 
>> https://reviews.apache.org/r/40375/ .
>>
>> Thanks
>> klaus
>>
>> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler wrote:
>>>
>>> Hey folks,
>>>
>>> In the resource allocation working group we've been looking into a few 
>>> projects that will make the allocator able to offer out resources as 
>>> revocable. For example:
>>>
>>> -We'll want to eventually allocate resources as revocable _by default_, 
>>> only allowing non-revocable when there are guarantees put in place (static 
>>> reservations or quota).
>>>
>>> -On the path to revocable by default, we can incrementally start to 
>>> offer certain resources as revocable. Consider when quota is set but the 
>>> role isn't using all of the quota. The unallocated quota can be offered to 
>>> other roles, but it should be revocable because we may revoke them should 
>>> the quota'ed role want to use the resources. Unused reservations fall into 
>>> a similar category.
>>>
>>> -Going revocable by default also allows us to enforce fairness in a 
>>> dynamically changing cluster by revoking resources as weights are changed, 
>>> frameworks are added or removed, etc.
>>>
>>> In this context, "revocable" means that the resources may be taken away 
>>> and the container will be destroyed. The meaning of "revocable" in the 
>>> context of usage oversubscription includes this, but also the container may 
>>> experience a throttling (e.g. lower cpu shares, less network priority, etc).
>>>
>>> For this reason, and because we internally need to distinguish revocable 
>>> resources between the those that are generated by usage oversubscription 
>>> and those that are generated by the allocator, we're thinking of the 
>>> following change to the API:
>>>
>>>
>>>
>>> -  message RevocableInfo {}
>>> +  message RevocableInfo {
>>> +    message ThrottleInfo {}
>>> +
>>> +    // If set, indicates that the resources may be throttled at
>>> +    // any time. Throttle-able resoruces can be used for tasks
>>> +    // that do not have strict performance requirements and are
>>> +    // capable of handling being throttled.
>>> +    optional ThrottleInfo throttle_info;
>>> +  }
>>>
>>>    // If this is set, the resources are revocable, i.e., any tasks or
>>> -  // executors launched using these resources could get preempted or
>>> -  // throttled at any time. This could be used by frameworks to run
>>> -  // best effort tasks that do not need strict uptime or performance
>>> +  // executors launched using these resources could be terminated at
>>> +  // any time. This could be used by frameworks to run
>>> +  // best effort tasks that do not need strict uptime
>>>    // guarantees. Note that if this is set, 'disk' or 'reservation'
>>>    // cannot be set.
>>>    optional RevocableInfo revocable = 9;
>>>
>>>
>>>
>>> Essentially we want to distinguish between revocable and revocable + 
>>> throttle-able. This is because usage-oversubscription generates 
>>> throttle-able revocable resources, whereas the allocator does not. This 
>>> also solves our problem of distinguishing between these two kinds of 
>>> revocable resources internally.
>>>
>>> Feedback welcome!
>>>
>>> Ben
>>>
>>>

Re: RFC: RevocableInfo Changes

Posted by Guangya Liu <gy...@gmail.com>.

Also please show your comments if any for the name here, the current name 
is *ThrottleInfo*, in Kubernetes resources qos design document, they are 
using scavenging as the key work for such behaviour, so a possible name 
here could be *ScavengeInfo , *please show your comments if any for those 
two names or even if you want to propose a new name here.

message RevocableInfo {
    *message ThrottleInfo {}*

*    // If set, indicates that the resources may be throttled at*
*    // any time. Throttle-able resoruces can be used for tasks*
*    // that do not have strict performance requirements and are*
*    // capable of handling being throttled.*
*    optional ThrottleInfo throttle_info = 1;*
  }

在 2016年3月16日星期三 UTC+8上午10:24:14，Klaus Ma写道：
>
> The patches are updated accordingly; JIRA: MESOS-3888 
> <https://issues.apache.org/jira/browse/MESOS-3888> , RR: 
> https://reviews.apache.org/r/40375/ .
>
> Thanks
> klaus
>
> On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler wrote:
>>
>> Hey folks,
>>
>> In the resource allocation working group we've been looking into a few 
>> projects that will make the allocator able to offer out resources as 
>> revocable. For example:
>>
>> -We'll want to eventually allocate resources as revocable _by default_, 
>> only allowing non-revocable when there are guarantees put in place (static 
>> reservations or quota).
>>
>> -On the path to revocable by default, we can incrementally start to offer 
>> certain resources as revocable. Consider when quota is set but the role 
>> isn't using all of the quota. The unallocated quota can be offered to other 
>> roles, but it should be revocable because we may revoke them should the 
>> quota'ed role want to use the resources. Unused reservations fall into a 
>> similar category.
>>
>> -Going revocable by default also allows us to enforce fairness in a 
>> dynamically changing cluster by revoking resources as weights are changed, 
>> frameworks are added or removed, etc.
>>
>> In this context, "revocable" means that the resources may be taken away 
>> and the container will be destroyed. The meaning of "revocable" in the 
>> context of usage oversubscription includes this, but also the container may 
>> experience a throttling (e.g. lower cpu shares, less network priority, etc).
>>
>> For this reason, and because we internally need to distinguish revocable 
>> resources between the those that are generated by usage oversubscription 
>> and those that are generated by the allocator, we're thinking of the 
>> following change to the API:
>>
>>
>>
>> -  message RevocableInfo {}
>> +  message RevocableInfo {
>> +    message ThrottleInfo {}
>> +
>> +    // If set, indicates that the resources may be throttled at
>> +    // any time. Throttle-able resoruces can be used for tasks
>> +    // that do not have strict performance requirements and are
>> +    // capable of handling being throttled.
>> +    optional ThrottleInfo throttle_info;
>> +  }
>>
>>    // If this is set, the resources are revocable, i.e., any tasks or
>> -  // executors launched using these resources could get preempted or
>> -  // throttled at any time. This could be used by frameworks to run
>> -  // best effort tasks that do not need strict uptime or performance
>> +  // executors launched using these resources could be terminated at
>> +  // any time. This could be used by frameworks to run
>> +  // best effort tasks that do not need strict uptime
>>    // guarantees. Note that if this is set, 'disk' or 'reservation'
>>    // cannot be set.
>>    optional RevocableInfo revocable = 9;
>>
>>
>>
>> Essentially we want to distinguish between revocable and revocable + 
>> throttle-able. This is because usage-oversubscription generates 
>> throttle-able revocable resources, whereas the allocator does not. This 
>> also solves our problem of distinguishing between these two kinds of 
>> revocable resources internally.
>>
>> Feedback welcome!
>>
>> Ben
>>
>>

Re: RFC: RevocableInfo Changes

Posted by Klaus Ma <kl...@gmail.com>.

The patches are updated accordingly; JIRA: MESOS-3888 
<https://issues.apache.org/jira/browse/MESOS-3888> , RR: 
https://reviews.apache.org/r/40375/ .

Thanks
klaus

On Saturday, March 12, 2016 at 11:09:46 AM UTC+8, Benjamin Mahler wrote:
>
> Hey folks,
>
> In the resource allocation working group we've been looking into a few 
> projects that will make the allocator able to offer out resources as 
> revocable. For example:
>
> -We'll want to eventually allocate resources as revocable _by default_, 
> only allowing non-revocable when there are guarantees put in place (static 
> reservations or quota).
>
> -On the path to revocable by default, we can incrementally start to offer 
> certain resources as revocable. Consider when quota is set but the role 
> isn't using all of the quota. The unallocated quota can be offered to other 
> roles, but it should be revocable because we may revoke them should the 
> quota'ed role want to use the resources. Unused reservations fall into a 
> similar category.
>
> -Going revocable by default also allows us to enforce fairness in a 
> dynamically changing cluster by revoking resources as weights are changed, 
> frameworks are added or removed, etc.
>
> In this context, "revocable" means that the resources may be taken away 
> and the container will be destroyed. The meaning of "revocable" in the 
> context of usage oversubscription includes this, but also the container may 
> experience a throttling (e.g. lower cpu shares, less network priority, etc).
>
> For this reason, and because we internally need to distinguish revocable 
> resources between the those that are generated by usage oversubscription 
> and those that are generated by the allocator, we're thinking of the 
> following change to the API:
>
>
>
> -  message RevocableInfo {}
> +  message RevocableInfo {
> +    message ThrottleInfo {}
> +
> +    // If set, indicates that the resources may be throttled at
> +    // any time. Throttle-able resoruces can be used for tasks
> +    // that do not have strict performance requirements and are
> +    // capable of handling being throttled.
> +    optional ThrottleInfo throttle_info;
> +  }
>
>    // If this is set, the resources are revocable, i.e., any tasks or
> -  // executors launched using these resources could get preempted or
> -  // throttled at any time. This could be used by frameworks to run
> -  // best effort tasks that do not need strict uptime or performance
> +  // executors launched using these resources could be terminated at
> +  // any time. This could be used by frameworks to run
> +  // best effort tasks that do not need strict uptime
>    // guarantees. Note that if this is set, 'disk' or 'reservation'
>    // cannot be set.
>    optional RevocableInfo revocable = 9;
>
>
>
> Essentially we want to distinguish between revocable and revocable + 
> throttle-able. This is because usage-oversubscription generates 
> throttle-able revocable resources, whereas the allocator does not. This 
> also solves our problem of distinguishing between these two kinds of 
> revocable resources internally.
>
> Feedback welcome!
>
> Ben
>
>