You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@aurora.apache.org by Meghdoot bhattacharya <me...@yahoo.com.INVALID> on 2018/05/02 17:40:54 UTC
Scheduling improvements followups
Just wanted to follow up on the current state.
1. Is there any work on implementing soft constraints?
2. How much work (and feasibility?) would be to add other filter conditions to sort the list of hosts to schedule even without a full fitness score kind of model. So, say the default algorithm picks a set of probable hosts but we apply external filters to sort it. Example even though cpu/mem is available on the host based on network or iops (as read from a real time monitor) we choose to avoid them.
Good write up on what K8 is providing today with options to override.
https://thenewstack.io/implementing-advanced-scheduling-techniques-with-kubernetes/
Thx
Re: Scheduling improvements followups
Posted by Meghdoot bhattacharya <me...@yahoo.com.INVALID>.
Bill, much appreciated and we will definitely take up on your guidance offer. Most likely Renan would follow up on this.
Thx
On May 2, 2018, at 1:37 PM, Bill Farner <wf...@apache.org> wrote:
>>
>> How much work (and feasibility?) would be to add other filter conditions
>
>
> In my opinion, relatively little work. An extension point exists as a thin
> interface [1], injectable via a module [2]. OfferOrder [3] may look like a
> promising alternative, but i see that as a less flexible option.
> Regardless of the path taken, i advise starting with customization in a
> downstream fork/extension of the apache codebase.
>
> I will be happy to offer high-level guidance to anyone looking to pursue
> this. You may also be interested in a commit of mine on a branch [4] that
> may be partially reusable as an approach to doing this while maintaining
> high scheduling throughput.
>
> [1] https://github.com/apache/aurora/blob/a3d596ead62404300edbbab1179476
> 410c8284ad/src/main/java/org/apache/aurora/scheduler/
> offers/OfferSet.java#L35-L40
> [2] https://github.com/apache/aurora/blob/a3d596ead62404300edbbab1179476
> 410c8284ad/src/main/java/org/apache/aurora/scheduler/
> offers/OfferManagerModule.java#L98-L101
> [3] https://github.com/apache/aurora/blob/a3d596ead62404300edbbab1179476
> 410c8284ad/src/main/java/org/apache/aurora/scheduler/
> offers/OfferOrder.java#L16-L25
> [4]
> https://github.com/wfarner/aurora/commit/f77d79a2d01c3b5b34f11b812d5dcff2789e0766
>
> On Wed, May 2, 2018 at 10:40 AM, Meghdoot bhattacharya <
> meghdoot_b@yahoo.com.invalid> wrote:
>
>> Just wanted to follow up on the current state.
>>
>> 1. Is there any work on implementing soft constraints?
>>
>> 2. How much work (and feasibility?) would be to add other filter
>> conditions to sort the list of hosts to schedule even without a full
>> fitness score kind of model. So, say the default algorithm picks a set of
>> probable hosts but we apply external filters to sort it. Example even
>> though cpu/mem is available on the host based on network or iops (as read
>> from a real time monitor) we choose to avoid them.
>>
>> Good write up on what K8 is providing today with options to override.
>>
>> https://thenewstack.io/implementing-advanced-scheduling-techniques-with-
>> kubernetes/
>>
>> Thx
Re: Scheduling improvements followups
Posted by Bill Farner <wf...@apache.org>.
>
> How much work (and feasibility?) would be to add other filter conditions
In my opinion, relatively little work. An extension point exists as a thin
interface [1], injectable via a module [2]. OfferOrder [3] may look like a
promising alternative, but i see that as a less flexible option.
Regardless of the path taken, i advise starting with customization in a
downstream fork/extension of the apache codebase.
I will be happy to offer high-level guidance to anyone looking to pursue
this. You may also be interested in a commit of mine on a branch [4] that
may be partially reusable as an approach to doing this while maintaining
high scheduling throughput.
[1] https://github.com/apache/aurora/blob/a3d596ead62404300edbbab1179476
410c8284ad/src/main/java/org/apache/aurora/scheduler/
offers/OfferSet.java#L35-L40
[2] https://github.com/apache/aurora/blob/a3d596ead62404300edbbab1179476
410c8284ad/src/main/java/org/apache/aurora/scheduler/
offers/OfferManagerModule.java#L98-L101
[3] https://github.com/apache/aurora/blob/a3d596ead62404300edbbab1179476
410c8284ad/src/main/java/org/apache/aurora/scheduler/
offers/OfferOrder.java#L16-L25
[4]
https://github.com/wfarner/aurora/commit/f77d79a2d01c3b5b34f11b812d5dcff2789e0766
On Wed, May 2, 2018 at 10:40 AM, Meghdoot bhattacharya <
meghdoot_b@yahoo.com.invalid> wrote:
> Just wanted to follow up on the current state.
>
> 1. Is there any work on implementing soft constraints?
>
> 2. How much work (and feasibility?) would be to add other filter
> conditions to sort the list of hosts to schedule even without a full
> fitness score kind of model. So, say the default algorithm picks a set of
> probable hosts but we apply external filters to sort it. Example even
> though cpu/mem is available on the host based on network or iops (as read
> from a real time monitor) we choose to avoid them.
>
> Good write up on what K8 is providing today with options to override.
>
> https://thenewstack.io/implementing-advanced-scheduling-techniques-with-
> kubernetes/
>
> Thx