You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by "Iwanowski, Maciej" <ma...@intel.com> on 2018/11/29 19:58:36 UTC
Sidecar for dynamic resource allocation in Mesos
Hello,
My team at Intel has recently opensourced a Mesos-related project: Orchestration-aware Workload Collocation Agent (https://github.com/intel/owca). Goal of the project is to: "dynamically manage platform isolation mechanisms to ensure that high priority jobs meet their service level objective (SLO) and best-effort jobs effectively utilize as many idle resources as possible" (https://github.com/intel/owca/blob/master/docs/OWCA_Architecture_v1.5.pdf). As we are aiming at integrating other orchestration platform we decided to build a sidecar rather than a Mesos module. Library is written in Python because of ubiquity of the language in Machine Learning and Big Data environments - we are planning to experiment with various resource allocation and anomaly detection algorithms.
I hope that you will find the idea interesting or even, one day, deployable 😉
Maciej
an engineer
--------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN.
Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek
przegladanie lub rozpowszechnianie jest zabronione.
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by
others is strictly prohibited.
Re: Sidecar for dynamic resource allocation in Mesos
Posted by Vinod Kone <vi...@apache.org>.
This is great to see. Thanks for sharing!
On Thu, Nov 29, 2018 at 1:59 PM Iwanowski, Maciej <
maciej.iwanowski@intel.com> wrote:
> Hello,
>
> My team at Intel has recently opensourced a Mesos-related project:
> Orchestration-aware Workload Collocation Agent (
> https://github.com/intel/owca). Goal of the project is to: "dynamically
> manage platform isolation mechanisms to ensure that high priority jobs meet
> their service level objective (SLO) and best-effort jobs effectively
> utilize as many idle resources as possible" (
> https://github.com/intel/owca/blob/master/docs/OWCA_Architecture_v1.5.pdf).
> As we are aiming at integrating other orchestration platform we decided to
> build a sidecar rather than a Mesos module. Library is written in Python
> because of ubiquity of the language in Machine Learning and Big Data
> environments - we are planning to experiment with various resource
> allocation and anomaly detection algorithms.
>
> I hope that you will find the idea interesting or even, one day,
> deployable 😉
>
> Maciej
> an engineer
>
> --------------------------------------------------------------------
>
> Intel Technology Poland sp. z o.o.
> ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII
> Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP
> 957-07-52-316 | Kapital zakladowy 200.000 PLN.
>
> Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego
> adresata i moze zawierac informacje poufne. W razie przypadkowego
> otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej
> usuniecie; jakiekolwiek
> przegladanie lub rozpowszechnianie jest zabronione.
> This e-mail and any attachments may contain confidential material for the
> sole use of the intended recipient(s). If you are not the intended
> recipient, please contact the sender and delete all copies; any review or
> distribution by
> others is strictly prohibited.
>
Re: Sidecar for dynamic resource allocation in Mesos
Posted by Radoslaw Gruchalski <ra...@gruchalski.com>.
Cool name, "owca" is "sheep" in Polish.
--
Best regards,
Radek Gruchalski
> On Nov 29, 2018, at 20:58, Iwanowski, Maciej <ma...@intel.com> wrote:
>
> Hello,
>
> My team at Intel has recently opensourced a Mesos-related project: Orchestration-aware Workload Collocation Agent (https://github.com/intel/owca). Goal of the project is to: "dynamically manage platform isolation mechanisms to ensure that high priority jobs meet their service level objective (SLO) and best-effort jobs effectively utilize as many idle resources as possible" (https://github.com/intel/owca/blob/master/docs/OWCA_Architecture_v1.5.pdf). As we are aiming at integrating other orchestration platform we decided to build a sidecar rather than a Mesos module. Library is written in Python because of ubiquity of the language in Machine Learning and Big Data environments - we are planning to experiment with various resource allocation and anomaly detection algorithms.
>
> I hope that you will find the idea interesting or even, one day, deployable 😉
>
> Maciej
> an engineer
>
> --------------------------------------------------------------------
>
> Intel Technology Poland sp. z o.o.
> ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN.
>
> Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek
> przegladanie lub rozpowszechnianie jest zabronione.
> This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by
> others is strictly prohibited.
RE: Sidecar for dynamic resource allocation in Mesos
Posted by "Iwanowski, Maciej" <ma...@intel.com>.
Hi Abel,
> -----Original Message-----
> From: Abel Souza <ab...@cs.umu.se>
> Sent: Thursday, November 29, 2018 1:01 PM
> Subject: Re: Sidecar for dynamic resource allocation in Mesos
>
> I've been working on a paper doing exactly that, but in a different
> context. I can share a draft version of it if you like.
I would love to read it if you can share it 😊
Maciej.
--------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN.
Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek
przegladanie lub rozpowszechnianie jest zabronione.
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by
others is strictly prohibited.
Re: Sidecar for dynamic resource allocation in Mesos
Posted by Abel Souza <ab...@cs.umu.se>.
Interesting.
I've been working on a paper doing exactly that, but in a different
context. I can share a draft version of it if you like.
Best,
/Abel
On 11/29/18 20:58, Iwanowski, Maciej wrote:
> Hello,
>
> My team at Intel has recently opensourced a Mesos-related project: Orchestration-aware Workload Collocation Agent (https://github.com/intel/owca). Goal of the project is to: "dynamically manage platform isolation mechanisms to ensure that high priority jobs meet their service level objective (SLO) and best-effort jobs effectively utilize as many idle resources as possible" (https://github.com/intel/owca/blob/master/docs/OWCA_Architecture_v1.5.pdf). As we are aiming at integrating other orchestration platform we decided to build a sidecar rather than a Mesos module. Library is written in Python because of ubiquity of the language in Machine Learning and Big Data environments - we are planning to experiment with various resource allocation and anomaly detection algorithms.
>
> I hope that you will find the idea interesting or even, one day, deployable 😉
>
> Maciej
> an engineer
>
> --------------------------------------------------------------------
>
> Intel Technology Poland sp. z o.o.
> ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN.
>
> Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek
> przegladanie lub rozpowszechnianie jest zabronione.
> This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by
> others is strictly prohibited.