You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Qian Zhang <zh...@gmail.com> on 2015/06/06 12:51:16 UTC
Can Mesos master offer resources to multiple frameworks simultaneously?
Hi,
I am new to Mesos, and I'd like to know if there are a lot resources in the
Mesos cluster, how will Mesos master offer these resources to the multiple
frameworks? I guess there can be two ways:
1. Mesos master offers all the resources to all the frameworks
simultaneously.
2. Mesos master offers resources to one framework at a time, e.g., it
offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to
f2, ...
If it is 1, then I'd like to know how Mesos master resolves the conflicts,
e.g., multiple frameworks accept the same resource.
If it is 2, then I see it is actually a serial process since Mesos master
handle the frameworks one by one, then what is advantage of Mesos against
traditional monolithic resource scheduler?
Thanks,
Qian
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Qian Zhang <zh...@gmail.com>.
Thanks Adam! It is clear to me now :-)
2015-06-12 7:49 GMT+08:00 Adam Bordelon <ad...@mesosphere.io>:
> 4. By default, Mesos will not revoke ("rescind") an *un*used offer being
> held by a framework, but you can enable such a timeout by specifying the
> `--offer_timeout` flag on the master.
>
> On Thu, Jun 11, 2015 at 4:48 PM, Adam Bordelon <ad...@mesosphere.io> wrote:
>
>> 1. The modularized allocator will still be a C++ interface, but you could
>> just create a C++ wrapper around whatever Python/Go/Java/etc.
>> implementation that you prefer.
>>
>> Your assessment of 2&3 sounds correct.
>>
>> 4. By default, Mesos will not revoke ("rescind") an used offer being held
>> by a framework, but you can enable such a timeout by specifying the
>> `--offer_timeout` flag on the master.
>>
>> On Thu, Jun 11, 2015 at 1:41 AM, baotiao <ba...@gmail.com> wrote:
>>
>>> Hi Qian Zhang
>>>
>>> I can answer the fourth question.
>>>
>>> if a framework has not responded to an offer for a sufficiently long
>>> time, Mesos rescinds the offer and re-offers the resources to other
>>> frameworks.
>>> You cant get it
>>>
>>> I am not clear in how Mesos divide all resources into multiple subsets?
>>>
>>> ----------------------------------------
>>> 陈宗志
>>>
>>> Blog: baotiao.github.io
>>>
>>>
>>>
>>>
>>> On Jun 11, 2015, at 08:35, Qian Zhang <zh...@gmail.com> wrote:
>>>
>>> Thanks Alex.
>>>
>>> For 1. I understand currently the only choice is C++. However, as Adam
>>> mentioned, true pluggable allocator modules (MESOS-2160
>>> <https://issues.apache.org/jira/browse/MESOS-2160>) are landing in
>>> Mesos 0.23, so at that time, I assume we will have more choices, right?
>>>
>>> For 2 and 3, my understanding is Mesos allocator will partition all the
>>> available resources into multiple subsets, and there is no overlap between
>>> these subsets (i.e., a single resource can only be in one subset), and then
>>> offer these subsets to multiple frameworks (e.g., offer subset1 to
>>> framework1, offer subset2 to framework2, and so on), and it is up to each
>>> framework's scheduler to determine if it accept the resource to launch task
>>> or reject it. In this way, each framework's scheduler can actually make
>>> scheduling decision independently since they will never compete for the
>>> same resource.
>>>
>>> If my understanding is correct, then I have one more question:
>>> 4. What if it takes very long time (e.g., mins or hours) for a
>>> framework's scheduler to make the scheduling decision? Does that mean
>>> during this long period, the resources offered to this framework will not
>>> be used by any other frameworks? Is there a timeout for the
>>> framework's scheduler to make the scheduling decision? So when the timeout
>>> is reached, the resources offered to it will be revoked by Mesos allocator
>>> and can be offered to another framework.
>>>
>>>
>>>
>>
>
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Adam Bordelon <ad...@mesosphere.io>.
4. By default, Mesos will not revoke ("rescind") an *un*used offer being
held by a framework, but you can enable such a timeout by specifying the
`--offer_timeout` flag on the master.
On Thu, Jun 11, 2015 at 4:48 PM, Adam Bordelon <ad...@mesosphere.io> wrote:
> 1. The modularized allocator will still be a C++ interface, but you could
> just create a C++ wrapper around whatever Python/Go/Java/etc.
> implementation that you prefer.
>
> Your assessment of 2&3 sounds correct.
>
> 4. By default, Mesos will not revoke ("rescind") an used offer being held
> by a framework, but you can enable such a timeout by specifying the
> `--offer_timeout` flag on the master.
>
> On Thu, Jun 11, 2015 at 1:41 AM, baotiao <ba...@gmail.com> wrote:
>
>> Hi Qian Zhang
>>
>> I can answer the fourth question.
>>
>> if a framework has not responded to an offer for a sufficiently long
>> time, Mesos rescinds the offer and re-offers the resources to other
>> frameworks.
>> You cant get it
>>
>> I am not clear in how Mesos divide all resources into multiple subsets?
>>
>> ----------------------------------------
>> 陈宗志
>>
>> Blog: baotiao.github.io
>>
>>
>>
>>
>> On Jun 11, 2015, at 08:35, Qian Zhang <zh...@gmail.com> wrote:
>>
>> Thanks Alex.
>>
>> For 1. I understand currently the only choice is C++. However, as Adam
>> mentioned, true pluggable allocator modules (MESOS-2160
>> <https://issues.apache.org/jira/browse/MESOS-2160>) are landing in Mesos
>> 0.23, so at that time, I assume we will have more choices, right?
>>
>> For 2 and 3, my understanding is Mesos allocator will partition all the
>> available resources into multiple subsets, and there is no overlap between
>> these subsets (i.e., a single resource can only be in one subset), and then
>> offer these subsets to multiple frameworks (e.g., offer subset1 to
>> framework1, offer subset2 to framework2, and so on), and it is up to each
>> framework's scheduler to determine if it accept the resource to launch task
>> or reject it. In this way, each framework's scheduler can actually make
>> scheduling decision independently since they will never compete for the
>> same resource.
>>
>> If my understanding is correct, then I have one more question:
>> 4. What if it takes very long time (e.g., mins or hours) for a
>> framework's scheduler to make the scheduling decision? Does that mean
>> during this long period, the resources offered to this framework will not
>> be used by any other frameworks? Is there a timeout for the
>> framework's scheduler to make the scheduling decision? So when the timeout
>> is reached, the resources offered to it will be revoked by Mesos allocator
>> and can be offered to another framework.
>>
>>
>>
>
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Adam Bordelon <ad...@mesosphere.io>.
1. The modularized allocator will still be a C++ interface, but you could
just create a C++ wrapper around whatever Python/Go/Java/etc.
implementation that you prefer.
Your assessment of 2&3 sounds correct.
4. By default, Mesos will not revoke ("rescind") an used offer being held
by a framework, but you can enable such a timeout by specifying the
`--offer_timeout` flag on the master.
On Thu, Jun 11, 2015 at 1:41 AM, baotiao <ba...@gmail.com> wrote:
> Hi Qian Zhang
>
> I can answer the fourth question.
>
> if a framework has not responded to an offer for a sufficiently long time,
> Mesos rescinds the offer and re-offers the resources to other frameworks.
> You cant get it
>
> I am not clear in how Mesos divide all resources into multiple subsets?
>
> ----------------------------------------
> 陈宗志
>
> Blog: baotiao.github.io
>
>
>
>
> On Jun 11, 2015, at 08:35, Qian Zhang <zh...@gmail.com> wrote:
>
> Thanks Alex.
>
> For 1. I understand currently the only choice is C++. However, as Adam
> mentioned, true pluggable allocator modules (MESOS-2160
> <https://issues.apache.org/jira/browse/MESOS-2160>) are landing in Mesos
> 0.23, so at that time, I assume we will have more choices, right?
>
> For 2 and 3, my understanding is Mesos allocator will partition all the
> available resources into multiple subsets, and there is no overlap between
> these subsets (i.e., a single resource can only be in one subset), and then
> offer these subsets to multiple frameworks (e.g., offer subset1 to
> framework1, offer subset2 to framework2, and so on), and it is up to each
> framework's scheduler to determine if it accept the resource to launch task
> or reject it. In this way, each framework's scheduler can actually make
> scheduling decision independently since they will never compete for the
> same resource.
>
> If my understanding is correct, then I have one more question:
> 4. What if it takes very long time (e.g., mins or hours) for a framework's
> scheduler to make the scheduling decision? Does that mean during this long
> period, the resources offered to this framework will not be used by any
> other frameworks? Is there a timeout for the framework's scheduler to make
> the scheduling decision? So when the timeout is reached, the resources
> offered to it will be revoked by Mesos allocator and can be offered to
> another framework.
>
>
>
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by baotiao <ba...@gmail.com>.
Hi Qian Zhang
I can answer the fourth question.
if a framework has not responded to an offer for a sufficiently long time, Mesos rescinds the offer and re-offers the resources to other frameworks.
You cant get it
I am not clear in how Mesos divide all resources into multiple subsets?
----------------------------------------
陈宗志
Blog: baotiao.github.io
> On Jun 11, 2015, at 08:35, Qian Zhang <zh...@gmail.com> wrote:
>
> Thanks Alex.
>
> For 1. I understand currently the only choice is C++. However, as Adam mentioned, true pluggable allocator modules (MESOS-2160 <https://issues.apache.org/jira/browse/MESOS-2160>) are landing in Mesos 0.23, so at that time, I assume we will have more choices, right?
>
> For 2 and 3, my understanding is Mesos allocator will partition all the available resources into multiple subsets, and there is no overlap between these subsets (i.e., a single resource can only be in one subset), and then offer these subsets to multiple frameworks (e.g., offer subset1 to framework1, offer subset2 to framework2, and so on), and it is up to each framework's scheduler to determine if it accept the resource to launch task or reject it. In this way, each framework's scheduler can actually make scheduling decision independently since they will never compete for the same resource.
>
> If my understanding is correct, then I have one more question:
> 4. What if it takes very long time (e.g., mins or hours) for a framework's scheduler to make the scheduling decision? Does that mean during this long period, the resources offered to this framework will not be used by any other frameworks? Is there a timeout for the framework's scheduler to make the scheduling decision? So when the timeout is reached, the resources offered to it will be revoked by Mesos allocator and can be offered to another framework.
>
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Qian Zhang <zh...@gmail.com>.
Thanks Alex.
For 1. I understand currently the only choice is C++. However, as Adam
mentioned, true pluggable allocator modules (MESOS-2160
<https://issues.apache.org/jira/browse/MESOS-2160>) are landing in Mesos
0.23, so at that time, I assume we will have more choices, right?
For 2 and 3, my understanding is Mesos allocator will partition all the
available resources into multiple subsets, and there is no overlap between
these subsets (i.e., a single resource can only be in one subset), and then
offer these subsets to multiple frameworks (e.g., offer subset1 to
framework1, offer subset2 to framework2, and so on), and it is up to each
framework's scheduler to determine if it accept the resource to launch task
or reject it. In this way, each framework's scheduler can actually make
scheduling decision independently since they will never compete for the
same resource.
If my understanding is correct, then I have one more question:
4. What if it takes very long time (e.g., mins or hours) for a framework's
scheduler to make the scheduling decision? Does that mean during this long
period, the resources offered to this framework will not be used by any
other frameworks? Is there a timeout for the framework's scheduler to make
the scheduling decision? So when the timeout is reached, the resources
offered to it will be revoked by Mesos allocator and can be offered to
another framework.
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Alex Rukletsov <al...@mesosphere.com>.
I'll try to answer these questions.
1. Currently, the only language you can use is C++. You can workaround this
by writing a proxy in c++ that delegates the calls to, say, python scripts.
See http://mesos.apache.org/documentation/latest/allocation-module/ for
more details.
2. The default allocator is called "dominant resource fairness" since it
tries to distribute resources fairly between active frameworks. This means
it will offer all available resources to all frameworks, but each framework
will get only a certain share. For more information I encourage you to take
a look at the DRF paper.
3. Offered and not declined resources are considered to be used, therefore
they can't be re-offered until freed.
Hope this helps.
On 10 Jun 2015 7:53 am, "Qian Zhang" <zh...@gmail.com> wrote:
> Thanks Adam, this is very helpful!
>
> I have a few more questions:
> 1. For the pluggable allocator modules, can I write my own allocator in
> any programming language (e.g., Python, Go, etc)?
> 2. For the default DRF allocator, when it offer resources to a framework,
> will it offer all the available resources (resources not being used by any
> frameworks) to it? Or just part of the available resources?
> 3. If there are multiple frameworks and the default DRF allocator will
> only offer resources to a single framework at a time, then that means
> framework 2 has to wait for framework 1 until framework 1 makes its
> placement decision?
>
>
>
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Qian Zhang <zh...@gmail.com>.
Thanks Adam, this is very helpful!
I have a few more questions:
1. For the pluggable allocator modules, can I write my own allocator in any
programming language (e.g., Python, Go, etc)?
2. For the default DRF allocator, when it offer resources to a framework,
will it offer all the available resources (resources not being used by any
frameworks) to it? Or just part of the available resources?
3. If there are multiple frameworks and the default DRF allocator will only
offer resources to a single framework at a time, then that means framework
2 has to wait for framework 1 until framework 1 makes its placement
decision?
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Adam Bordelon <ad...@mesosphere.io>.
The allocator is currently only pluggable insomuch as it is a C++ class,
and one could subclass/replace it with a custom implementation and then
recompile Mesos. The default/only allocator implementation is
(hierarchical) DRF, which will only offer a resource to a single framework
at a time.
True pluggable allocator modules (MESOS-2160
<https://issues.apache.org/jira/browse/MESOS-2160>) are landing in Mesos
0.23, so you can't use them in a stable release yet.
Optimistic Offers (MESOS-1607
<https://issues.apache.org/jira/browse/MESOS-1607>) is currently
in-progress, but won't land until at least Mesos 0.24.
As for advantages over a single monolithic scheduler, there are a few:
- Fair scheduling: Mesos compares the current resource allocation for each
framework against its (weighted) fair share, and offers newly available
resources to the role/framework that is furthest below its fair share.
- Two-level scheduling: Mesos only chooses which framework to offer
resources to, and then the framework decides which of its tasks to launch
using those offered resources. This means that a complex scheduler could
contemplate offers for seconds/minutes to make an optimal placement
decision, or a low-latency scheduler could just launch tasks on the
first-available resources. The Mesos scheduling decision is very
simple/fast, leaving the rest up to the frameworks for maximum
flexibility/scalability. This also means that the complex scheduler will
not impact the scheduling latency of other lower-latency schedulers.
On Mon, Jun 8, 2015 at 8:31 AM, Michael Hausenblas <
michael.hausenblas@gmail.com> wrote:
>
> > So Mesos can support both 1 and 2 which actually depends on which
> allocator being used, right?
>
> I might be wrong and happy if someone wants to correct me but while the
> allocator module is per design pluggable* the (one and only) current
> implementation is the DRF.
>
> So, can Mesos support 1 and 2? Yes, as I pointed out one way ahead is
> https://issues.apache.org/jira/browse/MESOS-1607
>
> How does Mesos behave per default, ATM? IMHO it is 1.
>
>
> *) http://mesos.apache.org/documentation/latest/allocation-module/
>
> Cheers,
> Michael
>
> --
> Michael Hausenblas
> Ireland, Europe
> http://mhausenblas.info/
>
> > On 8 Jun 2015, at 17:01, Qian Zhang <zh...@gmail.com> wrote:
> >
> > Hi Michael,
> >
> > I think it may not be the latter becauseI see this in the comments of
> the function resourceOffers():
> > * Note that resources may be concurrently offered to more than one
> > * framework at a time (depending on the allocator being used). In
> > * that case, the first framework to launch tasks using those
> > * resources will be able to use them while the other frameworks
> > * will have those resources rescinded (or if a framework has
> > * already launched tasks with those resources then those tasks will
> > * fail with a TASK_LOST status and a message saying as much).
> > */
> >
> > So Mesos can support both 1 and 2 which actually depends on which
> allocator being used, right?
> >
> >
> > 2015-06-06 19:06 GMT+08:00 Michael Hausenblas <
> michael.hausenblas@gmail.com>:
> >
> > > 1. Mesos master offers all the resources to all the frameworks
> simultaneously.
> > > 2. Mesos master offers resources to one framework at a time, e.g., it
> offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to
> f2, ...
> >
> > The latter, yes.
> >
> > For a quick overview, I suggest you have a look at
> http://mesos.apache.org/documentation/latest/mesos-architecture/ which
> covers the resource offer cycle.
> >
> > If you want to dive deeper, you might want to read:
> >
> > 1. http://mesos.berkeley.edu/mesos_tech_report.pdf
> > 2. https://www.cs.berkeley.edu/~alig/papers/drf.pdf
> >
> >
> > Note that there's a feature in the works that would be closer to your
> 1., see https://issues.apache.org/jira/browse/MESOS-1607
> >
> > Cheers,
> > Michael
> >
> > --
> > Michael Hausenblas
> > Ireland, Europe
> > http://mhausenblas.info/
> >
> > > On 6 Jun 2015, at 12:51, Qian Zhang <zh...@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > I am new to Mesos, and I'd like to know if there are a lot resources
> in the Mesos cluster, how will Mesos master offer these resources to the
> multiple frameworks? I guess there can be two ways:
> > > 1. Mesos master offers all the resources to all the frameworks
> simultaneously.
> > > 2. Mesos master offers resources to one framework at a time, e.g., it
> offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to
> f2, ...
> > >
> > > If it is 1, then I'd like to know how Mesos master resolves the
> conflicts, e.g., multiple frameworks accept the same resource.
> > > If it is 2, then I see it is actually a serial process since Mesos
> master handle the frameworks one by one, then what is advantage of Mesos
> against traditional monolithic resource scheduler?
> > >
> > >
> > > Thanks,
> > > Qian
> >
> >
>
>
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Michael Hausenblas <mi...@gmail.com>.
> So Mesos can support both 1 and 2 which actually depends on which allocator being used, right?
I might be wrong and happy if someone wants to correct me but while the allocator module is per design pluggable* the (one and only) current implementation is the DRF.
So, can Mesos support 1 and 2? Yes, as I pointed out one way ahead is https://issues.apache.org/jira/browse/MESOS-1607
How does Mesos behave per default, ATM? IMHO it is 1.
*) http://mesos.apache.org/documentation/latest/allocation-module/
Cheers,
Michael
--
Michael Hausenblas
Ireland, Europe
http://mhausenblas.info/
> On 8 Jun 2015, at 17:01, Qian Zhang <zh...@gmail.com> wrote:
>
> Hi Michael,
>
> I think it may not be the latter becauseI see this in the comments of the function resourceOffers():
> * Note that resources may be concurrently offered to more than one
> * framework at a time (depending on the allocator being used). In
> * that case, the first framework to launch tasks using those
> * resources will be able to use them while the other frameworks
> * will have those resources rescinded (or if a framework has
> * already launched tasks with those resources then those tasks will
> * fail with a TASK_LOST status and a message saying as much).
> */
>
> So Mesos can support both 1 and 2 which actually depends on which allocator being used, right?
>
>
> 2015-06-06 19:06 GMT+08:00 Michael Hausenblas <mi...@gmail.com>:
>
> > 1. Mesos master offers all the resources to all the frameworks simultaneously.
> > 2. Mesos master offers resources to one framework at a time, e.g., it offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to f2, ...
>
> The latter, yes.
>
> For a quick overview, I suggest you have a look at http://mesos.apache.org/documentation/latest/mesos-architecture/ which covers the resource offer cycle.
>
> If you want to dive deeper, you might want to read:
>
> 1. http://mesos.berkeley.edu/mesos_tech_report.pdf
> 2. https://www.cs.berkeley.edu/~alig/papers/drf.pdf
>
>
> Note that there's a feature in the works that would be closer to your 1., see https://issues.apache.org/jira/browse/MESOS-1607
>
> Cheers,
> Michael
>
> --
> Michael Hausenblas
> Ireland, Europe
> http://mhausenblas.info/
>
> > On 6 Jun 2015, at 12:51, Qian Zhang <zh...@gmail.com> wrote:
> >
> > Hi,
> >
> > I am new to Mesos, and I'd like to know if there are a lot resources in the Mesos cluster, how will Mesos master offer these resources to the multiple frameworks? I guess there can be two ways:
> > 1. Mesos master offers all the resources to all the frameworks simultaneously.
> > 2. Mesos master offers resources to one framework at a time, e.g., it offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to f2, ...
> >
> > If it is 1, then I'd like to know how Mesos master resolves the conflicts, e.g., multiple frameworks accept the same resource.
> > If it is 2, then I see it is actually a serial process since Mesos master handle the frameworks one by one, then what is advantage of Mesos against traditional monolithic resource scheduler?
> >
> >
> > Thanks,
> > Qian
>
>
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Qian Zhang <zh...@gmail.com>.
Hi Michael,
I think it may not be the latter becauseI see this in the comments of the
function resourceOffers():
* Note that resources may be concurrently offered to more than one
* framework at a time (depending on the allocator being used). In
* that case, the first framework to launch tasks using those
* resources will be able to use them while the other frameworks
* will have those resources rescinded (or if a framework has
* already launched tasks with those resources then those tasks will
* fail with a TASK_LOST status and a message saying as much).
*/
So Mesos can support both 1 and 2 which actually depends on which allocator
being used, right?
2015-06-06 19:06 GMT+08:00 Michael Hausenblas <mi...@gmail.com>
:
>
> > 1. Mesos master offers all the resources to all the frameworks
> simultaneously.
> > 2. Mesos master offers resources to one framework at a time, e.g., it
> offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to
> f2, ...
>
> The latter, yes.
>
> For a quick overview, I suggest you have a look at
> http://mesos.apache.org/documentation/latest/mesos-architecture/ which
> covers the resource offer cycle.
>
> If you want to dive deeper, you might want to read:
>
> 1. http://mesos.berkeley.edu/mesos_tech_report.pdf
> 2. https://www.cs.berkeley.edu/~alig/papers/drf.pdf
>
>
> Note that there's a feature in the works that would be closer to your 1.,
> see https://issues.apache.org/jira/browse/MESOS-1607
>
> Cheers,
> Michael
>
> --
> Michael Hausenblas
> Ireland, Europe
> http://mhausenblas.info/
>
> > On 6 Jun 2015, at 12:51, Qian Zhang <zh...@gmail.com> wrote:
> >
> > Hi,
> >
> > I am new to Mesos, and I'd like to know if there are a lot resources in
> the Mesos cluster, how will Mesos master offer these resources to the
> multiple frameworks? I guess there can be two ways:
> > 1. Mesos master offers all the resources to all the frameworks
> simultaneously.
> > 2. Mesos master offers resources to one framework at a time, e.g., it
> offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to
> f2, ...
> >
> > If it is 1, then I'd like to know how Mesos master resolves the
> conflicts, e.g., multiple frameworks accept the same resource.
> > If it is 2, then I see it is actually a serial process since Mesos
> master handle the frameworks one by one, then what is advantage of Mesos
> against traditional monolithic resource scheduler?
> >
> >
> > Thanks,
> > Qian
>
>
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Qian Zhang <zh...@gmail.com>.
Thanks Michael, I will take a close look at those paper.
2015-06-06 19:06 GMT+08:00 Michael Hausenblas <mi...@gmail.com>
:
>
> > 1. Mesos master offers all the resources to all the frameworks
> simultaneously.
> > 2. Mesos master offers resources to one framework at a time, e.g., it
> offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to
> f2, ...
>
> The latter, yes.
>
> For a quick overview, I suggest you have a look at
> http://mesos.apache.org/documentation/latest/mesos-architecture/ which
> covers the resource offer cycle.
>
> If you want to dive deeper, you might want to read:
>
> 1. http://mesos.berkeley.edu/mesos_tech_report.pdf
> 2. https://www.cs.berkeley.edu/~alig/papers/drf.pdf
>
>
> Note that there's a feature in the works that would be closer to your 1.,
> see https://issues.apache.org/jira/browse/MESOS-1607
>
> Cheers,
> Michael
>
> --
> Michael Hausenblas
> Ireland, Europe
> http://mhausenblas.info/
>
> > On 6 Jun 2015, at 12:51, Qian Zhang <zh...@gmail.com> wrote:
> >
> > Hi,
> >
> > I am new to Mesos, and I'd like to know if there are a lot resources in
> the Mesos cluster, how will Mesos master offer these resources to the
> multiple frameworks? I guess there can be two ways:
> > 1. Mesos master offers all the resources to all the frameworks
> simultaneously.
> > 2. Mesos master offers resources to one framework at a time, e.g., it
> offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to
> f2, ...
> >
> > If it is 1, then I'd like to know how Mesos master resolves the
> conflicts, e.g., multiple frameworks accept the same resource.
> > If it is 2, then I see it is actually a serial process since Mesos
> master handle the frameworks one by one, then what is advantage of Mesos
> against traditional monolithic resource scheduler?
> >
> >
> > Thanks,
> > Qian
>
>
Re: Can Mesos master offer resources to multiple frameworks simultaneously?
Posted by Michael Hausenblas <mi...@gmail.com>.
> 1. Mesos master offers all the resources to all the frameworks simultaneously.
> 2. Mesos master offers resources to one framework at a time, e.g., it offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to f2, ...
The latter, yes.
For a quick overview, I suggest you have a look at http://mesos.apache.org/documentation/latest/mesos-architecture/ which covers the resource offer cycle.
If you want to dive deeper, you might want to read:
1. http://mesos.berkeley.edu/mesos_tech_report.pdf
2. https://www.cs.berkeley.edu/~alig/papers/drf.pdf
Note that there's a feature in the works that would be closer to your 1., see https://issues.apache.org/jira/browse/MESOS-1607
Cheers,
Michael
--
Michael Hausenblas
Ireland, Europe
http://mhausenblas.info/
> On 6 Jun 2015, at 12:51, Qian Zhang <zh...@gmail.com> wrote:
>
> Hi,
>
> I am new to Mesos, and I'd like to know if there are a lot resources in the Mesos cluster, how will Mesos master offer these resources to the multiple frameworks? I guess there can be two ways:
> 1. Mesos master offers all the resources to all the frameworks simultaneously.
> 2. Mesos master offers resources to one framework at a time, e.g., it offers r1, r2, r3 to f1, and f1 accepts r1, and then it offers r2 and r3 to f2, ...
>
> If it is 1, then I'd like to know how Mesos master resolves the conflicts, e.g., multiple frameworks accept the same resource.
> If it is 2, then I see it is actually a serial process since Mesos master handle the frameworks one by one, then what is advantage of Mesos against traditional monolithic resource scheduler?
>
>
> Thanks,
> Qian