You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by 许立剑 <xu...@qiyi.com> on 2013/09/02 04:04:50 UTC
Frameworks with different priorities
Hi all,
We are trying to use Mesos to manage different frameworks, which have
different priorities. Say, framework B should only get resource offers
when framework A is free.
Is there any exsiting mechanism that helps implement this? What is the
current framework-level scheduler like?
Thanks,
Iven
Re: Frameworks with different priorities
Posted by 许立剑 <xu...@qiyi.com>.
Thanks for your reply. It seem that the flag is not supported by Mesos
0.12.0. I'll dig into it later.
于 2013年09月04日 01:29, Benjamin Hindman 写道:
> Hi Iven,
>
> You can pass the flag --weights=role1=weight1,role2=weight2 to the
> mesos-master to specify different "priorities" for DRF (e.g.,
> mesos-master --weights=hadoop-production=4,hadoop-experimental=1).
> Then you can set the role in the FrameworkInfo (or the configuration
> property 'mapred.mesos.role' for Hadoop).
>
> Ben.
>
>
> On Sun, Sep 1, 2013 at 7:46 PM, 许立剑 <xulijian@qiyi.com
> <ma...@qiyi.com>> wrote:
>
> Hi Paco,
>
> Sorry, mesos core doesn't know that, it's a mistake. I think my
> question is the same as Li Jin: Does the current DRF
> implementation support different weights for different users?
>
> 于 2013年09月02日 10:38, Paco Nathan 写道:
>> Hi Iven -
>>
>> What kind of metrics do you use to determine whether framework A
>> is "free" ?
>>
>>
>> On Sun, Sep 1, 2013 at 7:04 PM, 许立剑 <xulijian@qiyi.com
>> <ma...@qiyi.com>> wrote:
>>
>> Hi all,
>>
>> We are trying to use Mesos to manage different frameworks,
>> which have
>> different priorities. Say, framework B should only get
>> resource offers
>> when framework A is free.
>>
>> Is there any exsiting mechanism that helps implement this?
>> What is the
>> current framework-level scheduler like?
>>
>> Thanks,
>> Iven
>>
>>
>
>
Re: Frameworks with different priorities
Posted by Benjamin Hindman <be...@gmail.com>.
Hi Iven,
You can pass the flag --weights=role1=weight1,role2=weight2 to the
mesos-master to specify different "priorities" for DRF (e.g., mesos-master
--weights=hadoop-production=4,hadoop-experimental=1). Then you can set the
role in the FrameworkInfo (or the configuration property
'mapred.mesos.role' for Hadoop).
Ben.
On Sun, Sep 1, 2013 at 7:46 PM, 许立剑 <xu...@qiyi.com> wrote:
> Hi Paco,
>
> Sorry, mesos core doesn't know that, it's a mistake. I think my question
> is the same as Li Jin: Does the current DRF implementation support
> different weights for different users?
>
> 于 2013年09月02日 10:38, Paco Nathan 写道:
>
> Hi Iven -
>
> What kind of metrics do you use to determine whether framework A is
> "free" ?
>
>
> On Sun, Sep 1, 2013 at 7:04 PM, 许立剑 <xu...@qiyi.com> wrote:
>
>> Hi all,
>>
>> We are trying to use Mesos to manage different frameworks, which have
>> different priorities. Say, framework B should only get resource offers
>> when framework A is free.
>>
>> Is there any exsiting mechanism that helps implement this? What is the
>> current framework-level scheduler like?
>>
>> Thanks,
>> Iven
>>
>
>
>
Re: Frameworks with different priorities
Posted by 许立剑 <xu...@qiyi.com>.
When we design the framework to fit our exsiting projects, we have two
models:
1. Framework receive tasks sent from the task queue, save them in a
buffer, and wait for resource offers. So it depends on if the buffer is
free.
2. Framework fetch tasks from the queue when resourceOffer callback is
called. If no task got for a period, it's free.
Currently neither could be known by mesos core, so possible way may be
offer more resources to the frameworks that have more weight?
于 2013年09月02日 11:06, Paco Nathan 写道:
> But wait a moment, if there were that capability in core Mesos --
> again, what would you define as the criteria for framework A begin
> considered "free" ? That use case / feedback is valuable, and the
> implementation may not be hard to do.
>
>
> On Sun, Sep 1, 2013 at 7:46 PM, 许立剑 <xulijian@qiyi.com
> <ma...@qiyi.com>> wrote:
>
> Hi Paco,
>
> Sorry, mesos core doesn't know that, it's a mistake. I think my
> question is the same as Li Jin: Does the current DRF
> implementation support different weights for different users?
>
> 于 2013年09月02日 10:38, Paco Nathan 写道:
>> Hi Iven -
>>
>> What kind of metrics do you use to determine whether framework A
>> is "free" ?
>>
>>
>> On Sun, Sep 1, 2013 at 7:04 PM, 许立剑 <xulijian@qiyi.com
>> <ma...@qiyi.com>> wrote:
>>
>> Hi all,
>>
>> We are trying to use Mesos to manage different frameworks,
>> which have
>> different priorities. Say, framework B should only get
>> resource offers
>> when framework A is free.
>>
>> Is there any exsiting mechanism that helps implement this?
>> What is the
>> current framework-level scheduler like?
>>
>> Thanks,
>> Iven
>>
>>
>
>
Re: Frameworks with different priorities
Posted by Paco Nathan <ce...@gmail.com>.
But wait a moment, if there were that capability in core Mesos -- again,
what would you define as the criteria for framework A begin considered
"free" ? That use case / feedback is valuable, and the implementation may
not be hard to do.
On Sun, Sep 1, 2013 at 7:46 PM, 许立剑 <xu...@qiyi.com> wrote:
> Hi Paco,
>
> Sorry, mesos core doesn't know that, it's a mistake. I think my question
> is the same as Li Jin: Does the current DRF implementation support
> different weights for different users?
>
> 于 2013年09月02日 10:38, Paco Nathan 写道:
>
> Hi Iven -
>
> What kind of metrics do you use to determine whether framework A is
> "free" ?
>
>
> On Sun, Sep 1, 2013 at 7:04 PM, 许立剑 <xu...@qiyi.com> wrote:
>
>> Hi all,
>>
>> We are trying to use Mesos to manage different frameworks, which have
>> different priorities. Say, framework B should only get resource offers
>> when framework A is free.
>>
>> Is there any exsiting mechanism that helps implement this? What is the
>> current framework-level scheduler like?
>>
>> Thanks,
>> Iven
>>
>
>
>
Re: Frameworks with different priorities
Posted by 许立剑 <xu...@qiyi.com>.
Hi Paco,
Sorry, mesos core doesn't know that, it's a mistake. I think my question
is the same as Li Jin: Does the current DRF implementation support
different weights for different users?
于 2013年09月02日 10:38, Paco Nathan 写道:
> Hi Iven -
>
> What kind of metrics do you use to determine whether framework A is
> "free" ?
>
>
> On Sun, Sep 1, 2013 at 7:04 PM, 许立剑 <xulijian@qiyi.com
> <ma...@qiyi.com>> wrote:
>
> Hi all,
>
> We are trying to use Mesos to manage different frameworks, which have
> different priorities. Say, framework B should only get resource offers
> when framework A is free.
>
> Is there any exsiting mechanism that helps implement this? What is the
> current framework-level scheduler like?
>
> Thanks,
> Iven
>
>
Re: Frameworks with different priorities
Posted by Paco Nathan <ce...@gmail.com>.
Hi Iven -
What kind of metrics do you use to determine whether framework A is "free" ?
On Sun, Sep 1, 2013 at 7:04 PM, 许立剑 <xu...@qiyi.com> wrote:
> Hi all,
>
> We are trying to use Mesos to manage different frameworks, which have
> different priorities. Say, framework B should only get resource offers
> when framework A is free.
>
> Is there any exsiting mechanism that helps implement this? What is the
> current framework-level scheduler like?
>
> Thanks,
> Iven
>