You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Hendrik Haddorp <he...@gmx.net> on 2016/09/26 06:47:58 UTC

resource offers

Hi,

I have three Mesos cluster test setups. On two my frameworks gets the 
resource offers from all slaves in one "resourceOffers" call. In one 
three node setup I do however sometimes get offers for all slaves but 
most of the time I get first two offers and then the third in a separate 
call. Should I get all offers in one call or do I need to collect 
offers? I couldn't find any documentation on this. Problem in my case is 
that I need to start some tasks only once I have offers from multiple 
slaves / agents. So far I expected to get all offers in one call.

thanks,
Hendrik

Re: resource offers

Posted by Hendrik Haddorp <he...@gmx.net>.
yes, thanks, I did that already.

On 27/09/16 00:27, Guangya Liu wrote:
> Great, then you can set a 0s filter, please refer
> to https://github.com/apache/mesos/blob/master/src/tests/containerizer/docker_containerizer_tests.cpp#L1551-L1554
>
> On Mon, Sep 26, 2016 at 9:50 PM, Hendrik Haddorp
> <hendrik.haddorp@gmx.net <ma...@gmx.net>> wrote:
>
>     Your tip helped :-) I'm seeing lines like this in the log:
>     mesos-master[3781]: I0926 23:36:11.292654  3805
>     hierarchical.cpp:927] Framework
>     12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-0007 filtered slave
>     12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-S0 for 5secs
>
>     This is caused by Mesos doing a 5sec filter if you do not set a
>     filter on an offer decline. Didn't quite expect that ;-)
>
>
>     On 26.09.2016 11 <tel:26.09.2016%2011>:15, Hendrik Haddorp wrote:
>
>         I don't "own" the setup so that is not so easy. For now I'm
>         keeping the offers for a few seconds to get around it. If I
>         get traces enabled I'll let you know. I was first of all only
>         interested if I'm supposed to get all offers at once or not.
>
>         On 26.09.2016 11 <tel:26.09.2016%2011>:08, Guangya Liu wrote:
>
>             Can you please turn on the `GLOG_v=2` in mesos master and
>             append more logs here?
>
>             On Mon, Sep 26, 2016 at 4:36 PM, Hendrik Haddorp
>             <hendrik.haddorp@gmx.net <ma...@gmx.net>
>             <mailto:hendrik.haddorp@gmx.net
>             <ma...@gmx.net>>> wrote:
>
>                 Thanks for the tip. No, I did not set any filters.
>
>                 On 26.09.2016 09 <tel:26.09.2016%2009>
>             <tel:26.09.2016%2009>:53, Guangya Liu wrote:
>
>                     This depends, it may be caused by that one of the
>             offer is
>                     declined and then got a filter, this will cause
>             the filtered
>                     offer will be offered after the expire time. Did
>             you set
>                     filters when decline offer? If you are enabling
>             GLOG_v=2 for
>                     mesos master, you will get some log as "Framework
>             xxx filtered
>                     agent <slaveId> for <seconds>"
>
>                     On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp
>                     <hendrik.haddorp@gmx.net
>             <ma...@gmx.net>
>             <mailto:hendrik.haddorp@gmx.net
>             <ma...@gmx.net>>
>                     <mailto:hendrik.haddorp@gmx.net
>             <ma...@gmx.net>
>                     <mailto:hendrik.haddorp@gmx.net
>             <ma...@gmx.net>>>> wrote:
>
>                         Hi,
>
>                         I have three Mesos cluster test setups. On two my
>                     frameworks gets
>                         the resource offers from all slaves in one
>                     "resourceOffers" call.
>                         In one three node setup I do however sometimes
>             get offers
>                     for all
>                         slaves but most of the time I get first two
>             offers and
>                     then the
>                         third in a separate call. Should I get all
>             offers in one
>                     call or
>                         do I need to collect offers? I couldn't find any
>                     documentation on
>                         this. Problem in my case is that I need to
>             start some
>                     tasks only
>                         once I have offers from multiple slaves /
>             agents. So far I
>                         expected to get all offers in one call.
>
>                         thanks,
>                         Hendrik
>
>
>
>
>
>
>


Re: resource offers

Posted by Guangya Liu <gy...@gmail.com>.
Great, then you can set a 0s filter, please refer to
https://github.com/apache/mesos/blob/master/src/tests/containerizer/docker_containerizer_tests.cpp#L1551-L1554

On Mon, Sep 26, 2016 at 9:50 PM, Hendrik Haddorp <he...@gmx.net>
wrote:

> Your tip helped :-) I'm seeing lines like this in the log:
> mesos-master[3781]: I0926 23:36:11.292654  3805 hierarchical.cpp:927]
> Framework 12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-0007 filtered slave
> 12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-S0 for 5secs
>
> This is caused by Mesos doing a 5sec filter if you do not set a filter on
> an offer decline. Didn't quite expect that ;-)
>
>
> On 26.09.2016 11:15, Hendrik Haddorp wrote:
>
>> I don't "own" the setup so that is not so easy. For now I'm keeping the
>> offers for a few seconds to get around it. If I get traces enabled I'll let
>> you know. I was first of all only interested if I'm supposed to get all
>> offers at once or not.
>>
>> On 26.09.2016 11:08, Guangya Liu wrote:
>>
>>> Can you please turn on the `GLOG_v=2` in mesos master and append more
>>> logs here?
>>>
>>> On Mon, Sep 26, 2016 at 4:36 PM, Hendrik Haddorp <
>>> hendrik.haddorp@gmx.net <ma...@gmx.net>> wrote:
>>>
>>>     Thanks for the tip. No, I did not set any filters.
>>>
>>>     On 26.09.2016 09 <tel:26.09.2016%2009>:53, Guangya Liu wrote:
>>>
>>>         This depends, it may be caused by that one of the offer is
>>>         declined and then got a filter, this will cause the filtered
>>>         offer will be offered after the expire time. Did you set
>>>         filters when decline offer? If you are enabling GLOG_v=2 for
>>>         mesos master, you will get some log as "Framework xxx filtered
>>>         agent <slaveId> for <seconds>"
>>>
>>>         On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp
>>>         <hendrik.haddorp@gmx.net <ma...@gmx.net>
>>>         <mailto:hendrik.haddorp@gmx.net
>>>         <ma...@gmx.net>>> wrote:
>>>
>>>             Hi,
>>>
>>>             I have three Mesos cluster test setups. On two my
>>>         frameworks gets
>>>             the resource offers from all slaves in one
>>>         "resourceOffers" call.
>>>             In one three node setup I do however sometimes get offers
>>>         for all
>>>             slaves but most of the time I get first two offers and
>>>         then the
>>>             third in a separate call. Should I get all offers in one
>>>         call or
>>>             do I need to collect offers? I couldn't find any
>>>         documentation on
>>>             this. Problem in my case is that I need to start some
>>>         tasks only
>>>             once I have offers from multiple slaves / agents. So far I
>>>             expected to get all offers in one call.
>>>
>>>             thanks,
>>>             Hendrik
>>>
>>>
>>>
>>>
>>>
>>
>

Re: resource offers

Posted by Hendrik Haddorp <he...@gmx.net>.
Your tip helped :-) I'm seeing lines like this in the log:
mesos-master[3781]: I0926 23:36:11.292654  3805 hierarchical.cpp:927] 
Framework 12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-0007 filtered slave 
12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-S0 for 5secs

This is caused by Mesos doing a 5sec filter if you do not set a filter 
on an offer decline. Didn't quite expect that ;-)

On 26.09.2016 11:15, Hendrik Haddorp wrote:
> I don't "own" the setup so that is not so easy. For now I'm keeping 
> the offers for a few seconds to get around it. If I get traces enabled 
> I'll let you know. I was first of all only interested if I'm supposed 
> to get all offers at once or not.
>
> On 26.09.2016 11:08, Guangya Liu wrote:
>> Can you please turn on the `GLOG_v=2` in mesos master and append more 
>> logs here?
>>
>> On Mon, Sep 26, 2016 at 4:36 PM, Hendrik Haddorp 
>> <hendrik.haddorp@gmx.net <ma...@gmx.net>> wrote:
>>
>>     Thanks for the tip. No, I did not set any filters.
>>
>>     On 26.09.2016 09 <tel:26.09.2016%2009>:53, Guangya Liu wrote:
>>
>>         This depends, it may be caused by that one of the offer is
>>         declined and then got a filter, this will cause the filtered
>>         offer will be offered after the expire time. Did you set
>>         filters when decline offer? If you are enabling GLOG_v=2 for
>>         mesos master, you will get some log as "Framework xxx filtered
>>         agent <slaveId> for <seconds>"
>>
>>         On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp
>>         <hendrik.haddorp@gmx.net <ma...@gmx.net>
>>         <mailto:hendrik.haddorp@gmx.net
>>         <ma...@gmx.net>>> wrote:
>>
>>             Hi,
>>
>>             I have three Mesos cluster test setups. On two my
>>         frameworks gets
>>             the resource offers from all slaves in one
>>         "resourceOffers" call.
>>             In one three node setup I do however sometimes get offers
>>         for all
>>             slaves but most of the time I get first two offers and
>>         then the
>>             third in a separate call. Should I get all offers in one
>>         call or
>>             do I need to collect offers? I couldn't find any
>>         documentation on
>>             this. Problem in my case is that I need to start some
>>         tasks only
>>             once I have offers from multiple slaves / agents. So far I
>>             expected to get all offers in one call.
>>
>>             thanks,
>>             Hendrik
>>
>>
>>
>>
>


Re: resource offers

Posted by Hendrik Haddorp <he...@gmx.net>.
I don't "own" the setup so that is not so easy. For now I'm keeping the 
offers for a few seconds to get around it. If I get traces enabled I'll 
let you know. I was first of all only interested if I'm supposed to get 
all offers at once or not.

On 26.09.2016 11:08, Guangya Liu wrote:
> Can you please turn on the `GLOG_v=2` in mesos master and append more 
> logs here?
>
> On Mon, Sep 26, 2016 at 4:36 PM, Hendrik Haddorp 
> <hendrik.haddorp@gmx.net <ma...@gmx.net>> wrote:
>
>     Thanks for the tip. No, I did not set any filters.
>
>     On 26.09.2016 09 <tel:26.09.2016%2009>:53, Guangya Liu wrote:
>
>         This depends, it may be caused by that one of the offer is
>         declined and then got a filter, this will cause the filtered
>         offer will be offered after the expire time. Did you set
>         filters when decline offer? If you are enabling GLOG_v=2 for
>         mesos master, you will get some log as "Framework xxx filtered
>         agent <slaveId> for <seconds>"
>
>         On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp
>         <hendrik.haddorp@gmx.net <ma...@gmx.net>
>         <mailto:hendrik.haddorp@gmx.net
>         <ma...@gmx.net>>> wrote:
>
>             Hi,
>
>             I have three Mesos cluster test setups. On two my
>         frameworks gets
>             the resource offers from all slaves in one
>         "resourceOffers" call.
>             In one three node setup I do however sometimes get offers
>         for all
>             slaves but most of the time I get first two offers and
>         then the
>             third in a separate call. Should I get all offers in one
>         call or
>             do I need to collect offers? I couldn't find any
>         documentation on
>             this. Problem in my case is that I need to start some
>         tasks only
>             once I have offers from multiple slaves / agents. So far I
>             expected to get all offers in one call.
>
>             thanks,
>             Hendrik
>
>
>
>


Re: resource offers

Posted by Guangya Liu <gy...@gmail.com>.
Can you please turn on the `GLOG_v=2` in mesos master and append more logs
here?

On Mon, Sep 26, 2016 at 4:36 PM, Hendrik Haddorp <he...@gmx.net>
wrote:

> Thanks for the tip. No, I did not set any filters.
>
> On 26.09.2016 09:53, Guangya Liu wrote:
>
>> This depends, it may be caused by that one of the offer is declined and
>> then got a filter, this will cause the filtered offer will be offered after
>> the expire time. Did you set filters when decline offer? If you are
>> enabling GLOG_v=2 for mesos master, you will get some log as "Framework xxx
>> filtered agent <slaveId> for <seconds>"
>>
>> On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp <hendrik.haddorp@gmx.net
>> <ma...@gmx.net>> wrote:
>>
>>     Hi,
>>
>>     I have three Mesos cluster test setups. On two my frameworks gets
>>     the resource offers from all slaves in one "resourceOffers" call.
>>     In one three node setup I do however sometimes get offers for all
>>     slaves but most of the time I get first two offers and then the
>>     third in a separate call. Should I get all offers in one call or
>>     do I need to collect offers? I couldn't find any documentation on
>>     this. Problem in my case is that I need to start some tasks only
>>     once I have offers from multiple slaves / agents. So far I
>>     expected to get all offers in one call.
>>
>>     thanks,
>>     Hendrik
>>
>>
>>
>

Re: resource offers

Posted by Hendrik Haddorp <he...@gmx.net>.
Thanks for the tip. No, I did not set any filters.

On 26.09.2016 09:53, Guangya Liu wrote:
> This depends, it may be caused by that one of the offer is declined 
> and then got a filter, this will cause the filtered offer will be 
> offered after the expire time. Did you set filters when decline offer? 
> If you are enabling GLOG_v=2 for mesos master, you will get some log 
> as "Framework xxx filtered agent <slaveId> for <seconds>"
>
> On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp 
> <hendrik.haddorp@gmx.net <ma...@gmx.net>> wrote:
>
>     Hi,
>
>     I have three Mesos cluster test setups. On two my frameworks gets
>     the resource offers from all slaves in one "resourceOffers" call.
>     In one three node setup I do however sometimes get offers for all
>     slaves but most of the time I get first two offers and then the
>     third in a separate call. Should I get all offers in one call or
>     do I need to collect offers? I couldn't find any documentation on
>     this. Problem in my case is that I need to start some tasks only
>     once I have offers from multiple slaves / agents. So far I
>     expected to get all offers in one call.
>
>     thanks,
>     Hendrik
>
>


Re: resource offers

Posted by Guangya Liu <gy...@gmail.com>.
This depends, it may be caused by that one of the offer is declined and
then got a filter, this will cause the filtered offer will be offered after
the expire time. Did you set filters when decline offer? If you are
enabling GLOG_v=2 for mesos master, you will get some log as "Framework xxx
filtered agent <slaveId> for <seconds>"

On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp <he...@gmx.net>
wrote:

> Hi,
>
> I have three Mesos cluster test setups. On two my frameworks gets the
> resource offers from all slaves in one "resourceOffers" call. In one three
> node setup I do however sometimes get offers for all slaves but most of the
> time I get first two offers and then the third in a separate call. Should I
> get all offers in one call or do I need to collect offers? I couldn't find
> any documentation on this. Problem in my case is that I need to start some
> tasks only once I have offers from multiple slaves / agents. So far I
> expected to get all offers in one call.
>
> thanks,
> Hendrik
>