You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Hendrik Haddorp <he...@gmx.net> on 2016/09/26 06:47:58 UTC
resource offers
Hi,
I have three Mesos cluster test setups. On two my frameworks gets the
resource offers from all slaves in one "resourceOffers" call. In one
three node setup I do however sometimes get offers for all slaves but
most of the time I get first two offers and then the third in a separate
call. Should I get all offers in one call or do I need to collect
offers? I couldn't find any documentation on this. Problem in my case is
that I need to start some tasks only once I have offers from multiple
slaves / agents. So far I expected to get all offers in one call.
thanks,
Hendrik
Re: resource offers
Posted by Hendrik Haddorp <he...@gmx.net>.
yes, thanks, I did that already.
On 27/09/16 00:27, Guangya Liu wrote:
> Great, then you can set a 0s filter, please refer
> to https://github.com/apache/mesos/blob/master/src/tests/containerizer/docker_containerizer_tests.cpp#L1551-L1554
>
> On Mon, Sep 26, 2016 at 9:50 PM, Hendrik Haddorp
> <hendrik.haddorp@gmx.net <ma...@gmx.net>> wrote:
>
> Your tip helped :-) I'm seeing lines like this in the log:
> mesos-master[3781]: I0926 23:36:11.292654 3805
> hierarchical.cpp:927] Framework
> 12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-0007 filtered slave
> 12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-S0 for 5secs
>
> This is caused by Mesos doing a 5sec filter if you do not set a
> filter on an offer decline. Didn't quite expect that ;-)
>
>
> On 26.09.2016 11 <tel:26.09.2016%2011>:15, Hendrik Haddorp wrote:
>
> I don't "own" the setup so that is not so easy. For now I'm
> keeping the offers for a few seconds to get around it. If I
> get traces enabled I'll let you know. I was first of all only
> interested if I'm supposed to get all offers at once or not.
>
> On 26.09.2016 11 <tel:26.09.2016%2011>:08, Guangya Liu wrote:
>
> Can you please turn on the `GLOG_v=2` in mesos master and
> append more logs here?
>
> On Mon, Sep 26, 2016 at 4:36 PM, Hendrik Haddorp
> <hendrik.haddorp@gmx.net <ma...@gmx.net>
> <mailto:hendrik.haddorp@gmx.net
> <ma...@gmx.net>>> wrote:
>
> Thanks for the tip. No, I did not set any filters.
>
> On 26.09.2016 09 <tel:26.09.2016%2009>
> <tel:26.09.2016%2009>:53, Guangya Liu wrote:
>
> This depends, it may be caused by that one of the
> offer is
> declined and then got a filter, this will cause
> the filtered
> offer will be offered after the expire time. Did
> you set
> filters when decline offer? If you are enabling
> GLOG_v=2 for
> mesos master, you will get some log as "Framework
> xxx filtered
> agent <slaveId> for <seconds>"
>
> On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp
> <hendrik.haddorp@gmx.net
> <ma...@gmx.net>
> <mailto:hendrik.haddorp@gmx.net
> <ma...@gmx.net>>
> <mailto:hendrik.haddorp@gmx.net
> <ma...@gmx.net>
> <mailto:hendrik.haddorp@gmx.net
> <ma...@gmx.net>>>> wrote:
>
> Hi,
>
> I have three Mesos cluster test setups. On two my
> frameworks gets
> the resource offers from all slaves in one
> "resourceOffers" call.
> In one three node setup I do however sometimes
> get offers
> for all
> slaves but most of the time I get first two
> offers and
> then the
> third in a separate call. Should I get all
> offers in one
> call or
> do I need to collect offers? I couldn't find any
> documentation on
> this. Problem in my case is that I need to
> start some
> tasks only
> once I have offers from multiple slaves /
> agents. So far I
> expected to get all offers in one call.
>
> thanks,
> Hendrik
>
>
>
>
>
>
>
Re: resource offers
Posted by Guangya Liu <gy...@gmail.com>.
Great, then you can set a 0s filter, please refer to
https://github.com/apache/mesos/blob/master/src/tests/containerizer/docker_containerizer_tests.cpp#L1551-L1554
On Mon, Sep 26, 2016 at 9:50 PM, Hendrik Haddorp <he...@gmx.net>
wrote:
> Your tip helped :-) I'm seeing lines like this in the log:
> mesos-master[3781]: I0926 23:36:11.292654 3805 hierarchical.cpp:927]
> Framework 12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-0007 filtered slave
> 12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-S0 for 5secs
>
> This is caused by Mesos doing a 5sec filter if you do not set a filter on
> an offer decline. Didn't quite expect that ;-)
>
>
> On 26.09.2016 11:15, Hendrik Haddorp wrote:
>
>> I don't "own" the setup so that is not so easy. For now I'm keeping the
>> offers for a few seconds to get around it. If I get traces enabled I'll let
>> you know. I was first of all only interested if I'm supposed to get all
>> offers at once or not.
>>
>> On 26.09.2016 11:08, Guangya Liu wrote:
>>
>>> Can you please turn on the `GLOG_v=2` in mesos master and append more
>>> logs here?
>>>
>>> On Mon, Sep 26, 2016 at 4:36 PM, Hendrik Haddorp <
>>> hendrik.haddorp@gmx.net <ma...@gmx.net>> wrote:
>>>
>>> Thanks for the tip. No, I did not set any filters.
>>>
>>> On 26.09.2016 09 <tel:26.09.2016%2009>:53, Guangya Liu wrote:
>>>
>>> This depends, it may be caused by that one of the offer is
>>> declined and then got a filter, this will cause the filtered
>>> offer will be offered after the expire time. Did you set
>>> filters when decline offer? If you are enabling GLOG_v=2 for
>>> mesos master, you will get some log as "Framework xxx filtered
>>> agent <slaveId> for <seconds>"
>>>
>>> On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp
>>> <hendrik.haddorp@gmx.net <ma...@gmx.net>
>>> <mailto:hendrik.haddorp@gmx.net
>>> <ma...@gmx.net>>> wrote:
>>>
>>> Hi,
>>>
>>> I have three Mesos cluster test setups. On two my
>>> frameworks gets
>>> the resource offers from all slaves in one
>>> "resourceOffers" call.
>>> In one three node setup I do however sometimes get offers
>>> for all
>>> slaves but most of the time I get first two offers and
>>> then the
>>> third in a separate call. Should I get all offers in one
>>> call or
>>> do I need to collect offers? I couldn't find any
>>> documentation on
>>> this. Problem in my case is that I need to start some
>>> tasks only
>>> once I have offers from multiple slaves / agents. So far I
>>> expected to get all offers in one call.
>>>
>>> thanks,
>>> Hendrik
>>>
>>>
>>>
>>>
>>>
>>
>
Re: resource offers
Posted by Hendrik Haddorp <he...@gmx.net>.
Your tip helped :-) I'm seeing lines like this in the log:
mesos-master[3781]: I0926 23:36:11.292654 3805 hierarchical.cpp:927]
Framework 12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-0007 filtered slave
12da1693-ec1b-4eb6-bec9-6e6ef1b9fe11-S0 for 5secs
This is caused by Mesos doing a 5sec filter if you do not set a filter
on an offer decline. Didn't quite expect that ;-)
On 26.09.2016 11:15, Hendrik Haddorp wrote:
> I don't "own" the setup so that is not so easy. For now I'm keeping
> the offers for a few seconds to get around it. If I get traces enabled
> I'll let you know. I was first of all only interested if I'm supposed
> to get all offers at once or not.
>
> On 26.09.2016 11:08, Guangya Liu wrote:
>> Can you please turn on the `GLOG_v=2` in mesos master and append more
>> logs here?
>>
>> On Mon, Sep 26, 2016 at 4:36 PM, Hendrik Haddorp
>> <hendrik.haddorp@gmx.net <ma...@gmx.net>> wrote:
>>
>> Thanks for the tip. No, I did not set any filters.
>>
>> On 26.09.2016 09 <tel:26.09.2016%2009>:53, Guangya Liu wrote:
>>
>> This depends, it may be caused by that one of the offer is
>> declined and then got a filter, this will cause the filtered
>> offer will be offered after the expire time. Did you set
>> filters when decline offer? If you are enabling GLOG_v=2 for
>> mesos master, you will get some log as "Framework xxx filtered
>> agent <slaveId> for <seconds>"
>>
>> On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp
>> <hendrik.haddorp@gmx.net <ma...@gmx.net>
>> <mailto:hendrik.haddorp@gmx.net
>> <ma...@gmx.net>>> wrote:
>>
>> Hi,
>>
>> I have three Mesos cluster test setups. On two my
>> frameworks gets
>> the resource offers from all slaves in one
>> "resourceOffers" call.
>> In one three node setup I do however sometimes get offers
>> for all
>> slaves but most of the time I get first two offers and
>> then the
>> third in a separate call. Should I get all offers in one
>> call or
>> do I need to collect offers? I couldn't find any
>> documentation on
>> this. Problem in my case is that I need to start some
>> tasks only
>> once I have offers from multiple slaves / agents. So far I
>> expected to get all offers in one call.
>>
>> thanks,
>> Hendrik
>>
>>
>>
>>
>
Re: resource offers
Posted by Hendrik Haddorp <he...@gmx.net>.
I don't "own" the setup so that is not so easy. For now I'm keeping the
offers for a few seconds to get around it. If I get traces enabled I'll
let you know. I was first of all only interested if I'm supposed to get
all offers at once or not.
On 26.09.2016 11:08, Guangya Liu wrote:
> Can you please turn on the `GLOG_v=2` in mesos master and append more
> logs here?
>
> On Mon, Sep 26, 2016 at 4:36 PM, Hendrik Haddorp
> <hendrik.haddorp@gmx.net <ma...@gmx.net>> wrote:
>
> Thanks for the tip. No, I did not set any filters.
>
> On 26.09.2016 09 <tel:26.09.2016%2009>:53, Guangya Liu wrote:
>
> This depends, it may be caused by that one of the offer is
> declined and then got a filter, this will cause the filtered
> offer will be offered after the expire time. Did you set
> filters when decline offer? If you are enabling GLOG_v=2 for
> mesos master, you will get some log as "Framework xxx filtered
> agent <slaveId> for <seconds>"
>
> On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp
> <hendrik.haddorp@gmx.net <ma...@gmx.net>
> <mailto:hendrik.haddorp@gmx.net
> <ma...@gmx.net>>> wrote:
>
> Hi,
>
> I have three Mesos cluster test setups. On two my
> frameworks gets
> the resource offers from all slaves in one
> "resourceOffers" call.
> In one three node setup I do however sometimes get offers
> for all
> slaves but most of the time I get first two offers and
> then the
> third in a separate call. Should I get all offers in one
> call or
> do I need to collect offers? I couldn't find any
> documentation on
> this. Problem in my case is that I need to start some
> tasks only
> once I have offers from multiple slaves / agents. So far I
> expected to get all offers in one call.
>
> thanks,
> Hendrik
>
>
>
>
Re: resource offers
Posted by Guangya Liu <gy...@gmail.com>.
Can you please turn on the `GLOG_v=2` in mesos master and append more logs
here?
On Mon, Sep 26, 2016 at 4:36 PM, Hendrik Haddorp <he...@gmx.net>
wrote:
> Thanks for the tip. No, I did not set any filters.
>
> On 26.09.2016 09:53, Guangya Liu wrote:
>
>> This depends, it may be caused by that one of the offer is declined and
>> then got a filter, this will cause the filtered offer will be offered after
>> the expire time. Did you set filters when decline offer? If you are
>> enabling GLOG_v=2 for mesos master, you will get some log as "Framework xxx
>> filtered agent <slaveId> for <seconds>"
>>
>> On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp <hendrik.haddorp@gmx.net
>> <ma...@gmx.net>> wrote:
>>
>> Hi,
>>
>> I have three Mesos cluster test setups. On two my frameworks gets
>> the resource offers from all slaves in one "resourceOffers" call.
>> In one three node setup I do however sometimes get offers for all
>> slaves but most of the time I get first two offers and then the
>> third in a separate call. Should I get all offers in one call or
>> do I need to collect offers? I couldn't find any documentation on
>> this. Problem in my case is that I need to start some tasks only
>> once I have offers from multiple slaves / agents. So far I
>> expected to get all offers in one call.
>>
>> thanks,
>> Hendrik
>>
>>
>>
>
Re: resource offers
Posted by Hendrik Haddorp <he...@gmx.net>.
Thanks for the tip. No, I did not set any filters.
On 26.09.2016 09:53, Guangya Liu wrote:
> This depends, it may be caused by that one of the offer is declined
> and then got a filter, this will cause the filtered offer will be
> offered after the expire time. Did you set filters when decline offer?
> If you are enabling GLOG_v=2 for mesos master, you will get some log
> as "Framework xxx filtered agent <slaveId> for <seconds>"
>
> On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp
> <hendrik.haddorp@gmx.net <ma...@gmx.net>> wrote:
>
> Hi,
>
> I have three Mesos cluster test setups. On two my frameworks gets
> the resource offers from all slaves in one "resourceOffers" call.
> In one three node setup I do however sometimes get offers for all
> slaves but most of the time I get first two offers and then the
> third in a separate call. Should I get all offers in one call or
> do I need to collect offers? I couldn't find any documentation on
> this. Problem in my case is that I need to start some tasks only
> once I have offers from multiple slaves / agents. So far I
> expected to get all offers in one call.
>
> thanks,
> Hendrik
>
>
Re: resource offers
Posted by Guangya Liu <gy...@gmail.com>.
This depends, it may be caused by that one of the offer is declined and
then got a filter, this will cause the filtered offer will be offered after
the expire time. Did you set filters when decline offer? If you are
enabling GLOG_v=2 for mesos master, you will get some log as "Framework xxx
filtered agent <slaveId> for <seconds>"
On Mon, Sep 26, 2016 at 2:47 PM, Hendrik Haddorp <he...@gmx.net>
wrote:
> Hi,
>
> I have three Mesos cluster test setups. On two my frameworks gets the
> resource offers from all slaves in one "resourceOffers" call. In one three
> node setup I do however sometimes get offers for all slaves but most of the
> time I get first two offers and then the third in a separate call. Should I
> get all offers in one call or do I need to collect offers? I couldn't find
> any documentation on this. Problem in my case is that I need to start some
> tasks only once I have offers from multiple slaves / agents. So far I
> expected to get all offers in one call.
>
> thanks,
> Hendrik
>