You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by F21 <f2...@gmail.com> on 2015/09/03 03:12:51 UTC
How does mesos determine how much memory on a node is available for
offer?
I have 3 CoreOS nodes running in vagrant. Mesos is run natively (not in
docker containers).
There is 1 master/slave and 2 slaves.
If I ssh into one of my slaves and run free -m, I see:
Total: 2005
Used 1342
Free 662
Shares 273
Buffers 13
Cached 1210
In the mesos web-ui, I see that the slave has 1002 MB of memory to offer.
How is this 1002 MB determined (I am running the masters and slaves with
stock defaults and no customizations)?
Is the 1002MB included in the used memory (1342)? If so, why is the
662MB free? It seems to be a waste and I am sure it should be able to
offer another 500MB making the total 1502MB.
Re: How does mesos determine how much memory on a node is available
for offer?
Posted by Klaus Ma <kl...@cguru.net>.
Yes; if totalMemory > 2G, report totalMemory - 1G; otherwise, report
totalMemory/2.
On 2015年09月03日 20:11, Alex Rukletsov wrote:
> Mesos agent (aka slave) estimates the memory available and advertises
> all of it minus 1GB. If there is less than 2GB available, only half is
> advertised [1].
>
> [1]:
> https://github.com/apache/mesos/blob/master/src/slave/containerizer/containerizer.cpp#L98
>
> On Thu, Sep 3, 2015 at 4:01 AM, Anand Mazumdar <anand@mesosphere.io
> <ma...@mesosphere.io>> wrote:
>
> My bad, Seeing the 1002mb(~1024) number made me think the agent
> was not able to get the memory estimates from the OS and
> defaulting to the constant values.
>
> The slave executes a `sysinfo` system call and populates the
> memory numbers based on it. If you want a more fine-grained
> control, try to specify it directly using the —resources flag as I
> had mentioned earlier.
>
> -anand
>
>> On Sep 2, 2015, at 6:48 PM, F21 <f21.groups@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> There seems to be some dynamicness to it. I just bumped the
>> memory for each VM up to 2.5GB and now mesos is offering 1.5GB on
>> it's slave. Is there some percentage value that I can set so that
>> more memory is available to mesos?
>>
>> On 3/09/2015 11:23 AM, Anand Mazumdar wrote:
>>> In case you don’t specify the resources via “—resources” flag
>>> when you start your agent, it picks up the default values.
>>> (Example: --resources="cpus:4;mem:1024;disk:20000”)
>>>
>>> The default value for memory is here:
>>> https://github.com/apache/mesos/blob/master/src/slave/constants.cpp#L46
>>>
>>> -anand
>>>
>
>
--
Klaus Ma (马达), PMP® | http://www.cguru.net
Re: How does mesos determine how much memory on a node is available
for offer?
Posted by Alex Rukletsov <al...@mesosphere.com>.
Mesos agent (aka slave) estimates the memory available and advertises all
of it minus 1GB. If there is less than 2GB available, only half is
advertised [1].
[1]:
https://github.com/apache/mesos/blob/master/src/slave/containerizer/containerizer.cpp#L98
On Thu, Sep 3, 2015 at 4:01 AM, Anand Mazumdar <an...@mesosphere.io> wrote:
> My bad, Seeing the 1002mb(~1024) number made me think the agent was not
> able to get the memory estimates from the OS and defaulting to the constant
> values.
>
> The slave executes a `sysinfo` system call and populates the memory
> numbers based on it. If you want a more fine-grained control, try to
> specify it directly using the —resources flag as I had mentioned earlier.
>
> -anand
>
> On Sep 2, 2015, at 6:48 PM, F21 <f2...@gmail.com> wrote:
>
> There seems to be some dynamicness to it. I just bumped the memory for
> each VM up to 2.5GB and now mesos is offering 1.5GB on it's slave. Is there
> some percentage value that I can set so that more memory is available to
> mesos?
>
> On 3/09/2015 11:23 AM, Anand Mazumdar wrote:
>
> In case you don’t specify the resources via “—resources” flag when you
> start your agent, it picks up the default values. (Example:
> --resources="cpus:4;mem:1024;disk:20000”)
>
> The default value for memory is here:
> <https://github.com/apache/mesos/blob/master/src/slave/constants.cpp#L46>
> https://github.com/apache/mesos/blob/master/src/slave/constants.cpp#L46
>
> -anand
>
>
>
Re: How does mesos determine how much memory on a node is available for offer?
Posted by Anand Mazumdar <an...@mesosphere.io>.
My bad, Seeing the 1002mb(~1024) number made me think the agent was not able to get the memory estimates from the OS and defaulting to the constant values.
The slave executes a `sysinfo` system call and populates the memory numbers based on it. If you want a more fine-grained control, try to specify it directly using the —resources flag as I had mentioned earlier.
-anand
> On Sep 2, 2015, at 6:48 PM, F21 <f2...@gmail.com> wrote:
>
> There seems to be some dynamicness to it. I just bumped the memory for each VM up to 2.5GB and now mesos is offering 1.5GB on it's slave. Is there some percentage value that I can set so that more memory is available to mesos?
>
> On 3/09/2015 11:23 AM, Anand Mazumdar wrote:
>> In case you don’t specify the resources via “—resources” flag when you start your agent, it picks up the default values. (Example: --resources="cpus:4;mem:1024;disk:20000”)
>>
>> The default value for memory is here: <https://github.com/apache/mesos/blob/master/src/slave/constants.cpp#L46>https://github.com/apache/mesos/blob/master/src/slave/constants.cpp#L46 <https://github.com/apache/mesos/blob/master/src/slave/constants.cpp#L46>
>>
>> -anand
>>
Re: How does mesos determine how much memory on a node is available
for offer?
Posted by F21 <f2...@gmail.com>.
There seems to be some dynamicness to it. I just bumped the memory for
each VM up to 2.5GB and now mesos is offering 1.5GB on it's slave. Is
there some percentage value that I can set so that more memory is
available to mesos?
On 3/09/2015 11:23 AM, Anand Mazumdar wrote:
> In case you don’t specify the resources via “—resources” flag when you
> start your agent, it picks up the default values. (Example:
> --resources="cpus:4;mem:1024;disk:20000”)
>
> The default value for memory is here:
> https://github.com/apache/mesos/blob/master/src/slave/constants.cpp#L46
>
> -anand
>
Re: How does mesos determine how much memory on a node is available for offer?
Posted by Anand Mazumdar <an...@mesosphere.io>.
In case you don’t specify the resources via “—resources” flag when you start your agent, it picks up the default values. (Example: --resources="cpus:4;mem:1024;disk:20000”)
The default value for memory is here: https://github.com/apache/mesos/blob/master/src/slave/constants.cpp#L46 <https://github.com/apache/mesos/blob/master/src/slave/constants.cpp#L46>
-anand
> On Sep 2, 2015, at 6:12 PM, F21 <f2...@gmail.com> wrote:
>
> I have 3 CoreOS nodes running in vagrant. Mesos is run natively (not in docker containers).
>
> There is 1 master/slave and 2 slaves.
>
> If I ssh into one of my slaves and run free -m, I see:
>
> Total: 2005
> Used 1342
> Free 662
> Shares 273
> Buffers 13
> Cached 1210
>
> In the mesos web-ui, I see that the slave has 1002 MB of memory to offer.
>
> How is this 1002 MB determined (I am running the masters and slaves with stock defaults and no customizations)?
>
> Is the 1002MB included in the used memory (1342)? If so, why is the 662MB free? It seems to be a waste and I am sure it should be able to offer another 500MB making the total 1502MB.