You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Ondrej Smola <on...@gmail.com> on 2015/05/21 13:04:12 UTC
(Unknown)
Hi Aaron,
You can set memory in /etc/mesos-slave/resources
example:
cpus(*):4;mem(*):16067;ports(*):[80-80,31000-32000]
with this configuration mesos offers 15.7GB RAM on one of our nodes.
2015-05-21 12:51 GMT+02:00 Aaron Carey <ac...@ilm.com>:
> I've managed to increase the disksize by playing with some docker
> options,
>
> Anyone have any idea about the memory?
>
> Thanks,
> Aaron
>
> ------------------------------
> *From:* Aaron Carey [acarey@ilm.com]
> *Sent:* 21 May 2015 11:19
> *To:* user@mesos.apache.org
> *Subject:* How slaves calculate resources
>
> Hi,
>
> I was just trying to figure out how Mesos slaves report the amount of
> resources available to them on the host?
>
> We have some slaves running on AWS t2.medium machines (2cpu, 4Gb RAM) with
> 32GB disks.
>
> The slaves are running inside docker containers.
>
> They report 2 cpus (correct), 2.5GB RAM and 4.9GB disk.
>
> Any ideas why this is different from what I can see on the machine? (both
> on the host and within the slave docker container)?
>
> Thanks,
> Aaron
>
RE: Re:
Posted by Aaron Carey <ac...@ilm.com>.
Ah perfect! Thanks for the info!
________________________________
From: Adam Bordelon [adam@mesosphere.io]
Sent: 01 June 2015 06:48
To: user@mesos.apache.org
Subject: Re:
FYI, Mesos will exclude 1GB from what it auto-detects, so that the mesos-slave process and other system processes can use some memory. See https://github.com/apache/mesos/blob/0.22.1/src/slave/containerizer/containerizer.cpp#L107
If you explicitly set the memory requirements as Ondrej suggests, you can override this. However, you run the risk of your tasks consuming all the memory in the system so that Mesos itself cannot run effectively.
On Thu, May 21, 2015 at 4:24 AM, Ondrej Smola <on...@gmail.com>> wrote:
It is little more complicated and it depends on your environment - you need to give some RAM to OS and running processes (Mesos, Docker etc.). Quick test - VM with 3GB RAM and Mesos offers 1.9G - so there should is no problem related to your mesos setup (mesos in both cases offers around 63% of RAM).
About manual setup: you can use some automation tool (Ansible, Puppet) if you plan to setup large number of nodes.
2015-05-21 13:10 GMT+02:00 Aaron Carey <ac...@ilm.com>>:
Thanks Ondrej,
Do I have to do this? I was under the impression if you didn't specify the resources then mesos would just offer everything available?
Thanks,
Aaron
________________________________
From: Ondrej Smola [ondrej.smola@gmail.com<ma...@gmail.com>]
Sent: 21 May 2015 12:04
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject:
Hi Aaron,
You can set memory in /etc/mesos-slave/resources
example:
cpus(*):4;mem(*):16067;ports(*):[80-80,31000-32000]
with this configuration mesos offers 15.7GB RAM on one of our nodes.
2015-05-21 12:51 GMT+02:00 Aaron Carey <ac...@ilm.com>>:
I've managed to increase the disksize by playing with some docker options,
Anyone have any idea about the memory?
Thanks,
Aaron
________________________________
From: Aaron Carey [acarey@ilm.com<ma...@ilm.com>]
Sent: 21 May 2015 11:19
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: How slaves calculate resources
Hi,
I was just trying to figure out how Mesos slaves report the amount of resources available to them on the host?
We have some slaves running on AWS t2.medium machines (2cpu, 4Gb RAM) with 32GB disks.
The slaves are running inside docker containers.
They report 2 cpus (correct), 2.5GB RAM and 4.9GB disk.
Any ideas why this is different from what I can see on the machine? (both on the host and within the slave docker container)?
Thanks,
Aaron
Re:
Posted by Adam Bordelon <ad...@mesosphere.io>.
FYI, Mesos will exclude 1GB from what it auto-detects, so that the
mesos-slave process and other system processes can use some memory. See
https://github.com/apache/mesos/blob/0.22.1/src/slave/containerizer/containerizer.cpp#L107
If you explicitly set the memory requirements as Ondrej suggests, you can
override this. However, you run the risk of your tasks consuming all the
memory in the system so that Mesos itself cannot run effectively.
On Thu, May 21, 2015 at 4:24 AM, Ondrej Smola <on...@gmail.com>
wrote:
> It is little more complicated and it depends on your environment - you
> need to give some RAM to OS and running processes (Mesos, Docker etc.).
> Quick test - VM with 3GB RAM and Mesos offers 1.9G - so there should is no
> problem related to your mesos setup (mesos in both cases offers around 63%
> of RAM).
>
> About manual setup: you can use some automation tool (Ansible, Puppet) if
> you plan to setup large number of nodes.
>
>
>
> 2015-05-21 13:10 GMT+02:00 Aaron Carey <ac...@ilm.com>:
>
>> Thanks Ondrej,
>>
>> Do I have to do this? I was under the impression if you didn't specify
>> the resources then mesos would just offer everything available?
>>
>> Thanks,
>> Aaron
>>
>> ------------------------------
>> *From:* Ondrej Smola [ondrej.smola@gmail.com]
>> *Sent:* 21 May 2015 12:04
>> *To:* user@mesos.apache.org
>> *Subject:*
>>
>> Hi Aaron,
>>
>> You can set memory in /etc/mesos-slave/resources
>>
>> example:
>>
>> cpus(*):4;mem(*):16067;ports(*):[80-80,31000-32000]
>>
>> with this configuration mesos offers 15.7GB RAM on one of our nodes.
>>
>>
>>
>>
>>
>>
>>
>> 2015-05-21 12:51 GMT+02:00 Aaron Carey <ac...@ilm.com>:
>>
>>> I've managed to increase the disksize by playing with some docker
>>> options,
>>>
>>> Anyone have any idea about the memory?
>>>
>>> Thanks,
>>> Aaron
>>>
>>> ------------------------------
>>> *From:* Aaron Carey [acarey@ilm.com]
>>> *Sent:* 21 May 2015 11:19
>>> *To:* user@mesos.apache.org
>>> *Subject:* How slaves calculate resources
>>>
>>> Hi,
>>>
>>> I was just trying to figure out how Mesos slaves report the amount of
>>> resources available to them on the host?
>>>
>>> We have some slaves running on AWS t2.medium machines (2cpu, 4Gb RAM)
>>> with 32GB disks.
>>>
>>> The slaves are running inside docker containers.
>>>
>>> They report 2 cpus (correct), 2.5GB RAM and 4.9GB disk.
>>>
>>> Any ideas why this is different from what I can see on the machine?
>>> (both on the host and within the slave docker container)?
>>>
>>> Thanks,
>>> Aaron
>>>
>>
>>
>
Re:
Posted by Ondrej Smola <on...@gmail.com>.
It is little more complicated and it depends on your environment - you need
to give some RAM to OS and running processes (Mesos, Docker etc.). Quick
test - VM with 3GB RAM and Mesos offers 1.9G - so there should is no
problem related to your mesos setup (mesos in both cases offers around 63%
of RAM).
About manual setup: you can use some automation tool (Ansible, Puppet) if
you plan to setup large number of nodes.
2015-05-21 13:10 GMT+02:00 Aaron Carey <ac...@ilm.com>:
> Thanks Ondrej,
>
> Do I have to do this? I was under the impression if you didn't specify the
> resources then mesos would just offer everything available?
>
> Thanks,
> Aaron
>
> ------------------------------
> *From:* Ondrej Smola [ondrej.smola@gmail.com]
> *Sent:* 21 May 2015 12:04
> *To:* user@mesos.apache.org
> *Subject:*
>
> Hi Aaron,
>
> You can set memory in /etc/mesos-slave/resources
>
> example:
>
> cpus(*):4;mem(*):16067;ports(*):[80-80,31000-32000]
>
> with this configuration mesos offers 15.7GB RAM on one of our nodes.
>
>
>
>
>
>
>
> 2015-05-21 12:51 GMT+02:00 Aaron Carey <ac...@ilm.com>:
>
>> I've managed to increase the disksize by playing with some docker
>> options,
>>
>> Anyone have any idea about the memory?
>>
>> Thanks,
>> Aaron
>>
>> ------------------------------
>> *From:* Aaron Carey [acarey@ilm.com]
>> *Sent:* 21 May 2015 11:19
>> *To:* user@mesos.apache.org
>> *Subject:* How slaves calculate resources
>>
>> Hi,
>>
>> I was just trying to figure out how Mesos slaves report the amount of
>> resources available to them on the host?
>>
>> We have some slaves running on AWS t2.medium machines (2cpu, 4Gb RAM)
>> with 32GB disks.
>>
>> The slaves are running inside docker containers.
>>
>> They report 2 cpus (correct), 2.5GB RAM and 4.9GB disk.
>>
>> Any ideas why this is different from what I can see on the machine? (both
>> on the host and within the slave docker container)?
>>
>> Thanks,
>> Aaron
>>
>
>
RE:
Posted by Aaron Carey <ac...@ilm.com>.
Thanks Ondrej,
Do I have to do this? I was under the impression if you didn't specify the resources then mesos would just offer everything available?
Thanks,
Aaron
________________________________
From: Ondrej Smola [ondrej.smola@gmail.com]
Sent: 21 May 2015 12:04
To: user@mesos.apache.org
Subject:
Hi Aaron,
You can set memory in /etc/mesos-slave/resources
example:
cpus(*):4;mem(*):16067;ports(*):[80-80,31000-32000]
with this configuration mesos offers 15.7GB RAM on one of our nodes.
2015-05-21 12:51 GMT+02:00 Aaron Carey <ac...@ilm.com>>:
I've managed to increase the disksize by playing with some docker options,
Anyone have any idea about the memory?
Thanks,
Aaron
________________________________
From: Aaron Carey [acarey@ilm.com<ma...@ilm.com>]
Sent: 21 May 2015 11:19
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: How slaves calculate resources
Hi,
I was just trying to figure out how Mesos slaves report the amount of resources available to them on the host?
We have some slaves running on AWS t2.medium machines (2cpu, 4Gb RAM) with 32GB disks.
The slaves are running inside docker containers.
They report 2 cpus (correct), 2.5GB RAM and 4.9GB disk.
Any ideas why this is different from what I can see on the machine? (both on the host and within the slave docker container)?
Thanks,
Aaron