You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by ASHUTOSH JAIN <ha...@gmail.com> on 2014/03/17 19:07:11 UTC
Review Request 19312: TODO(benh): choose a better scheme(% based) to set mem
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/
-----------------------------------------------------------
Review request for mesos and Ben Mahler.
Bugs: MESOS-1105
https://issues.apache.org/jira/browse/MESOS-1105
Repository: mesos-git
Description
-------
using 50% as criteria.
Diffs
-----
src/slave/containerizer/containerizer.cpp d0a1023
Diff: https://reviews.apache.org/r/19312/diff/
Testing
-------
make check
Thanks,
ASHUTOSH JAIN
Re: Review Request 19312: For memory more than 2GB,
leave 1GB free else use 50% of system memory for the slave
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37769
-----------------------------------------------------------
Patch looks great!
Reviews applied: [19312]
All tests passed.
- Mesos ReviewBot
On March 19, 2014, 6:35 p.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 19, 2014, 6:35 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria for mem less than 2GB else leave 1GB for OS.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: For memory more than 2GB,
leave 1GB free else use 50% of system memory for the slave
Posted by Jiang Yan Xu <ya...@jxu.me>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37786
-----------------------------------------------------------
Ship it!
src/slave/containerizer/containerizer.cpp
<https://reviews.apache.org/r/19312/#comment69509>
rm empty line.
- Jiang Yan Xu
On March 19, 2014, 11:35 a.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 19, 2014, 11:35 a.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria for mem less than 2GB else leave 1GB for OS.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: For memory more than 2GB,
leave 1GB free else use 50% of system memory for the slave
Posted by ASHUTOSH JAIN <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37789
-----------------------------------------------------------
src/slave/containerizer/containerizer.cpp
<https://reviews.apache.org/r/19312/#comment69513>
ok
- ASHUTOSH JAIN
On March 19, 2014, 6:35 p.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 19, 2014, 6:35 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria for mem less than 2GB else leave 1GB for OS.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: For memory more than 2GB,
leave 1GB free else use 50% of system memory for the slave
Posted by Adam B <ad...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37822
-----------------------------------------------------------
Ship it!
Ship It!
- Adam B
On March 19, 2014, 1:25 p.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 19, 2014, 1:25 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria for mem less than 2GB else leave 1GB for OS.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: For memory more than 2GB,
leave 1GB free else use 50% of system memory for the slave
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37799
-----------------------------------------------------------
Patch looks great!
Reviews applied: [19312]
All tests passed.
- Mesos ReviewBot
On March 19, 2014, 8:25 p.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 19, 2014, 8:25 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria for mem less than 2GB else leave 1GB for OS.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: For memory more than 2GB,
leave 1GB free else use 50% of system memory for the slave
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37973
-----------------------------------------------------------
Ship it!
Ship It!
- Ben Mahler
On March 19, 2014, 8:25 p.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 19, 2014, 8:25 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria for mem less than 2GB else leave 1GB for OS.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: For memory more than 2GB,
leave 1GB free else use 50% of system memory for the slave
Posted by ASHUTOSH JAIN <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/
-----------------------------------------------------------
(Updated March 19, 2014, 8:25 p.m.)
Review request for mesos and Ben Mahler.
Changes
-------
additional space removed
Bugs: MESOS-1105
https://issues.apache.org/jira/browse/MESOS-1105
Repository: mesos-git
Description
-------
using 50% as criteria for mem less than 2GB else leave 1GB for OS.
Diffs (updated)
-----
src/slave/containerizer/containerizer.cpp d0a1023
Diff: https://reviews.apache.org/r/19312/diff/
Testing
-------
make check
Thanks,
ASHUTOSH JAIN
Re: Review Request 19312: For memory more than 2GB,
leave 1GB free else use 50% of system memory for the slave
Posted by ASHUTOSH JAIN <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/
-----------------------------------------------------------
(Updated March 19, 2014, 6:35 p.m.)
Review request for mesos and Ben Mahler.
Changes
-------
new diff according to suggestion by Bmahler
Bugs: MESOS-1105
https://issues.apache.org/jira/browse/MESOS-1105
Repository: mesos-git
Description
-------
using 50% as criteria for mem less than 2GB else leave 1GB for OS.
Diffs (updated)
-----
src/slave/containerizer/containerizer.cpp d0a1023
Diff: https://reviews.apache.org/r/19312/diff/
Testing
-------
make check
Thanks,
ASHUTOSH JAIN
Re: Review Request 19312: For memory more than 2GB,
leave 1GB free else use 50% of system memory for the slave
Posted by ASHUTOSH JAIN <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/
-----------------------------------------------------------
(Updated March 19, 2014, 7:15 a.m.)
Review request for mesos and Ben Mahler.
Summary (updated)
-----------------
For memory more than 2GB, leave 1GB free else use 50% of system memory for the slave
Bugs: MESOS-1105
https://issues.apache.org/jira/browse/MESOS-1105
Repository: mesos-git
Description
-------
using 50% as criteria for mem less than 2GB else leave 1GB for OS.
Diffs
-----
src/slave/containerizer/containerizer.cpp d0a1023
Diff: https://reviews.apache.org/r/19312/diff/
Testing
-------
make check
Thanks,
ASHUTOSH JAIN
Re: Review Request 19312: TODO(benh): choose a better scheme(% based) to set
mem
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37673
-----------------------------------------------------------
This is indeed a better approach! Can you update the review summary to describe your change? The summary is ultimately what will go into the commit message.
src/slave/containerizer/containerizer.cpp
<https://reviews.apache.org/r/19312/#comment69302>
How about:
} else {
Bytes total = mem_.get().total;
if (total >= Gigabytes(2)) {
mem = total - Gigabytes(1); // Leave 1GB free.
} else {
mem = Bytes(total.bytes() / 2); // Use 50% of the memory.
}
}
- Ben Mahler
On March 18, 2014, 5 a.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 18, 2014, 5 a.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria for mem less than 2GB else leave 1GB for OS.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: TODO(benh): choose a better scheme(% based) to set
mem
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37558
-----------------------------------------------------------
Patch looks great!
Reviews applied: [19312]
All tests passed.
- Mesos ReviewBot
On March 18, 2014, 5 a.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 18, 2014, 5 a.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria for mem less than 2GB else leave 1GB for OS.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: TODO(benh): choose a better scheme(% based) to set
mem
Posted by ASHUTOSH JAIN <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/
-----------------------------------------------------------
(Updated March 18, 2014, 5 a.m.)
Review request for mesos and Ben Mahler.
Bugs: MESOS-1105
https://issues.apache.org/jira/browse/MESOS-1105
Repository: mesos-git
Description (updated)
-------
using 50% as criteria for mem less than 2GB else leave 1GB for OS.
Diffs
-----
src/slave/containerizer/containerizer.cpp d0a1023
Diff: https://reviews.apache.org/r/19312/diff/
Testing
-------
make check
Thanks,
ASHUTOSH JAIN
Re: Review Request 19312: TODO(benh): choose a better scheme(% based) to set
mem
Posted by ASHUTOSH JAIN <ha...@gmail.com>.
> On March 17, 2014, 7:11 p.m., Ben Mahler wrote:
> > src/slave/containerizer/containerizer.cpp, lines 105-112
> > <https://reviews.apache.org/r/19312/diff/1/?file=523002#file523002line105>
> >
> > Can you elaborate on why you think using 50% of the memory is a better approach than leaving 1GB free? I think most users are looking for the slave to offer the "right" amount of memory, leaving enough for the operating system and the slave itself. Given this, 50% seems too conservative.
> >
> > I'm not sure how beneficial a percentage-based approach is here, since it assumes that the amount of memory overhead of the OS/mesos-slave is relative to the total memory on the machine.
> >
> > It would be great to look at the cases where this logic leads to a poor choice of memory allocation:
> >
> > 1. If the machine has 1.1GB we will take 1GB and leave .1GB for OS/mesos-slave.
> >
> > 2. If the machine has 768MB, we will take 768MB and leave 0MB for the OS/mesos-slave.
I think you have explained the problem with the earlier code .
If a machine has less than 1GB mem then we take all of it, leaving none for the OS. So I have provided some memory for OS as well.
- ASHUTOSH
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37431
-----------------------------------------------------------
On March 17, 2014, 6:07 p.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 17, 2014, 6:07 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: TODO(benh): choose a better scheme(% based) to set
mem
Posted by Ben Mahler <be...@gmail.com>.
> On March 17, 2014, 7:11 p.m., Ben Mahler wrote:
> > src/slave/containerizer/containerizer.cpp, lines 105-112
> > <https://reviews.apache.org/r/19312/diff/1/?file=523002#file523002line105>
> >
> > Can you elaborate on why you think using 50% of the memory is a better approach than leaving 1GB free? I think most users are looking for the slave to offer the "right" amount of memory, leaving enough for the operating system and the slave itself. Given this, 50% seems too conservative.
> >
> > I'm not sure how beneficial a percentage-based approach is here, since it assumes that the amount of memory overhead of the OS/mesos-slave is relative to the total memory on the machine.
> >
> > It would be great to look at the cases where this logic leads to a poor choice of memory allocation:
> >
> > 1. If the machine has 1.1GB we will take 1GB and leave .1GB for OS/mesos-slave.
> >
> > 2. If the machine has 768MB, we will take 768MB and leave 0MB for the OS/mesos-slave.
>
> ASHUTOSH JAIN wrote:
> I think you have explained the problem with the earlier code .
> If a machine has less than 1GB mem then we take all of it, leaving none for the OS. So I have provided some memory for OS as well.
>
>
And what happens in your code when the machine has 32GB or memory?
The old behavior was to offer 31GB and reserve 1GB for the OS/mesos-slave.
The new behavior you're proposing is to offer 16Gb and reserve 16GB for the OS/mesos-slave.
- Ben
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37431
-----------------------------------------------------------
On March 17, 2014, 6:07 p.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 17, 2014, 6:07 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: TODO(benh): choose a better scheme(% based) to set
mem
Posted by ASHUTOSH JAIN <ha...@gmail.com>.
> On March 17, 2014, 7:11 p.m., Ben Mahler wrote:
> > src/slave/containerizer/containerizer.cpp, lines 105-112
> > <https://reviews.apache.org/r/19312/diff/1/?file=523002#file523002line105>
> >
> > Can you elaborate on why you think using 50% of the memory is a better approach than leaving 1GB free? I think most users are looking for the slave to offer the "right" amount of memory, leaving enough for the operating system and the slave itself. Given this, 50% seems too conservative.
> >
> > I'm not sure how beneficial a percentage-based approach is here, since it assumes that the amount of memory overhead of the OS/mesos-slave is relative to the total memory on the machine.
> >
> > It would be great to look at the cases where this logic leads to a poor choice of memory allocation:
> >
> > 1. If the machine has 1.1GB we will take 1GB and leave .1GB for OS/mesos-slave.
> >
> > 2. If the machine has 768MB, we will take 768MB and leave 0MB for the OS/mesos-slave.
>
> ASHUTOSH JAIN wrote:
> I think you have explained the problem with the earlier code .
> If a machine has less than 1GB mem then we take all of it, leaving none for the OS. So I have provided some memory for OS as well.
>
>
>
> Ben Mahler wrote:
> And what happens in your code when the machine has 32GB or memory?
>
> The old behavior was to offer 31GB and reserve 1GB for the OS/mesos-slave.
> The new behavior you're proposing is to offer 16Gb and reserve 16GB for the OS/mesos-slave.
If we have 32GB of memory then my code will return 31GB. Actually my code returns 50% when there is less than 2GB of memory else it returns mem - 1GB.
- ASHUTOSH
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37431
-----------------------------------------------------------
On March 17, 2014, 6:07 p.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 17, 2014, 6:07 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>
Re: Review Request 19312: TODO(benh): choose a better scheme(% based) to set
mem
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19312/#review37431
-----------------------------------------------------------
src/slave/containerizer/containerizer.cpp
<https://reviews.apache.org/r/19312/#comment68957>
Can you elaborate on why you think using 50% of the memory is a better approach than leaving 1GB free? I think most users are looking for the slave to offer the "right" amount of memory, leaving enough for the operating system and the slave itself. Given this, 50% seems too conservative.
I'm not sure how beneficial a percentage-based approach is here, since it assumes that the amount of memory overhead of the OS/mesos-slave is relative to the total memory on the machine.
It would be great to look at the cases where this logic leads to a poor choice of memory allocation:
1. If the machine has 1.1GB we will take 1GB and leave .1GB for OS/mesos-slave.
2. If the machine has 768MB, we will take 768MB and leave 0MB for the OS/mesos-slave.
- Ben Mahler
On March 17, 2014, 6:07 p.m., ASHUTOSH JAIN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19312/
> -----------------------------------------------------------
>
> (Updated March 17, 2014, 6:07 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1105
> https://issues.apache.org/jira/browse/MESOS-1105
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> using 50% as criteria.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.cpp d0a1023
>
> Diff: https://reviews.apache.org/r/19312/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> ASHUTOSH JAIN
>
>