You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Joris Van Remoortere <jo...@gmail.com> on 2015/02/10 00:21:47 UTC
Review Request 30808: Fix high CPU bug. MESOS-2325.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30808/
-----------------------------------------------------------
Review request for mesos, Benjamin Hindman, Ben Mahler, Jie Yu, Niklas Nielsen, and Vinod Kone.
Bugs: MESOS-2325
https://issues.apache.org/jira/browse/MESOS-2325
Repository: mesos
Description
-------
1) Maintain an outstanding timers set to prevent scheduling multiple delay functions for the same timeout.
2) Change comment and behavior for paused clock to match original behavior.
Ended up moving tick() into the clock namespace to make it more clear that next() and tick() share this global set of outstanding timers.
The original behavior used a single timer and adjusted the timeout. We didn't go back to this as benh wanted a generalized way to delay an arbitrary function. Therefore we handle the maintenance of this optimization in next() and tick() rather than pushing it down into the delay_function and management of ev_timers.
Diffs
-----
3rdparty/libprocess/src/clock.cpp fcc1eb0b921b3d9b9c033a17e5f6dcecf0a30f08
Diff: https://reviews.apache.org/r/30808/diff/
Testing
-------
make check
A worst-case scenario test for the example listed in MESOS-2325. Constructed many timers in decrementing time (i.e. 500, 499,498, etc.). CPU utilization during this test went down greatly and is on par with code prior to the clock refactoring changes.
Thanks,
Joris Van Remoortere
Re: Review Request 30808: Fix high CPU bug. MESOS-2325.
Posted by Joris Van Remoortere <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30808/#review71723
-----------------------------------------------------------
3rdparty/libprocess/src/clock.cpp
<https://reviews.apache.org/r/30808/#comment117590>
return None() if we don't want to fire.
- Joris Van Remoortere
On Feb. 9, 2015, 11:21 p.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30808/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 11:21 p.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, Jie Yu, Niklas Nielsen, and Vinod Kone.
>
>
> Bugs: MESOS-2325
> https://issues.apache.org/jira/browse/MESOS-2325
>
>
> Repository: mesos
>
>
> Description
> -------
>
> 1) Maintain an outstanding timers set to prevent scheduling multiple delay functions for the same timeout.
> 2) Change comment and behavior for paused clock to match original behavior.
>
> Ended up moving tick() into the clock namespace to make it more clear that next() and tick() share this global set of outstanding timers.
>
> The original behavior used a single timer and adjusted the timeout. We didn't go back to this as benh wanted a generalized way to delay an arbitrary function. Therefore we handle the maintenance of this optimization in next() and tick() rather than pushing it down into the delay_function and management of ev_timers.
>
>
> Diffs
> -----
>
> 3rdparty/libprocess/src/clock.cpp fcc1eb0b921b3d9b9c033a17e5f6dcecf0a30f08
>
> Diff: https://reviews.apache.org/r/30808/diff/
>
>
> Testing
> -------
>
> make check
> A worst-case scenario test for the example listed in MESOS-2325. Constructed many timers in decrementing time (i.e. 500, 499,498, etc.). CPU utilization during this test went down greatly and is on par with code prior to the clock refactoring changes.
>
>
> Thanks,
>
> Joris Van Remoortere
>
>
Re: Review Request 30808: Fix high CPU bug. MESOS-2325.
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30808/#review71782
-----------------------------------------------------------
Patch looks great!
Reviews applied: [30808]
All tests passed.
- Mesos ReviewBot
On Feb. 10, 2015, 2:22 a.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30808/
> -----------------------------------------------------------
>
> (Updated Feb. 10, 2015, 2:22 a.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, Jie Yu, Niklas Nielsen, and Vinod Kone.
>
>
> Bugs: MESOS-2325
> https://issues.apache.org/jira/browse/MESOS-2325
>
>
> Repository: mesos
>
>
> Description
> -------
>
> 1) Maintain an outstanding timers set to prevent scheduling multiple delay functions for the same timeout.
>
> Ended up moving tick() into the clock namespace to make it more clear that next() and tick() share this global set of outstanding timers.
>
> The original behavior used a single timer and adjusted the timeout. We didn't go back to this as benh wanted a generalized way to delay an arbitrary function. Therefore we handle the maintenance of this optimization in next() and tick() rather than pushing it down into the delay_function and management of ev_timers.
>
>
> Diffs
> -----
>
> 3rdparty/libprocess/src/clock.cpp fcc1eb0b921b3d9b9c033a17e5f6dcecf0a30f08
>
> Diff: https://reviews.apache.org/r/30808/diff/
>
>
> Testing
> -------
>
> make check
> A worst-case scenario test for the example listed in MESOS-2325. Constructed many timers in decrementing time (i.e. 500, 499,498, etc.). CPU utilization during this test went down greatly and is on par with code prior to the clock refactoring changes.
>
>
> Thanks,
>
> Joris Van Remoortere
>
>
Re: Review Request 30808: Fix high CPU bug. MESOS-2325.
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30808/#review72326
-----------------------------------------------------------
Patch looks great!
Reviews applied: [30808]
All tests passed.
- Mesos ReviewBot
On Feb. 12, 2015, 11:14 p.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30808/
> -----------------------------------------------------------
>
> (Updated Feb. 12, 2015, 11:14 p.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, Jie Yu, Niklas Nielsen, and Vinod Kone.
>
>
> Bugs: MESOS-2325
> https://issues.apache.org/jira/browse/MESOS-2325
>
>
> Repository: mesos
>
>
> Description
> -------
>
> 1) Maintain an outstanding timers stack to prevent scheduling multiple delay functions for the same timeout.
>
> Ended up moving tick() into the clock namespace to make it more clear that scheduleNext() and tick() share this global stack of outstanding timeouts.
>
> The original behavior used a single timer and adjusted the timeout. We didn't go back to this as benh wanted a generalized way to delay an arbitrary function. Therefore we handle the maintenance of this optimization in scheduleNext() and tick() rather than pushing it down into the delay_function and management of ev_timers.
>
>
> Diffs
> -----
>
> 3rdparty/libprocess/src/clock.cpp b92d5de2631986506361e28c05cb6169e6632a70
>
> Diff: https://reviews.apache.org/r/30808/diff/
>
>
> Testing
> -------
>
> make check
> A worst-case scenario test for the example listed in MESOS-2325. Constructed many timers in decrementing time (i.e. 500, 499,498, etc.). CPU utilization during this test went down greatly and is on par with code prior to the clock refactoring changes.
> Ran this test with 25K timers, with 10 firing every millisecond. ~3% cpu usage on a macbook.
>
>
> Thanks,
>
> Joris Van Remoortere
>
>
Re: Review Request 30808: Fix high CPU bug. MESOS-2325.
Posted by Joris Van Remoortere <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30808/#review72803
-----------------------------------------------------------
3rdparty/libprocess/src/clock.cpp
<https://reviews.apache.org/r/30808/#comment118878>
Do this in clock::pause() once instead of multiple places. improves readability. Here and below.
- Joris Van Remoortere
On Feb. 12, 2015, 11:14 p.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30808/
> -----------------------------------------------------------
>
> (Updated Feb. 12, 2015, 11:14 p.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, Jie Yu, Niklas Nielsen, and Vinod Kone.
>
>
> Bugs: MESOS-2325
> https://issues.apache.org/jira/browse/MESOS-2325
>
>
> Repository: mesos
>
>
> Description
> -------
>
> 1) Maintain an outstanding timers stack to prevent scheduling multiple delay functions for the same timeout.
>
> Ended up moving tick() into the clock namespace to make it more clear that scheduleNext() and tick() share this global stack of outstanding timeouts.
>
> The original behavior used a single timer and adjusted the timeout. We didn't go back to this as benh wanted a generalized way to delay an arbitrary function. Therefore we handle the maintenance of this optimization in scheduleNext() and tick() rather than pushing it down into the delay_function and management of ev_timers.
>
>
> Diffs
> -----
>
> 3rdparty/libprocess/src/clock.cpp b92d5de2631986506361e28c05cb6169e6632a70
>
> Diff: https://reviews.apache.org/r/30808/diff/
>
>
> Testing
> -------
>
> make check
> A worst-case scenario test for the example listed in MESOS-2325. Constructed many timers in decrementing time (i.e. 500, 499,498, etc.). CPU utilization during this test went down greatly and is on par with code prior to the clock refactoring changes.
> Ran this test with 25K timers, with 10 firing every millisecond. ~3% cpu usage on a macbook.
>
>
> Thanks,
>
> Joris Van Remoortere
>
>
Re: Review Request 30808: Fix high CPU bug. MESOS-2325.
Posted by Joris Van Remoortere <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30808/
-----------------------------------------------------------
(Updated Feb. 12, 2015, 11:14 p.m.)
Review request for mesos, Benjamin Hindman, Ben Mahler, Jie Yu, Niklas Nielsen, and Vinod Kone.
Changes
-------
Re-organized so that we keep next() simple, and wrap it with scheduleNext().
Used a stack to represent the cached scheduled timeouts.
Invalidate the cache when we advance / update the clock so we force a recalculation of the duration of the timeouts.
Bugs: MESOS-2325
https://issues.apache.org/jira/browse/MESOS-2325
Repository: mesos
Description (updated)
-------
1) Maintain an outstanding timers stack to prevent scheduling multiple delay functions for the same timeout.
Ended up moving tick() into the clock namespace to make it more clear that scheduleNext() and tick() share this global stack of outstanding timeouts.
The original behavior used a single timer and adjusted the timeout. We didn't go back to this as benh wanted a generalized way to delay an arbitrary function. Therefore we handle the maintenance of this optimization in scheduleNext() and tick() rather than pushing it down into the delay_function and management of ev_timers.
Diffs (updated)
-----
3rdparty/libprocess/src/clock.cpp b92d5de2631986506361e28c05cb6169e6632a70
Diff: https://reviews.apache.org/r/30808/diff/
Testing (updated)
-------
make check
A worst-case scenario test for the example listed in MESOS-2325. Constructed many timers in decrementing time (i.e. 500, 499,498, etc.). CPU utilization during this test went down greatly and is on par with code prior to the clock refactoring changes.
Ran this test with 25K timers, with 10 firing every millisecond. ~3% cpu usage on a macbook.
Thanks,
Joris Van Remoortere
Re: Review Request 30808: Fix high CPU bug. MESOS-2325.
Posted by Joris Van Remoortere <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30808/#review71881
-----------------------------------------------------------
3rdparty/libprocess/src/clock.cpp
<https://reviews.apache.org/r/30808/#comment117777>
Re-organize this to keep next() simple. Introduce a schedule() to uses next and manipulates the global.
Return an Option<Time> rather than duration so there is no information loss.
- Joris Van Remoortere
On Feb. 10, 2015, 2:22 a.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30808/
> -----------------------------------------------------------
>
> (Updated Feb. 10, 2015, 2:22 a.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, Jie Yu, Niklas Nielsen, and Vinod Kone.
>
>
> Bugs: MESOS-2325
> https://issues.apache.org/jira/browse/MESOS-2325
>
>
> Repository: mesos
>
>
> Description
> -------
>
> 1) Maintain an outstanding timers set to prevent scheduling multiple delay functions for the same timeout.
>
> Ended up moving tick() into the clock namespace to make it more clear that next() and tick() share this global set of outstanding timers.
>
> The original behavior used a single timer and adjusted the timeout. We didn't go back to this as benh wanted a generalized way to delay an arbitrary function. Therefore we handle the maintenance of this optimization in next() and tick() rather than pushing it down into the delay_function and management of ev_timers.
>
>
> Diffs
> -----
>
> 3rdparty/libprocess/src/clock.cpp fcc1eb0b921b3d9b9c033a17e5f6dcecf0a30f08
>
> Diff: https://reviews.apache.org/r/30808/diff/
>
>
> Testing
> -------
>
> make check
> A worst-case scenario test for the example listed in MESOS-2325. Constructed many timers in decrementing time (i.e. 500, 499,498, etc.). CPU utilization during this test went down greatly and is on par with code prior to the clock refactoring changes.
>
>
> Thanks,
>
> Joris Van Remoortere
>
>
Re: Review Request 30808: Fix high CPU bug. MESOS-2325.
Posted by Joris Van Remoortere <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30808/
-----------------------------------------------------------
(Updated Feb. 10, 2015, 2:22 a.m.)
Review request for mesos, Benjamin Hindman, Ben Mahler, Jie Yu, Niklas Nielsen, and Vinod Kone.
Changes
-------
Removed the change of behavior with regard to paused clock. It seems benh might have changed this originally to fix an underlying bug.
Used multi-set and single push-pop accounting for outstanding timers.
Bugs: MESOS-2325
https://issues.apache.org/jira/browse/MESOS-2325
Repository: mesos
Description (updated)
-------
1) Maintain an outstanding timers set to prevent scheduling multiple delay functions for the same timeout.
Ended up moving tick() into the clock namespace to make it more clear that next() and tick() share this global set of outstanding timers.
The original behavior used a single timer and adjusted the timeout. We didn't go back to this as benh wanted a generalized way to delay an arbitrary function. Therefore we handle the maintenance of this optimization in next() and tick() rather than pushing it down into the delay_function and management of ev_timers.
Diffs (updated)
-----
3rdparty/libprocess/src/clock.cpp fcc1eb0b921b3d9b9c033a17e5f6dcecf0a30f08
Diff: https://reviews.apache.org/r/30808/diff/
Testing
-------
make check
A worst-case scenario test for the example listed in MESOS-2325. Constructed many timers in decrementing time (i.e. 500, 499,498, etc.). CPU utilization during this test went down greatly and is on par with code prior to the clock refactoring changes.
Thanks,
Joris Van Remoortere