You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Anand Mazumdar <ma...@gmail.com> on 2015/12/01 04:38:59 UTC

Review Request 40811: Fixed flakiness in tests using the Scheduler library

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40811/
-----------------------------------------------------------

Review request for mesos and Ben Mahler.


Bugs: MESOS-4029
    https://issues.apache.org/jira/browse/MESOS-4029


Repository: mesos


Description
-------

Previously, the `event` queue object was defined after the `callbacks` object. However, the `callbacks` object held on a reference to the queue object. This led to a crash in some scenarios when we received another event from the master after the `event` object was destroyed. (Note that objects on the stack are destroyed in the reverse order of definition)

The short term fix is to move the `event` queue object above the `callbacks` object. This ensures that it is destroyed after the `callbacks` object. The long term fix would be worked on via `MESOS-3339` that moves the tests to a callback interface model similar to what we have for the driver tests.


Diffs
-----

  src/tests/master_maintenance_tests.cpp ade05f932020013ced19c1573be756a029396fac 
  src/tests/scheduler_tests.cpp 67432109c7df6be0aa76e94a03bd5b2e9c96d14e 

Diff: https://reviews.apache.org/r/40811/diff/


Testing
-------

make check + Reproduced this by introducing a sleep in the `Mesos` destructor + did not filter out the `HEARTBEAT` event.


Thanks,

Anand Mazumdar


Re: Review Request 40811: Fixed flakiness in tests using the Scheduler library

Posted by Mesos ReviewBot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40811/#review108549
-----------------------------------------------------------


Patch looks great!

Reviews applied: [40811]

Passed command: export OS=ubuntu:14.04;export CONFIGURATION="--verbose";export COMPILER=gcc; ./support/docker_build.sh

- Mesos ReviewBot


On Dec. 1, 2015, 3:38 a.m., Anand Mazumdar wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40811/
> -----------------------------------------------------------
> 
> (Updated Dec. 1, 2015, 3:38 a.m.)
> 
> 
> Review request for mesos and Ben Mahler.
> 
> 
> Bugs: MESOS-4029
>     https://issues.apache.org/jira/browse/MESOS-4029
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Previously, the `event` queue object was defined after the `callbacks` object. However, the `callbacks` object held on a reference to the queue object. This led to a crash in some scenarios when we received another event from the master after the `event` object was destroyed. (Note that objects on the stack are destroyed in the reverse order of definition)
> 
> The short term fix is to move the `event` queue object above the `callbacks` object. This ensures that it is destroyed after the `callbacks` object. The long term fix would be worked on via `MESOS-3339` that moves the tests to a callback interface model similar to what we have for the driver tests.
> 
> 
> Diffs
> -----
> 
>   src/tests/master_maintenance_tests.cpp ade05f932020013ced19c1573be756a029396fac 
>   src/tests/scheduler_tests.cpp 67432109c7df6be0aa76e94a03bd5b2e9c96d14e 
> 
> Diff: https://reviews.apache.org/r/40811/diff/
> 
> 
> Testing
> -------
> 
> make check + Reproduced this by introducing a sleep in the `Mesos` destructor + did not filter out the `HEARTBEAT` event.
> 
> 
> Thanks,
> 
> Anand Mazumdar
> 
>


Re: Review Request 40811: Fixed flakiness in tests using the Scheduler library

Posted by Alexander Rojas <al...@mesosphere.io>.

> On Dec. 1, 2015, 2:56 p.m., Alexander Rojas wrote:
> > Ship It!

I have to revert this ship-it, after playing with it longer, I manage to reproduce at least two types of crashes after applying this patch.


- Alexander


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40811/#review108495
-----------------------------------------------------------


On Dec. 1, 2015, 4:38 a.m., Anand Mazumdar wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40811/
> -----------------------------------------------------------
> 
> (Updated Dec. 1, 2015, 4:38 a.m.)
> 
> 
> Review request for mesos and Ben Mahler.
> 
> 
> Bugs: MESOS-4029
>     https://issues.apache.org/jira/browse/MESOS-4029
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Previously, the `event` queue object was defined after the `callbacks` object. However, the `callbacks` object held on a reference to the queue object. This led to a crash in some scenarios when we received another event from the master after the `event` object was destroyed. (Note that objects on the stack are destroyed in the reverse order of definition)
> 
> The short term fix is to move the `event` queue object above the `callbacks` object. This ensures that it is destroyed after the `callbacks` object. The long term fix would be worked on via `MESOS-3339` that moves the tests to a callback interface model similar to what we have for the driver tests.
> 
> 
> Diffs
> -----
> 
>   src/tests/master_maintenance_tests.cpp ade05f932020013ced19c1573be756a029396fac 
>   src/tests/scheduler_tests.cpp 67432109c7df6be0aa76e94a03bd5b2e9c96d14e 
> 
> Diff: https://reviews.apache.org/r/40811/diff/
> 
> 
> Testing
> -------
> 
> make check + Reproduced this by introducing a sleep in the `Mesos` destructor + did not filter out the `HEARTBEAT` event.
> 
> 
> Thanks,
> 
> Anand Mazumdar
> 
>


Re: Review Request 40811: Fixed flakiness in tests using the Scheduler library

Posted by Alexander Rojas <al...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40811/#review108495
-----------------------------------------------------------

Ship it!


Ship It!

- Alexander Rojas


On Dec. 1, 2015, 4:38 a.m., Anand Mazumdar wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40811/
> -----------------------------------------------------------
> 
> (Updated Dec. 1, 2015, 4:38 a.m.)
> 
> 
> Review request for mesos and Ben Mahler.
> 
> 
> Bugs: MESOS-4029
>     https://issues.apache.org/jira/browse/MESOS-4029
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Previously, the `event` queue object was defined after the `callbacks` object. However, the `callbacks` object held on a reference to the queue object. This led to a crash in some scenarios when we received another event from the master after the `event` object was destroyed. (Note that objects on the stack are destroyed in the reverse order of definition)
> 
> The short term fix is to move the `event` queue object above the `callbacks` object. This ensures that it is destroyed after the `callbacks` object. The long term fix would be worked on via `MESOS-3339` that moves the tests to a callback interface model similar to what we have for the driver tests.
> 
> 
> Diffs
> -----
> 
>   src/tests/master_maintenance_tests.cpp ade05f932020013ced19c1573be756a029396fac 
>   src/tests/scheduler_tests.cpp 67432109c7df6be0aa76e94a03bd5b2e9c96d14e 
> 
> Diff: https://reviews.apache.org/r/40811/diff/
> 
> 
> Testing
> -------
> 
> make check + Reproduced this by introducing a sleep in the `Mesos` destructor + did not filter out the `HEARTBEAT` event.
> 
> 
> Thanks,
> 
> Anand Mazumdar
> 
>