You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Niklas Nielsen <ni...@qni.dk> on 2014/02/14 19:58:08 UTC
Review Request 18138: Fixed comparator in tasks endpoint.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/
-----------------------------------------------------------
Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
Bugs: MESOS-979
https://issues.apache.org/jira/browse/MESOS-979
Repository: mesos-git
Description
-------
Related to MESOS-979, the tasks.json endpoint would sporadically cause the
master process to SEGFAULT. The code failed during the subsequent sort
(after gathering running and completed tasks) deferencing garbage pointer
values. The issue turns out to be an issue with std::sort and strict weak-
ordering. Descending order (which was default) was implemented as !ascending()
which is not a strict greather-than equal. This patch expands the descending
implementation and reenables endpoint.
Diffs
-----
src/master/http.cpp 966eed6
src/master/master.cpp a4e1b1f
Diff: https://reviews.apache.org/r/18138/diff/
Testing
-------
make check and functional testing; can't trigger earlier repro steps for segfault.
Thanks,
Niklas Nielsen
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Niklas Nielsen <ni...@qni.dk>.
> On Feb. 14, 2014, 11:20 a.m., Adam B wrote:
> > src/master/http.cpp, lines 536-542
> > <https://reviews.apache.org/r/18138/diff/1/?file=485754#file485754line536>
> >
> > What happens if both lhs and rhs are empty? Perhaps descending should be checking rhs first? Or maybe they should both check the 'return false' case first. empty,empty is neither ascending nor descending.
>
> Niklas Nielsen wrote:
> Should be taken care of when generating the list of pointers i.e. CHECK_NOTNULL(task) :)
Whoops - please ignore that comment; you are right. I'll get that fixed.
- Niklas
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/#review34536
-----------------------------------------------------------
On Feb. 14, 2014, 10:58 a.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18138/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2014, 10:58 a.m.)
>
>
> Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
>
>
> Bugs: MESOS-979
> https://issues.apache.org/jira/browse/MESOS-979
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Related to MESOS-979, the tasks.json endpoint would sporadically cause the
> master process to SEGFAULT. The code failed during the subsequent sort
> (after gathering running and completed tasks) deferencing garbage pointer
> values. The issue turns out to be an issue with std::sort and strict weak-
> ordering. Descending order (which was default) was implemented as !ascending()
> which is not a strict greather-than equal. This patch expands the descending
> implementation and reenables endpoint.
>
>
> Diffs
> -----
>
> src/master/http.cpp 966eed6
> src/master/master.cpp a4e1b1f
>
> Diff: https://reviews.apache.org/r/18138/diff/
>
>
> Testing
> -------
>
> make check and functional testing; can't trigger earlier repro steps for segfault.
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Niklas Nielsen <ni...@qni.dk>.
> On Feb. 14, 2014, 11:20 a.m., Adam B wrote:
> > src/master/http.cpp, lines 536-542
> > <https://reviews.apache.org/r/18138/diff/1/?file=485754#file485754line536>
> >
> > What happens if both lhs and rhs are empty? Perhaps descending should be checking rhs first? Or maybe they should both check the 'return false' case first. empty,empty is neither ascending nor descending.
Should be taken care of when generating the list of pointers i.e. CHECK_NOTNULL(task) :)
- Niklas
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/#review34536
-----------------------------------------------------------
On Feb. 14, 2014, 10:58 a.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18138/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2014, 10:58 a.m.)
>
>
> Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
>
>
> Bugs: MESOS-979
> https://issues.apache.org/jira/browse/MESOS-979
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Related to MESOS-979, the tasks.json endpoint would sporadically cause the
> master process to SEGFAULT. The code failed during the subsequent sort
> (after gathering running and completed tasks) deferencing garbage pointer
> values. The issue turns out to be an issue with std::sort and strict weak-
> ordering. Descending order (which was default) was implemented as !ascending()
> which is not a strict greather-than equal. This patch expands the descending
> implementation and reenables endpoint.
>
>
> Diffs
> -----
>
> src/master/http.cpp 966eed6
> src/master/master.cpp a4e1b1f
>
> Diff: https://reviews.apache.org/r/18138/diff/
>
>
> Testing
> -------
>
> make check and functional testing; can't trigger earlier repro steps for segfault.
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Adam B <ad...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/#review34536
-----------------------------------------------------------
Ship it!
Pretty straightforward, just a small concern about comparing two empty tasks.
src/master/http.cpp
<https://reviews.apache.org/r/18138/#comment64641>
What happens if both lhs and rhs are empty? Perhaps descending should be checking rhs first? Or maybe they should both check the 'return false' case first. empty,empty is neither ascending nor descending.
- Adam B
On Feb. 14, 2014, 10:58 a.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18138/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2014, 10:58 a.m.)
>
>
> Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
>
>
> Bugs: MESOS-979
> https://issues.apache.org/jira/browse/MESOS-979
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Related to MESOS-979, the tasks.json endpoint would sporadically cause the
> master process to SEGFAULT. The code failed during the subsequent sort
> (after gathering running and completed tasks) deferencing garbage pointer
> values. The issue turns out to be an issue with std::sort and strict weak-
> ordering. Descending order (which was default) was implemented as !ascending()
> which is not a strict greather-than equal. This patch expands the descending
> implementation and reenables endpoint.
>
>
> Diffs
> -----
>
> src/master/http.cpp 966eed6
> src/master/master.cpp a4e1b1f
>
> Diff: https://reviews.apache.org/r/18138/diff/
>
>
> Testing
> -------
>
> make check and functional testing; can't trigger earlier repro steps for segfault.
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/#review34534
-----------------------------------------------------------
Ship it!
Ship It!
- Ben Mahler
On Feb. 14, 2014, 6:58 p.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18138/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2014, 6:58 p.m.)
>
>
> Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
>
>
> Bugs: MESOS-979
> https://issues.apache.org/jira/browse/MESOS-979
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Related to MESOS-979, the tasks.json endpoint would sporadically cause the
> master process to SEGFAULT. The code failed during the subsequent sort
> (after gathering running and completed tasks) deferencing garbage pointer
> values. The issue turns out to be an issue with std::sort and strict weak-
> ordering. Descending order (which was default) was implemented as !ascending()
> which is not a strict greather-than equal. This patch expands the descending
> implementation and reenables endpoint.
>
>
> Diffs
> -----
>
> src/master/http.cpp 966eed6
> src/master/master.cpp a4e1b1f
>
> Diff: https://reviews.apache.org/r/18138/diff/
>
>
> Testing
> -------
>
> make check and functional testing; can't trigger earlier repro steps for segfault.
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Niklas Nielsen <ni...@qni.dk>.
> On Feb. 18, 2014, 11:36 a.m., Vinod Kone wrote:
> > Niklas, can you commit this?
> >
> > Also, I don't think I understand what the issue with sort and weak ordering is. Mind expanding on it a bit or pointing me to a reference?
You bet. This is the best explanation I could find on the issue: http://stackoverflow.com/questions/16535882/stdsort-with-equal-elements-gives-segmentation-fault
The strict weak ordering requirement is mentioned in http://www.cplusplus.com/reference/algorithm/sort/ and explained here: http://www.sgi.com/tech/stl/StrictWeakOrdering.html
- Niklas
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/#review34759
-----------------------------------------------------------
On Feb. 18, 2014, 8:43 a.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18138/
> -----------------------------------------------------------
>
> (Updated Feb. 18, 2014, 8:43 a.m.)
>
>
> Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
>
>
> Bugs: MESOS-979
> https://issues.apache.org/jira/browse/MESOS-979
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Related to MESOS-979, the tasks.json endpoint would sporadically cause the
> master process to SEGFAULT. The code failed during the subsequent sort
> (after gathering running and completed tasks) deferencing garbage pointer
> values. The issue turns out to be an issue with std::sort and strict weak-
> ordering. Descending order (which was default) was implemented as !ascending()
> which is not a strict greather-than equal. This patch expands the descending
> implementation and reenables endpoint.
>
>
> Diffs
> -----
>
> src/master/http.cpp 6aeb257
> src/master/master.cpp f4f5e04
>
> Diff: https://reviews.apache.org/r/18138/diff/
>
>
> Testing
> -------
>
> make check and functional testing; can't trigger earlier repro steps for segfault.
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Vinod Kone <vi...@gmail.com>.
> On Feb. 18, 2014, 7:36 p.m., Vinod Kone wrote:
> > Niklas, can you commit this?
> >
> > Also, I don't think I understand what the issue with sort and weak ordering is. Mind expanding on it a bit or pointing me to a reference?
>
> Niklas Nielsen wrote:
> You bet. This is the best explanation I could find on the issue: http://stackoverflow.com/questions/16535882/stdsort-with-equal-elements-gives-segmentation-fault
> The strict weak ordering requirement is mentioned in http://www.cplusplus.com/reference/algorithm/sort/ and explained here: http://www.sgi.com/tech/stl/StrictWeakOrdering.html
thank you!
- Vinod
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/#review34759
-----------------------------------------------------------
On Feb. 18, 2014, 4:43 p.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18138/
> -----------------------------------------------------------
>
> (Updated Feb. 18, 2014, 4:43 p.m.)
>
>
> Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
>
>
> Bugs: MESOS-979
> https://issues.apache.org/jira/browse/MESOS-979
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Related to MESOS-979, the tasks.json endpoint would sporadically cause the
> master process to SEGFAULT. The code failed during the subsequent sort
> (after gathering running and completed tasks) deferencing garbage pointer
> values. The issue turns out to be an issue with std::sort and strict weak-
> ordering. Descending order (which was default) was implemented as !ascending()
> which is not a strict greather-than equal. This patch expands the descending
> implementation and reenables endpoint.
>
>
> Diffs
> -----
>
> src/master/http.cpp 6aeb257
> src/master/master.cpp f4f5e04
>
> Diff: https://reviews.apache.org/r/18138/diff/
>
>
> Testing
> -------
>
> make check and functional testing; can't trigger earlier repro steps for segfault.
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/#review34759
-----------------------------------------------------------
Ship it!
Niklas, can you commit this?
Also, I don't think I understand what the issue with sort and weak ordering is. Mind expanding on it a bit or pointing me to a reference?
- Vinod Kone
On Feb. 18, 2014, 4:43 p.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18138/
> -----------------------------------------------------------
>
> (Updated Feb. 18, 2014, 4:43 p.m.)
>
>
> Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
>
>
> Bugs: MESOS-979
> https://issues.apache.org/jira/browse/MESOS-979
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Related to MESOS-979, the tasks.json endpoint would sporadically cause the
> master process to SEGFAULT. The code failed during the subsequent sort
> (after gathering running and completed tasks) deferencing garbage pointer
> values. The issue turns out to be an issue with std::sort and strict weak-
> ordering. Descending order (which was default) was implemented as !ascending()
> which is not a strict greather-than equal. This patch expands the descending
> implementation and reenables endpoint.
>
>
> Diffs
> -----
>
> src/master/http.cpp 6aeb257
> src/master/master.cpp f4f5e04
>
> Diff: https://reviews.apache.org/r/18138/diff/
>
>
> Testing
> -------
>
> make check and functional testing; can't trigger earlier repro steps for segfault.
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Adam B <ad...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/#review34908
-----------------------------------------------------------
Ship it!
This new version of the fix does address my concern about both lhs/rhs being empty, but the logic can be simplified a bit.
src/master/http.cpp
<https://reviews.apache.org/r/18138/#comment65292>
I like how clear the code is this way, making the case for both-empty very explicit.
But I will point out that all this boolean logic can be simplified. If you check the false case first in both methods, you'll achieve the same thing (and the compiler may be smart enough to clean this up for you).
ascending:
if (rhsSize==0) return false;
if (lhsSize==0) return true; // would have already returned false if both were empty.
descending:
if (lhsSize==0) return false;
if (rhsSize==0) return true; // would have already returned false if both were empty.
- Adam B
On Feb. 19, 2014, 11:30 a.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18138/
> -----------------------------------------------------------
>
> (Updated Feb. 19, 2014, 11:30 a.m.)
>
>
> Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
>
>
> Bugs: MESOS-979
> https://issues.apache.org/jira/browse/MESOS-979
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Related to MESOS-979, the tasks.json endpoint would sporadically cause the
> master process to SEGFAULT. The code failed during the subsequent sort
> (after gathering running and completed tasks) deferencing garbage pointer
> values. The issue turns out to be an issue with std::sort and strict weak-
> ordering. Descending order (which was default) was implemented as !ascending()
> which is not a strict greather-than equal. This patch expands the descending
> implementation and reenables endpoint.
>
>
> Diffs
> -----
>
> src/master/http.cpp 6aeb257
> src/master/master.cpp f4f5e04
>
> Diff: https://reviews.apache.org/r/18138/diff/
>
>
> Testing
> -------
>
> make check and functional testing; can't trigger earlier repro steps for segfault.
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/#review34904
-----------------------------------------------------------
Ship it!
Ship It!
- Vinod Kone
On Feb. 19, 2014, 7:30 p.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18138/
> -----------------------------------------------------------
>
> (Updated Feb. 19, 2014, 7:30 p.m.)
>
>
> Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
>
>
> Bugs: MESOS-979
> https://issues.apache.org/jira/browse/MESOS-979
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Related to MESOS-979, the tasks.json endpoint would sporadically cause the
> master process to SEGFAULT. The code failed during the subsequent sort
> (after gathering running and completed tasks) deferencing garbage pointer
> values. The issue turns out to be an issue with std::sort and strict weak-
> ordering. Descending order (which was default) was implemented as !ascending()
> which is not a strict greather-than equal. This patch expands the descending
> implementation and reenables endpoint.
>
>
> Diffs
> -----
>
> src/master/http.cpp 6aeb257
> src/master/master.cpp f4f5e04
>
> Diff: https://reviews.apache.org/r/18138/diff/
>
>
> Testing
> -------
>
> make check and functional testing; can't trigger earlier repro steps for segfault.
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Niklas Nielsen <ni...@qni.dk>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/
-----------------------------------------------------------
(Updated Feb. 19, 2014, 11:30 a.m.)
Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
Changes
-------
I managed to trigger the bug again and turned out to be related to Adam's concern about the semantics when both lhs and rhs is empty.
This patch should handle issue.
Bugs: MESOS-979
https://issues.apache.org/jira/browse/MESOS-979
Repository: mesos-git
Description
-------
Related to MESOS-979, the tasks.json endpoint would sporadically cause the
master process to SEGFAULT. The code failed during the subsequent sort
(after gathering running and completed tasks) deferencing garbage pointer
values. The issue turns out to be an issue with std::sort and strict weak-
ordering. Descending order (which was default) was implemented as !ascending()
which is not a strict greather-than equal. This patch expands the descending
implementation and reenables endpoint.
Diffs (updated)
-----
src/master/http.cpp 6aeb257
src/master/master.cpp f4f5e04
Diff: https://reviews.apache.org/r/18138/diff/
Testing
-------
make check and functional testing; can't trigger earlier repro steps for segfault.
Thanks,
Niklas Nielsen
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/#review34717
-----------------------------------------------------------
Ship it!
Ship It!
- Dominic Hamon
On Feb. 18, 2014, 8:43 a.m., Niklas Nielsen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18138/
> -----------------------------------------------------------
>
> (Updated Feb. 18, 2014, 8:43 a.m.)
>
>
> Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
>
>
> Bugs: MESOS-979
> https://issues.apache.org/jira/browse/MESOS-979
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Related to MESOS-979, the tasks.json endpoint would sporadically cause the
> master process to SEGFAULT. The code failed during the subsequent sort
> (after gathering running and completed tasks) deferencing garbage pointer
> values. The issue turns out to be an issue with std::sort and strict weak-
> ordering. Descending order (which was default) was implemented as !ascending()
> which is not a strict greather-than equal. This patch expands the descending
> implementation and reenables endpoint.
>
>
> Diffs
> -----
>
> src/master/http.cpp 6aeb257
> src/master/master.cpp f4f5e04
>
> Diff: https://reviews.apache.org/r/18138/diff/
>
>
> Testing
> -------
>
> make check and functional testing; can't trigger earlier repro steps for segfault.
>
>
> Thanks,
>
> Niklas Nielsen
>
>
Re: Review Request 18138: Fixed comparator in tasks endpoint.
Posted by Niklas Nielsen <ni...@qni.dk>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18138/
-----------------------------------------------------------
(Updated Feb. 18, 2014, 8:43 a.m.)
Review request for mesos, Adam B, Ben Mahler, and Vinod Kone.
Changes
-------
Addressed Adam's comment.
Bugs: MESOS-979
https://issues.apache.org/jira/browse/MESOS-979
Repository: mesos-git
Description
-------
Related to MESOS-979, the tasks.json endpoint would sporadically cause the
master process to SEGFAULT. The code failed during the subsequent sort
(after gathering running and completed tasks) deferencing garbage pointer
values. The issue turns out to be an issue with std::sort and strict weak-
ordering. Descending order (which was default) was implemented as !ascending()
which is not a strict greather-than equal. This patch expands the descending
implementation and reenables endpoint.
Diffs (updated)
-----
src/master/http.cpp 6aeb257
src/master/master.cpp f4f5e04
Diff: https://reviews.apache.org/r/18138/diff/
Testing
-------
make check and functional testing; can't trigger earlier repro steps for segfault.
Thanks,
Niklas Nielsen