You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Dominic Hamon <dh...@twopensource.com> on 2014/05/09 21:39:20 UTC
Review Request 21279: Added task gauges to Master.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/
-----------------------------------------------------------
Review request for mesos and Ben Mahler.
Bugs: MESOS-1332
https://issues.apache.org/jira/browse/MESOS-1332
Repository: mesos-git
Description
-------
see summary
Diffs
-----
src/master/master.hpp 0a350b0b402edb3ca648c91c920043f66c08fe0e
src/master/master.cpp d851a7291acce950ea9391ddfb8813a432aeda34
Diff: https://reviews.apache.org/r/21279/diff/
Testing
-------
make check
ran locally and manually verified.
Thanks,
Dominic Hamon
Re: Review Request 21279: Added task gauges to Master.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/#review42875
-----------------------------------------------------------
src/master/master.cpp
<https://reviews.apache.org/r/21279/#comment76782>
I would kill this.
When a slave re-registers with a failed over master, it doesn't currently send information about completed executors or completed tasks of a running framework. It only sends info about completed frameworks (and their completed executors and tasks).
So, updating the terminal task counts with the latter but not the former seems misleading.
Lets update the counters after we fix the slave to do the right thing.
- Vinod Kone
On May 13, 2014, 7:35 p.m., Dominic Hamon wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21279/
> -----------------------------------------------------------
>
> (Updated May 13, 2014, 7:35 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1332
> https://issues.apache.org/jira/browse/MESOS-1332
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> see summary
>
>
> Diffs
> -----
>
> src/master/master.hpp 923552e3c7f9942d6be7a0446b44e40f53c11433
> src/master/master.cpp 569995c2eaf758f5171c4e0d082d4e519a143025
>
> Diff: https://reviews.apache.org/r/21279/diff/
>
>
> Testing
> -------
>
> make check
>
> ran locally and manually verified.
>
>
> Thanks,
>
> Dominic Hamon
>
>
Re: Review Request 21279: Added task gauges to Master.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/#review42890
-----------------------------------------------------------
Ship it!
Ship It!
- Vinod Kone
On May 13, 2014, 9:31 p.m., Dominic Hamon wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21279/
> -----------------------------------------------------------
>
> (Updated May 13, 2014, 9:31 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1332
> https://issues.apache.org/jira/browse/MESOS-1332
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> see summary
>
>
> Diffs
> -----
>
> src/master/master.hpp 923552e3c7f9942d6be7a0446b44e40f53c11433
> src/master/master.cpp 569995c2eaf758f5171c4e0d082d4e519a143025
>
> Diff: https://reviews.apache.org/r/21279/diff/
>
>
> Testing
> -------
>
> make check
>
> ran locally and manually verified.
>
>
> Thanks,
>
> Dominic Hamon
>
>
Re: Review Request 21279: Added task gauges to Master.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/#review43020
-----------------------------------------------------------
Ship it!
Looks good, would like to see some consistency around post-increment vs. pre-increment for metrics and counters.
src/master/master.cpp
<https://reviews.apache.org/r/21279/#comment77003>
No period here
src/master/master.cpp
<https://reviews.apache.org/r/21279/#comment77006>
Now we're pre-incrementing 'count' here, post-incrementing 'count' below, pre-incrementing some 'metrics', post-incrementing some 'metrics' (in master and slave).
We should stick with one technique, and I think the preference in Mesos has been to post-increment for "readability" which is arguable but is the current state (avoiding the thought of optimization here as it's definitely premature and the compiler will often take care of it).
We're not consistent across the code by any means, and my personal preference is to only use post-increment when necessary, however the easiest change here would be to just post-increment in this change.
- Ben Mahler
On May 14, 2014, 8:50 p.m., Dominic Hamon wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21279/
> -----------------------------------------------------------
>
> (Updated May 14, 2014, 8:50 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1332
> https://issues.apache.org/jira/browse/MESOS-1332
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> see summary
>
>
> Diffs
> -----
>
> src/master/master.hpp 12111cf7b183438f540bb0aad17e50e3f367558a
> src/master/master.cpp 2f0e9027c5bea8066ac13996451d4905a422336e
> src/tests/master_tests.cpp dcda0c79a6693bc3c0b69b4c9b148f9608342738
>
> Diff: https://reviews.apache.org/r/21279/diff/
>
>
> Testing
> -------
>
> make check
>
> ran locally and manually verified.
>
>
> Thanks,
>
> Dominic Hamon
>
>
Re: Review Request 21279: Added task gauges to Master.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/#review43033
-----------------------------------------------------------
Ship it!
Thank you! Will make the minor adjustment below and commit this patch.
src/master/master.cpp
<https://reviews.apache.org/r/21279/#comment77028>
s/taskmap/TaskMap/ since this is a type
- Ben Mahler
On May 14, 2014, 9:49 p.m., Dominic Hamon wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21279/
> -----------------------------------------------------------
>
> (Updated May 14, 2014, 9:49 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1332
> https://issues.apache.org/jira/browse/MESOS-1332
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> see summary
>
>
> Diffs
> -----
>
> src/master/master.hpp baf2d40652742d3d79f81cb34d359ba31ab179d4
> src/master/master.cpp ebc00f61487fcccc61cfe9e9b74a815715b4ef92
> src/tests/master_tests.cpp ba1229df83d09969050e4190683e88ebd7c72ffc
>
> Diff: https://reviews.apache.org/r/21279/diff/
>
>
> Testing
> -------
>
> make check
>
> ran locally and manually verified.
>
>
> Thanks,
>
> Dominic Hamon
>
>
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/
-----------------------------------------------------------
(Updated May 14, 2014, 2:49 p.m.)
Review request for mesos and Ben Mahler.
Changes
-------
rebased
Bugs: MESOS-1332
https://issues.apache.org/jira/browse/MESOS-1332
Repository: mesos-git
Description
-------
see summary
Diffs (updated)
-----
src/master/master.hpp baf2d40652742d3d79f81cb34d359ba31ab179d4
src/master/master.cpp ebc00f61487fcccc61cfe9e9b74a815715b4ef92
src/tests/master_tests.cpp ba1229df83d09969050e4190683e88ebd7c72ffc
Diff: https://reviews.apache.org/r/21279/diff/
Testing
-------
make check
ran locally and manually verified.
Thanks,
Dominic Hamon
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/
-----------------------------------------------------------
(Updated May 14, 2014, 2:36 p.m.)
Review request for mesos and Ben Mahler.
Bugs: MESOS-1332
https://issues.apache.org/jira/browse/MESOS-1332
Repository: mesos-git
Description
-------
see summary
Diffs (updated)
-----
src/master/master.hpp 12111cf7b183438f540bb0aad17e50e3f367558a
src/master/master.cpp 2f0e9027c5bea8066ac13996451d4905a422336e
src/tests/master_tests.cpp dcda0c79a6693bc3c0b69b4c9b148f9608342738
Diff: https://reviews.apache.org/r/21279/diff/
Testing
-------
make check
ran locally and manually verified.
Thanks,
Dominic Hamon
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/
-----------------------------------------------------------
(Updated May 14, 2014, 1:50 p.m.)
Review request for mesos and Ben Mahler.
Changes
-------
switched to using slaves as source of truth for task state.
Bugs: MESOS-1332
https://issues.apache.org/jira/browse/MESOS-1332
Repository: mesos-git
Description
-------
see summary
Diffs (updated)
-----
src/master/master.hpp 12111cf7b183438f540bb0aad17e50e3f367558a
src/master/master.cpp 2f0e9027c5bea8066ac13996451d4905a422336e
src/tests/master_tests.cpp dcda0c79a6693bc3c0b69b4c9b148f9608342738
Diff: https://reviews.apache.org/r/21279/diff/
Testing
-------
make check
ran locally and manually verified.
Thanks,
Dominic Hamon
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
> On May 14, 2014, 1:05 p.m., Ben Mahler wrote:
> > Hey Dominic, what will these Gauge methods expose when:
> >
> > (1) A Master fails over.
> > (2) All the slaves re-register w/ tasks.
> > (3) The framework has not re-registered yet!
> > (4) We hit the endpoint, this will say 0 tasks in each state right?
> >
> > We may want to loop over the slaves instead here.
Yes, they'll show 0 until frameworks reregister. How long do we normally have slaves with tasks and no frameworks? My expectation is that the framework is the canonical source of truth for the tasks and their states; if it should be the slave structures instead then we should loop over the slaves.
> On May 14, 2014, 1:05 p.m., Ben Mahler wrote:
> > src/master/master.cpp, lines 3882-3920
> > <https://reviews.apache.org/r/21279/diff/6/?file=581336#file581336line3882>
> >
> > Shouldn't these be with the other gauge definitions, rather than below ~Metrics?
> >
> > How about s/taskCount/count/ and s/++count/count++/ to keep it consistent?
we really should default to pre-increment so that muscle memory doesn't cause us to create temporaries unnecessarily when not using base types.
This was consistent with how I wrote the previous patch ;)
- Dominic
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/#review43009
-----------------------------------------------------------
On May 14, 2014, 12:25 p.m., Dominic Hamon wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21279/
> -----------------------------------------------------------
>
> (Updated May 14, 2014, 12:25 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1332
> https://issues.apache.org/jira/browse/MESOS-1332
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> see summary
>
>
> Diffs
> -----
>
> src/master/master.hpp 12111cf7b183438f540bb0aad17e50e3f367558a
> src/master/master.cpp 2f0e9027c5bea8066ac13996451d4905a422336e
> src/tests/master_tests.cpp dcda0c79a6693bc3c0b69b4c9b148f9608342738
>
> Diff: https://reviews.apache.org/r/21279/diff/
>
>
> Testing
> -------
>
> make check
>
> ran locally and manually verified.
>
>
> Thanks,
>
> Dominic Hamon
>
>
Re: Review Request 21279: Added task gauges to Master.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/#review43009
-----------------------------------------------------------
Hey Dominic, what will these Gauge methods expose when:
(1) A Master fails over.
(2) All the slaves re-register w/ tasks.
(3) The framework has not re-registered yet!
(4) We hit the endpoint, this will say 0 tasks in each state right?
We may want to loop over the slaves instead here.
src/master/master.cpp
<https://reviews.apache.org/r/21279/#comment76996>
Shouldn't these be with the other gauge definitions, rather than below ~Metrics?
src/master/master.cpp
<https://reviews.apache.org/r/21279/#comment76997>
Shouldn't these be with the other gauge definitions, rather than below ~Metrics?
How about s/taskCount/count/ and s/++count/count++/ to keep it consistent?
src/master/master.cpp
<https://reviews.apache.org/r/21279/#comment76999>
This case is possible given the current state of the code, when we remove a framework, how about adjusting this message to say:
LOG(WARNING) << "Removing task " << task->task_id() << " of framework " << task->framework_id() << " in non-terminal state " << task->state();
- Ben Mahler
On May 14, 2014, 7:25 p.m., Dominic Hamon wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21279/
> -----------------------------------------------------------
>
> (Updated May 14, 2014, 7:25 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1332
> https://issues.apache.org/jira/browse/MESOS-1332
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> see summary
>
>
> Diffs
> -----
>
> src/master/master.hpp 12111cf7b183438f540bb0aad17e50e3f367558a
> src/master/master.cpp 2f0e9027c5bea8066ac13996451d4905a422336e
> src/tests/master_tests.cpp dcda0c79a6693bc3c0b69b4c9b148f9608342738
>
> Diff: https://reviews.apache.org/r/21279/diff/
>
>
> Testing
> -------
>
> make check
>
> ran locally and manually verified.
>
>
> Thanks,
>
> Dominic Hamon
>
>
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/
-----------------------------------------------------------
(Updated May 14, 2014, 12:25 p.m.)
Review request for mesos and Ben Mahler.
Changes
-------
rebased to master
Bugs: MESOS-1332
https://issues.apache.org/jira/browse/MESOS-1332
Repository: mesos-git
Description
-------
see summary
Diffs (updated)
-----
src/master/master.hpp 12111cf7b183438f540bb0aad17e50e3f367558a
src/master/master.cpp 2f0e9027c5bea8066ac13996451d4905a422336e
src/tests/master_tests.cpp dcda0c79a6693bc3c0b69b4c9b148f9608342738
Diff: https://reviews.apache.org/r/21279/diff/
Testing
-------
make check
ran locally and manually verified.
Thanks,
Dominic Hamon
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/
-----------------------------------------------------------
(Updated May 13, 2014, 10:03 p.m.)
Review request for mesos and Ben Mahler.
Changes
-------
moved task counters up.
added to MetricsInStatsEndpoint test.
Bugs: MESOS-1332
https://issues.apache.org/jira/browse/MESOS-1332
Repository: mesos-git
Description
-------
see summary
Diffs (updated)
-----
src/master/master.hpp 4f9ae36c822a16ea3baadf6b9fa3616d030d19f2
src/master/master.cpp d5453673a839326d00a3d45940bd4562c526cff2
src/tests/master_tests.cpp 7aa678afc94869c8243485bd0604532dec43a1e2
Diff: https://reviews.apache.org/r/21279/diff/
Testing
-------
make check
ran locally and manually verified.
Thanks,
Dominic Hamon
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/
-----------------------------------------------------------
(Updated May 13, 2014, 9:54 p.m.)
Review request for mesos and Ben Mahler.
Changes
-------
rebased to 19504
Bugs: MESOS-1332
https://issues.apache.org/jira/browse/MESOS-1332
Repository: mesos-git
Description
-------
see summary
Diffs (updated)
-----
src/master/master.hpp 4f9ae36c822a16ea3baadf6b9fa3616d030d19f2
src/master/master.cpp d5453673a839326d00a3d45940bd4562c526cff2
Diff: https://reviews.apache.org/r/21279/diff/
Testing
-------
make check
ran locally and manually verified.
Thanks,
Dominic Hamon
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
> On May 13, 2014, 6:47 p.m., Ben Mahler wrote:
> > Looks like this needs a rebase?
> >
> > $ ./support/apply-review.sh 21279
> > error: patch failed: src/master/master.cpp:3747
> > error: src/master/master.cpp: patch does not apply
> > Failed to apply patch
it needs 19504 .. did the dependency get lost?
- Dominic
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/#review42930
-----------------------------------------------------------
On May 13, 2014, 10:03 p.m., Dominic Hamon wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21279/
> -----------------------------------------------------------
>
> (Updated May 13, 2014, 10:03 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1332
> https://issues.apache.org/jira/browse/MESOS-1332
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> see summary
>
>
> Diffs
> -----
>
> src/master/master.hpp 4f9ae36c822a16ea3baadf6b9fa3616d030d19f2
> src/master/master.cpp d5453673a839326d00a3d45940bd4562c526cff2
> src/tests/master_tests.cpp 7aa678afc94869c8243485bd0604532dec43a1e2
>
> Diff: https://reviews.apache.org/r/21279/diff/
>
>
> Testing
> -------
>
> make check
>
> ran locally and manually verified.
>
>
> Thanks,
>
> Dominic Hamon
>
>
Re: Review Request 21279: Added task gauges to Master.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/#review42930
-----------------------------------------------------------
Ship it!
Looks like this needs a rebase?
$ ./support/apply-review.sh 21279
error: patch failed: src/master/master.cpp:3747
error: src/master/master.cpp: patch does not apply
Failed to apply patch
src/master/master.hpp
<https://reviews.apache.org/r/21279/#comment76857>
How about we move this above the message counters, since this is more important? Ditto for the constructor, initializer, and destructor.
src/master/master.cpp
<https://reviews.apache.org/r/21279/#comment76858>
How about s/tasks/count/?
- Ben Mahler
On May 13, 2014, 9:31 p.m., Dominic Hamon wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21279/
> -----------------------------------------------------------
>
> (Updated May 13, 2014, 9:31 p.m.)
>
>
> Review request for mesos and Ben Mahler.
>
>
> Bugs: MESOS-1332
> https://issues.apache.org/jira/browse/MESOS-1332
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> see summary
>
>
> Diffs
> -----
>
> src/master/master.hpp 923552e3c7f9942d6be7a0446b44e40f53c11433
> src/master/master.cpp 569995c2eaf758f5171c4e0d082d4e519a143025
>
> Diff: https://reviews.apache.org/r/21279/diff/
>
>
> Testing
> -------
>
> make check
>
> ran locally and manually verified.
>
>
> Thanks,
>
> Dominic Hamon
>
>
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/
-----------------------------------------------------------
(Updated May 13, 2014, 2:31 p.m.)
Review request for mesos and Ben Mahler.
Bugs: MESOS-1332
https://issues.apache.org/jira/browse/MESOS-1332
Repository: mesos-git
Description
-------
see summary
Diffs (updated)
-----
src/master/master.hpp 923552e3c7f9942d6be7a0446b44e40f53c11433
src/master/master.cpp 569995c2eaf758f5171c4e0d082d4e519a143025
Diff: https://reviews.apache.org/r/21279/diff/
Testing
-------
make check
ran locally and manually verified.
Thanks,
Dominic Hamon
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/
-----------------------------------------------------------
(Updated May 13, 2014, 12:35 p.m.)
Review request for mesos and Ben Mahler.
Changes
-------
switched some gauges to counters for simplicity.
Bugs: MESOS-1332
https://issues.apache.org/jira/browse/MESOS-1332
Repository: mesos-git
Description
-------
see summary
Diffs (updated)
-----
src/master/master.hpp 923552e3c7f9942d6be7a0446b44e40f53c11433
src/master/master.cpp 569995c2eaf758f5171c4e0d082d4e519a143025
Diff: https://reviews.apache.org/r/21279/diff/
Testing
-------
make check
ran locally and manually verified.
Thanks,
Dominic Hamon
Re: Review Request 21279: Added task gauges to Master.
Posted by Dominic Hamon <dh...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21279/
-----------------------------------------------------------
(Updated May 12, 2014, 10:35 a.m.)
Review request for mesos and Ben Mahler.
Changes
-------
rebased
Bugs: MESOS-1332
https://issues.apache.org/jira/browse/MESOS-1332
Repository: mesos-git
Description
-------
see summary
Diffs (updated)
-----
src/master/master.hpp 0a350b0b402edb3ca648c91c920043f66c08fe0e
src/master/master.cpp d851a7291acce950ea9391ddfb8813a432aeda34
Diff: https://reviews.apache.org/r/21279/diff/
Testing
-------
make check
ran locally and manually verified.
Thanks,
Dominic Hamon