You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Andrei Budnik <ab...@mesosphere.com> on 2019/06/19 14:49:24 UTC
Review Request 70889: Wrapped isolators in `IsolatorTracker`.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70889/
-----------------------------------------------------------
Review request for mesos, Gilbert Song and Qian Zhang.
Bugs: MESOS-9841
https://issues.apache.org/jira/browse/MESOS-9841
Repository: mesos
Description
-------
This patch wraps every isolator in instance of `IsolatorTracker` class.
If an isolator gets stuck in some operation, `pendingFutures` will
return the isolator name, method name along with its arguments such as
`containerId`, `pid`, etc.
Diffs
-----
src/slave/containerizer/mesos/containerizer.cpp 043244841a73fa3f5f7119bc38f6d3a04be8990b
Diff: https://reviews.apache.org/r/70889/diff/1/
Testing
-------
Thanks,
Andrei Budnik
Re: Review Request 70889: Wrapped isolators in `IsolatorTracker`.
Posted by Gilbert Song <so...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70889/#review216681
-----------------------------------------------------------
Fix it, then Ship it!
src/slave/containerizer/mesos/containerizer.cpp
Lines 550 (patched)
<https://reviews.apache.org/r/70889/#comment303884>
could we be explicit here?
if (futuretracker != nullptr)
src/slave/containerizer/mesos/containerizer.cpp
Lines 570 (patched)
<https://reviews.apache.org/r/70889/#comment303885>
ditto
- Gilbert Song
On June 19, 2019, 7:49 a.m., Andrei Budnik wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70889/
> -----------------------------------------------------------
>
> (Updated June 19, 2019, 7:49 a.m.)
>
>
> Review request for mesos, Gilbert Song and Qian Zhang.
>
>
> Bugs: MESOS-9841
> https://issues.apache.org/jira/browse/MESOS-9841
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch wraps every isolator in instance of `IsolatorTracker` class.
> If an isolator gets stuck in some operation, `pendingFutures` will
> return the isolator name, method name along with its arguments such as
> `containerId`, `pid`, etc.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.hpp d33c65cd3fae9bb4d347681cb00dbdb25ecf57b8
> src/slave/containerizer/containerizer.cpp 5ce0d9c4bfa707495ea644c90fa6e35eb0fe25cd
> src/slave/containerizer/mesos/containerizer.hpp 558e412b28a84ae0ca1bf05b596540072bb216d6
> src/slave/containerizer/mesos/containerizer.cpp c9a369bcdbfc1676912430c3e84fa567bbd7f766
> src/slave/main.cpp ef5ea02f169427d9913b807664bf3073d72cd9b6
> src/tests/cluster.hpp c04ee14beafe350568a295c8258566aa3704028a
> src/tests/cluster.cpp b43f806eab096b7d4e86e2b5d4b81d6baecb0ee5
>
>
> Diff: https://reviews.apache.org/r/70889/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Andrei Budnik
>
>
Re: Review Request 70889: Wrapped isolators in `IsolatorTracker`.
Posted by Gilbert Song <so...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70889/#review216727
-----------------------------------------------------------
Ship it!
Ship It!
- Gilbert Song
On June 19, 2019, 7:49 a.m., Andrei Budnik wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70889/
> -----------------------------------------------------------
>
> (Updated June 19, 2019, 7:49 a.m.)
>
>
> Review request for mesos, Gilbert Song and Qian Zhang.
>
>
> Bugs: MESOS-9841
> https://issues.apache.org/jira/browse/MESOS-9841
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch wraps every isolator in instance of `IsolatorTracker` class.
> If an isolator gets stuck in some operation, `pendingFutures` will
> return the isolator name, method name along with its arguments such as
> `containerId`, `pid`, etc.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.hpp d33c65cd3fae9bb4d347681cb00dbdb25ecf57b8
> src/slave/containerizer/containerizer.cpp 5ce0d9c4bfa707495ea644c90fa6e35eb0fe25cd
> src/slave/containerizer/mesos/containerizer.hpp 558e412b28a84ae0ca1bf05b596540072bb216d6
> src/slave/containerizer/mesos/containerizer.cpp c9a369bcdbfc1676912430c3e84fa567bbd7f766
> src/slave/main.cpp ef5ea02f169427d9913b807664bf3073d72cd9b6
> src/tests/cluster.cpp b43f806eab096b7d4e86e2b5d4b81d6baecb0ee5
>
>
> Diff: https://reviews.apache.org/r/70889/diff/6/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Andrei Budnik
>
>
Re: Review Request 70889: Wrapped isolators in `IsolatorTracker`.
Posted by Qian Zhang <zh...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70889/#review216260
-----------------------------------------------------------
src/slave/containerizer/mesos/containerizer.cpp
Lines 586-589 (patched)
<https://reviews.apache.org/r/70889/#comment303390>
Why do we need a dedicated for loop to do this? Can we just do it when each isolator is created (line 549 and line 563)?
- Qian Zhang
On June 19, 2019, 10:49 p.m., Andrei Budnik wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70889/
> -----------------------------------------------------------
>
> (Updated June 19, 2019, 10:49 p.m.)
>
>
> Review request for mesos, Gilbert Song and Qian Zhang.
>
>
> Bugs: MESOS-9841
> https://issues.apache.org/jira/browse/MESOS-9841
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch wraps every isolator in instance of `IsolatorTracker` class.
> If an isolator gets stuck in some operation, `pendingFutures` will
> return the isolator name, method name along with its arguments such as
> `containerId`, `pid`, etc.
>
>
> Diffs
> -----
>
> src/slave/containerizer/mesos/containerizer.cpp 043244841a73fa3f5f7119bc38f6d3a04be8990b
>
>
> Diff: https://reviews.apache.org/r/70889/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Andrei Budnik
>
>
Re: Review Request 70889: Wrapped isolators in `IsolatorTracker`.
Posted by Qian Zhang <zh...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70889/#review216647
-----------------------------------------------------------
src/slave/main.cpp
Lines 495 (patched)
<https://reviews.apache.org/r/70889/#comment303849>
Kill this newline.
- Qian Zhang
On June 19, 2019, 10:49 p.m., Andrei Budnik wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70889/
> -----------------------------------------------------------
>
> (Updated June 19, 2019, 10:49 p.m.)
>
>
> Review request for mesos, Gilbert Song and Qian Zhang.
>
>
> Bugs: MESOS-9841
> https://issues.apache.org/jira/browse/MESOS-9841
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch wraps every isolator in instance of `IsolatorTracker` class.
> If an isolator gets stuck in some operation, `pendingFutures` will
> return the isolator name, method name along with its arguments such as
> `containerId`, `pid`, etc.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.hpp d33c65cd3fae9bb4d347681cb00dbdb25ecf57b8
> src/slave/containerizer/containerizer.cpp 5ce0d9c4bfa707495ea644c90fa6e35eb0fe25cd
> src/slave/containerizer/mesos/containerizer.hpp 558e412b28a84ae0ca1bf05b596540072bb216d6
> src/slave/containerizer/mesos/containerizer.cpp c9a369bcdbfc1676912430c3e84fa567bbd7f766
> src/slave/main.cpp ef5ea02f169427d9913b807664bf3073d72cd9b6
> src/tests/cluster.hpp c04ee14beafe350568a295c8258566aa3704028a
> src/tests/cluster.cpp b43f806eab096b7d4e86e2b5d4b81d6baecb0ee5
>
>
> Diff: https://reviews.apache.org/r/70889/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Andrei Budnik
>
>
Re: Review Request 70889: Wrapped isolators in `IsolatorTracker`.
Posted by Andrei Budnik <ab...@mesosphere.com>.
> On Июль 16, 2019, 7:29 д.п., Gilbert Song wrote:
> > src/slave/main.cpp
> > Lines 494 (patched)
> > <https://reviews.apache.org/r/70889/diff/2/?file=2154669#file2154669line494>
> >
> > is it possible to construct the `PendingFutureTracker` process in containerizer::create()
An instance of the `PendingFutureTracker` is used agent-wide. The following patches pass `futureTracker` to the agent so that http handlers could query for pending futures.
- Andrei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70889/#review216643
-----------------------------------------------------------
On Июнь 19, 2019, 2:49 п.п., Andrei Budnik wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70889/
> -----------------------------------------------------------
>
> (Updated Июнь 19, 2019, 2:49 п.п.)
>
>
> Review request for mesos, Gilbert Song and Qian Zhang.
>
>
> Bugs: MESOS-9841
> https://issues.apache.org/jira/browse/MESOS-9841
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch wraps every isolator in instance of `IsolatorTracker` class.
> If an isolator gets stuck in some operation, `pendingFutures` will
> return the isolator name, method name along with its arguments such as
> `containerId`, `pid`, etc.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.hpp d33c65cd3fae9bb4d347681cb00dbdb25ecf57b8
> src/slave/containerizer/containerizer.cpp 5ce0d9c4bfa707495ea644c90fa6e35eb0fe25cd
> src/slave/containerizer/mesos/containerizer.hpp 558e412b28a84ae0ca1bf05b596540072bb216d6
> src/slave/containerizer/mesos/containerizer.cpp c9a369bcdbfc1676912430c3e84fa567bbd7f766
> src/slave/main.cpp ef5ea02f169427d9913b807664bf3073d72cd9b6
> src/tests/cluster.hpp c04ee14beafe350568a295c8258566aa3704028a
> src/tests/cluster.cpp b43f806eab096b7d4e86e2b5d4b81d6baecb0ee5
>
>
> Diff: https://reviews.apache.org/r/70889/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Andrei Budnik
>
>
Re: Review Request 70889: Wrapped isolators in `IsolatorTracker`.
Posted by Qian Zhang <zh...@gmail.com>.
> On July 16, 2019, 3:29 p.m., Gilbert Song wrote:
> > src/slave/containerizer/mesos/containerizer.cpp
> > Lines 550-552 (patched)
> > <https://reviews.apache.org/r/70889/diff/2/?file=2154668#file2154668line550>
> >
> > seems like we are going to have one more libprocess process for each isolator/module. we may have some concerns on perf.
> >
> > would you mind to collect some perf data for w/o isolatorTracker and compare?
> >
> > A quick method to test it out is to use mesos-executor and launch 100 containers in one single task group.
I think the performance overhead should be acceptable. Yes, we add a libprocess `PendingFutureTrackerProcess`, but for each isolator's method, we just need to dispatch a call (i.e., `PendingFutureTrackerProcess::addFuture`) to that libprocess but no need to wait for that libprocess to complete that call, so the performance overhead are just some functions calls but no extra thread switches.
- Qian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70889/#review216643
-----------------------------------------------------------
On June 19, 2019, 10:49 p.m., Andrei Budnik wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70889/
> -----------------------------------------------------------
>
> (Updated June 19, 2019, 10:49 p.m.)
>
>
> Review request for mesos, Gilbert Song and Qian Zhang.
>
>
> Bugs: MESOS-9841
> https://issues.apache.org/jira/browse/MESOS-9841
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch wraps every isolator in instance of `IsolatorTracker` class.
> If an isolator gets stuck in some operation, `pendingFutures` will
> return the isolator name, method name along with its arguments such as
> `containerId`, `pid`, etc.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.hpp d33c65cd3fae9bb4d347681cb00dbdb25ecf57b8
> src/slave/containerizer/containerizer.cpp 5ce0d9c4bfa707495ea644c90fa6e35eb0fe25cd
> src/slave/containerizer/mesos/containerizer.hpp 558e412b28a84ae0ca1bf05b596540072bb216d6
> src/slave/containerizer/mesos/containerizer.cpp c9a369bcdbfc1676912430c3e84fa567bbd7f766
> src/slave/main.cpp ef5ea02f169427d9913b807664bf3073d72cd9b6
> src/tests/cluster.hpp c04ee14beafe350568a295c8258566aa3704028a
> src/tests/cluster.cpp b43f806eab096b7d4e86e2b5d4b81d6baecb0ee5
>
>
> Diff: https://reviews.apache.org/r/70889/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Andrei Budnik
>
>
Re: Review Request 70889: Wrapped isolators in `IsolatorTracker`.
Posted by Gilbert Song <so...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70889/#review216643
-----------------------------------------------------------
src/slave/containerizer/mesos/containerizer.cpp
Lines 550-552 (patched)
<https://reviews.apache.org/r/70889/#comment303847>
seems like we are going to have one more libprocess process for each isolator/module. we may have some concerns on perf.
would you mind to collect some perf data for w/o isolatorTracker and compare?
A quick method to test it out is to use mesos-executor and launch 100 containers in one single task group.
src/slave/main.cpp
Lines 494 (patched)
<https://reviews.apache.org/r/70889/#comment303846>
is it possible to construct the `PendingFutureTracker` process in containerizer::create()
- Gilbert Song
On June 19, 2019, 7:49 a.m., Andrei Budnik wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70889/
> -----------------------------------------------------------
>
> (Updated June 19, 2019, 7:49 a.m.)
>
>
> Review request for mesos, Gilbert Song and Qian Zhang.
>
>
> Bugs: MESOS-9841
> https://issues.apache.org/jira/browse/MESOS-9841
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch wraps every isolator in instance of `IsolatorTracker` class.
> If an isolator gets stuck in some operation, `pendingFutures` will
> return the isolator name, method name along with its arguments such as
> `containerId`, `pid`, etc.
>
>
> Diffs
> -----
>
> src/slave/containerizer/containerizer.hpp d33c65cd3fae9bb4d347681cb00dbdb25ecf57b8
> src/slave/containerizer/containerizer.cpp 5ce0d9c4bfa707495ea644c90fa6e35eb0fe25cd
> src/slave/containerizer/mesos/containerizer.hpp 558e412b28a84ae0ca1bf05b596540072bb216d6
> src/slave/containerizer/mesos/containerizer.cpp c9a369bcdbfc1676912430c3e84fa567bbd7f766
> src/slave/main.cpp ef5ea02f169427d9913b807664bf3073d72cd9b6
> src/tests/cluster.hpp c04ee14beafe350568a295c8258566aa3704028a
> src/tests/cluster.cpp b43f806eab096b7d4e86e2b5d4b81d6baecb0ee5
>
>
> Diff: https://reviews.apache.org/r/70889/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Andrei Budnik
>
>