You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by haosdent huang <ha...@gmail.com> on 2016/02/16 18:55:10 UTC
Review Request 43608: Speed up FetcherCache test cases by disable
framework checkpoint.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43608/
-----------------------------------------------------------
Review request for mesos, Alexander Rukletsov, Benjamin Bannier, and Bernd Mathiske.
Bugs: MESOS-4685
https://issues.apache.org/jira/browse/MESOS-4685
Repository: mesos
Description
-------
Speed up FetcherCache test cases by disable framework checkpoint.
Diffs
-----
src/tests/fetcher_cache_tests.cpp 1cf45660691860793ac600363f7934e13a2e7ddf
Diff: https://reviews.apache.org/r/43608/diff/
Testing
-------
Before
```
[ OK ] FetcherCacheTest.LocalUncached (2645 ms)
[ OK ] FetcherCacheTest.LocalCached (2653 ms)
[ OK ] FetcherCacheTest.CachedFallback (1110 ms)
[ OK ] FetcherCacheTest.LocalUncachedExtract (2620 ms)
[ OK ] FetcherCacheTest.LocalCachedExtract (2758 ms)
[ OK ] FetcherCacheTest.SimpleEviction (5059 ms)
[ OK ] FetcherCacheTest.FallbackFromEviction (2648 ms)
[ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3661 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2600 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (2013 ms)
[ OK ] FetcherCacheHttpTest.HttpMixed (1746 ms)
```
After
```
[ OK ] FetcherCacheTest.LocalUncached (2580 ms)
[ OK ] FetcherCacheTest.LocalCached (2516 ms)
[ OK ] FetcherCacheTest.CachedFallback (957 ms)
[ OK ] FetcherCacheTest.LocalUncachedExtract (2519 ms)
[ OK ] FetcherCacheTest.LocalCachedExtract (2594 ms)
[ OK ] FetcherCacheTest.SimpleEviction (4532 ms)
[ OK ] FetcherCacheTest.FallbackFromEviction (2579 ms)
[ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3528 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2520 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (1534 ms)
[ OK ] FetcherCacheHttpTest.HttpMixed (1345 ms)
```
And test the recovery test manually:
```
./bin/mesos-tests.sh --gtest_filter="FetcherCacheHttpRecoveryTest.HttpCachedRecovery"
```
Thanks,
haosdent huang
Re: Review Request 43608: Speed up FetcherCache test cases by disable
framework checkpoint.
Posted by haosdent huang <ha...@gmail.com>.
> On Feb. 17, 2016, 3:21 p.m., Bernd Mathiske wrote:
> > Are a couple of seconds wirth the extra code complexity? Opinions, please!
Yes, @bernd, as you see here, this patch don't bring too much effects. How about let's discard it and reopen it if really necessary?
- haosdent
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43608/#review119462
-----------------------------------------------------------
On Feb. 17, 2016, 2:10 a.m., haosdent huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43608/
> -----------------------------------------------------------
>
> (Updated Feb. 17, 2016, 2:10 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Bannier, and Bernd Mathiske.
>
>
> Bugs: MESOS-4685
> https://issues.apache.org/jira/browse/MESOS-4685
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Speed up FetcherCache test cases by disable framework checkpoint.
>
>
> Diffs
> -----
>
> src/tests/fetcher_cache_tests.cpp fc89be46bc4cca81f6f3f5072539d3dae26f7c8f
>
> Diff: https://reviews.apache.org/r/43608/diff/
>
>
> Testing
> -------
>
> When we enable checkpoint for frameworks, StatusUpdateManager would waiting for write StatusUpdateRecord to file and then boardcast it. This cause uncessary disk operations and bring unstable delay if our test cases don't depends on recovery.
>
> Before
> ```
> [ OK ] FetcherCacheTest.LocalUncached (2645 ms)
> [ OK ] FetcherCacheTest.LocalCached (2653 ms)
> [ OK ] FetcherCacheTest.CachedFallback (1110 ms)
> [ OK ] FetcherCacheTest.LocalUncachedExtract (2620 ms)
> [ OK ] FetcherCacheTest.LocalCachedExtract (2758 ms)
> [ OK ] FetcherCacheTest.SimpleEviction (5059 ms)
> [ OK ] FetcherCacheTest.FallbackFromEviction (2648 ms)
> [ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3661 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2600 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (2013 ms)
> [ OK ] FetcherCacheHttpTest.HttpMixed (1746 ms)
> ```
>
> After
> ```
> [ OK ] FetcherCacheTest.LocalUncached (2580 ms)
> [ OK ] FetcherCacheTest.LocalCached (2516 ms)
> [ OK ] FetcherCacheTest.CachedFallback (957 ms)
> [ OK ] FetcherCacheTest.LocalUncachedExtract (2519 ms)
> [ OK ] FetcherCacheTest.LocalCachedExtract (2594 ms)
> [ OK ] FetcherCacheTest.SimpleEviction (4532 ms)
> [ OK ] FetcherCacheTest.FallbackFromEviction (2579 ms)
> [ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3528 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2520 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (1534 ms)
> [ OK ] FetcherCacheHttpTest.HttpMixed (1345 ms)
> ```
>
> And test the recovery test manually:
> ```
> ./bin/mesos-tests.sh --gtest_filter="FetcherCacheHttpRecoveryTest.HttpCachedRecovery"
> ```
>
>
> Thanks,
>
> haosdent huang
>
>
Re: Review Request 43608: Speed up FetcherCache test cases by disable
framework checkpoint.
Posted by Alexander Rukletsov <ru...@gmail.com>.
> On Feb. 17, 2016, 3:21 p.m., Bernd Mathiske wrote:
> > Are a couple of seconds wirth the extra code complexity? Opinions, please!
>
> haosdent huang wrote:
> Yes, @bernd, as you see here, this patch don't bring too much effects. How about let's discard it and reopen it if really necessary?
>
> Bernd Mathiske wrote:
> Agreed! Thanks!
I'd say the speed-up is marginal and maybe was not worth extra effort. However, disabling checkpointing for some tests also serves educational and explanatory role, hence I'm +1 for this change.
- Alexander
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43608/#review119462
-----------------------------------------------------------
On Feb. 17, 2016, 2:10 a.m., haosdent huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43608/
> -----------------------------------------------------------
>
> (Updated Feb. 17, 2016, 2:10 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Bannier, and Bernd Mathiske.
>
>
> Bugs: MESOS-4685
> https://issues.apache.org/jira/browse/MESOS-4685
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Speed up FetcherCache test cases by disable framework checkpoint.
>
>
> Diffs
> -----
>
> src/tests/fetcher_cache_tests.cpp fc89be46bc4cca81f6f3f5072539d3dae26f7c8f
>
> Diff: https://reviews.apache.org/r/43608/diff/
>
>
> Testing
> -------
>
> When we enable checkpoint for frameworks, StatusUpdateManager would waiting for write StatusUpdateRecord to file and then boardcast it. This cause uncessary disk operations and bring unstable delay if our test cases don't depends on recovery.
>
> Before
> ```
> [ OK ] FetcherCacheTest.LocalUncached (2645 ms)
> [ OK ] FetcherCacheTest.LocalCached (2653 ms)
> [ OK ] FetcherCacheTest.CachedFallback (1110 ms)
> [ OK ] FetcherCacheTest.LocalUncachedExtract (2620 ms)
> [ OK ] FetcherCacheTest.LocalCachedExtract (2758 ms)
> [ OK ] FetcherCacheTest.SimpleEviction (5059 ms)
> [ OK ] FetcherCacheTest.FallbackFromEviction (2648 ms)
> [ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3661 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2600 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (2013 ms)
> [ OK ] FetcherCacheHttpTest.HttpMixed (1746 ms)
> ```
>
> After
> ```
> [ OK ] FetcherCacheTest.LocalUncached (2580 ms)
> [ OK ] FetcherCacheTest.LocalCached (2516 ms)
> [ OK ] FetcherCacheTest.CachedFallback (957 ms)
> [ OK ] FetcherCacheTest.LocalUncachedExtract (2519 ms)
> [ OK ] FetcherCacheTest.LocalCachedExtract (2594 ms)
> [ OK ] FetcherCacheTest.SimpleEviction (4532 ms)
> [ OK ] FetcherCacheTest.FallbackFromEviction (2579 ms)
> [ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3528 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2520 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (1534 ms)
> [ OK ] FetcherCacheHttpTest.HttpMixed (1345 ms)
> ```
>
> And test the recovery test manually:
> ```
> ./bin/mesos-tests.sh --gtest_filter="FetcherCacheHttpRecoveryTest.HttpCachedRecovery"
> ```
>
>
> Thanks,
>
> haosdent huang
>
>
Re: Review Request 43608: Speed up FetcherCache test cases by disable
framework checkpoint.
Posted by Bernd Mathiske <be...@mesosphere.io>.
> On Feb. 17, 2016, 7:21 a.m., Bernd Mathiske wrote:
> > Are a couple of seconds wirth the extra code complexity? Opinions, please!
>
> haosdent huang wrote:
> Yes, @bernd, as you see here, this patch don't bring too much effects. How about let's discard it and reopen it if really necessary?
Agreed! Thanks!
- Bernd
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43608/#review119462
-----------------------------------------------------------
On Feb. 16, 2016, 6:10 p.m., haosdent huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43608/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2016, 6:10 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Bannier, and Bernd Mathiske.
>
>
> Bugs: MESOS-4685
> https://issues.apache.org/jira/browse/MESOS-4685
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Speed up FetcherCache test cases by disable framework checkpoint.
>
>
> Diffs
> -----
>
> src/tests/fetcher_cache_tests.cpp fc89be46bc4cca81f6f3f5072539d3dae26f7c8f
>
> Diff: https://reviews.apache.org/r/43608/diff/
>
>
> Testing
> -------
>
> When we enable checkpoint for frameworks, StatusUpdateManager would waiting for write StatusUpdateRecord to file and then boardcast it. This cause uncessary disk operations and bring unstable delay if our test cases don't depends on recovery.
>
> Before
> ```
> [ OK ] FetcherCacheTest.LocalUncached (2645 ms)
> [ OK ] FetcherCacheTest.LocalCached (2653 ms)
> [ OK ] FetcherCacheTest.CachedFallback (1110 ms)
> [ OK ] FetcherCacheTest.LocalUncachedExtract (2620 ms)
> [ OK ] FetcherCacheTest.LocalCachedExtract (2758 ms)
> [ OK ] FetcherCacheTest.SimpleEviction (5059 ms)
> [ OK ] FetcherCacheTest.FallbackFromEviction (2648 ms)
> [ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3661 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2600 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (2013 ms)
> [ OK ] FetcherCacheHttpTest.HttpMixed (1746 ms)
> ```
>
> After
> ```
> [ OK ] FetcherCacheTest.LocalUncached (2580 ms)
> [ OK ] FetcherCacheTest.LocalCached (2516 ms)
> [ OK ] FetcherCacheTest.CachedFallback (957 ms)
> [ OK ] FetcherCacheTest.LocalUncachedExtract (2519 ms)
> [ OK ] FetcherCacheTest.LocalCachedExtract (2594 ms)
> [ OK ] FetcherCacheTest.SimpleEviction (4532 ms)
> [ OK ] FetcherCacheTest.FallbackFromEviction (2579 ms)
> [ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3528 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2520 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (1534 ms)
> [ OK ] FetcherCacheHttpTest.HttpMixed (1345 ms)
> ```
>
> And test the recovery test manually:
> ```
> ./bin/mesos-tests.sh --gtest_filter="FetcherCacheHttpRecoveryTest.HttpCachedRecovery"
> ```
>
>
> Thanks,
>
> haosdent huang
>
>
Re: Review Request 43608: Speed up FetcherCache test cases by disable
framework checkpoint.
Posted by Bernd Mathiske <be...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43608/#review119462
-----------------------------------------------------------
Are a couple of seconds wirth the extra code complexity? Opinions, please!
- Bernd Mathiske
On Feb. 16, 2016, 6:10 p.m., haosdent huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43608/
> -----------------------------------------------------------
>
> (Updated Feb. 16, 2016, 6:10 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Bannier, and Bernd Mathiske.
>
>
> Bugs: MESOS-4685
> https://issues.apache.org/jira/browse/MESOS-4685
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Speed up FetcherCache test cases by disable framework checkpoint.
>
>
> Diffs
> -----
>
> src/tests/fetcher_cache_tests.cpp fc89be46bc4cca81f6f3f5072539d3dae26f7c8f
>
> Diff: https://reviews.apache.org/r/43608/diff/
>
>
> Testing
> -------
>
> When we enable checkpoint for frameworks, StatusUpdateManager would waiting for write StatusUpdateRecord to file and then boardcast it. This cause uncessary disk operations and bring unstable delay if our test cases don't depends on recovery.
>
> Before
> ```
> [ OK ] FetcherCacheTest.LocalUncached (2645 ms)
> [ OK ] FetcherCacheTest.LocalCached (2653 ms)
> [ OK ] FetcherCacheTest.CachedFallback (1110 ms)
> [ OK ] FetcherCacheTest.LocalUncachedExtract (2620 ms)
> [ OK ] FetcherCacheTest.LocalCachedExtract (2758 ms)
> [ OK ] FetcherCacheTest.SimpleEviction (5059 ms)
> [ OK ] FetcherCacheTest.FallbackFromEviction (2648 ms)
> [ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3661 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2600 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (2013 ms)
> [ OK ] FetcherCacheHttpTest.HttpMixed (1746 ms)
> ```
>
> After
> ```
> [ OK ] FetcherCacheTest.LocalUncached (2580 ms)
> [ OK ] FetcherCacheTest.LocalCached (2516 ms)
> [ OK ] FetcherCacheTest.CachedFallback (957 ms)
> [ OK ] FetcherCacheTest.LocalUncachedExtract (2519 ms)
> [ OK ] FetcherCacheTest.LocalCachedExtract (2594 ms)
> [ OK ] FetcherCacheTest.SimpleEviction (4532 ms)
> [ OK ] FetcherCacheTest.FallbackFromEviction (2579 ms)
> [ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3528 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2520 ms)
> [ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (1534 ms)
> [ OK ] FetcherCacheHttpTest.HttpMixed (1345 ms)
> ```
>
> And test the recovery test manually:
> ```
> ./bin/mesos-tests.sh --gtest_filter="FetcherCacheHttpRecoveryTest.HttpCachedRecovery"
> ```
>
>
> Thanks,
>
> haosdent huang
>
>
Re: Review Request 43608: Speed up FetcherCache test cases by disable
framework checkpoint.
Posted by haosdent huang <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43608/
-----------------------------------------------------------
(Updated Feb. 17, 2016, 2:10 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Bannier, and Bernd Mathiske.
Changes
-------
Rebase
Bugs: MESOS-4685
https://issues.apache.org/jira/browse/MESOS-4685
Repository: mesos
Description
-------
Speed up FetcherCache test cases by disable framework checkpoint.
Diffs (updated)
-----
src/tests/fetcher_cache_tests.cpp fc89be46bc4cca81f6f3f5072539d3dae26f7c8f
Diff: https://reviews.apache.org/r/43608/diff/
Testing
-------
When we enable checkpoint for frameworks, StatusUpdateManager would waiting for write StatusUpdateRecord to file and then boardcast it. This cause uncessary disk operations and bring unstable delay if our test cases don't depends on recovery.
Before
```
[ OK ] FetcherCacheTest.LocalUncached (2645 ms)
[ OK ] FetcherCacheTest.LocalCached (2653 ms)
[ OK ] FetcherCacheTest.CachedFallback (1110 ms)
[ OK ] FetcherCacheTest.LocalUncachedExtract (2620 ms)
[ OK ] FetcherCacheTest.LocalCachedExtract (2758 ms)
[ OK ] FetcherCacheTest.SimpleEviction (5059 ms)
[ OK ] FetcherCacheTest.FallbackFromEviction (2648 ms)
[ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3661 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2600 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (2013 ms)
[ OK ] FetcherCacheHttpTest.HttpMixed (1746 ms)
```
After
```
[ OK ] FetcherCacheTest.LocalUncached (2580 ms)
[ OK ] FetcherCacheTest.LocalCached (2516 ms)
[ OK ] FetcherCacheTest.CachedFallback (957 ms)
[ OK ] FetcherCacheTest.LocalUncachedExtract (2519 ms)
[ OK ] FetcherCacheTest.LocalCachedExtract (2594 ms)
[ OK ] FetcherCacheTest.SimpleEviction (4532 ms)
[ OK ] FetcherCacheTest.FallbackFromEviction (2579 ms)
[ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3528 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2520 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (1534 ms)
[ OK ] FetcherCacheHttpTest.HttpMixed (1345 ms)
```
And test the recovery test manually:
```
./bin/mesos-tests.sh --gtest_filter="FetcherCacheHttpRecoveryTest.HttpCachedRecovery"
```
Thanks,
haosdent huang
Re: Review Request 43608: Speed up FetcherCache test cases by disable
framework checkpoint.
Posted by haosdent huang <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43608/
-----------------------------------------------------------
(Updated Feb. 16, 2016, 6:12 p.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Bannier, and Bernd Mathiske.
Changes
-------
Add explanation in testing done.
Bugs: MESOS-4685
https://issues.apache.org/jira/browse/MESOS-4685
Repository: mesos
Description
-------
Speed up FetcherCache test cases by disable framework checkpoint.
Diffs
-----
src/tests/fetcher_cache_tests.cpp 1cf45660691860793ac600363f7934e13a2e7ddf
Diff: https://reviews.apache.org/r/43608/diff/
Testing (updated)
-------
When we enable checkpoint for frameworks, StatusUpdateManager would waiting for write StatusUpdateRecord to file and then boardcast it. This cause uncessary disk operations and bring unstable delay if our test cases don't depends on recovery.
Before
```
[ OK ] FetcherCacheTest.LocalUncached (2645 ms)
[ OK ] FetcherCacheTest.LocalCached (2653 ms)
[ OK ] FetcherCacheTest.CachedFallback (1110 ms)
[ OK ] FetcherCacheTest.LocalUncachedExtract (2620 ms)
[ OK ] FetcherCacheTest.LocalCachedExtract (2758 ms)
[ OK ] FetcherCacheTest.SimpleEviction (5059 ms)
[ OK ] FetcherCacheTest.FallbackFromEviction (2648 ms)
[ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3661 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2600 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (2013 ms)
[ OK ] FetcherCacheHttpTest.HttpMixed (1746 ms)
```
After
```
[ OK ] FetcherCacheTest.LocalUncached (2580 ms)
[ OK ] FetcherCacheTest.LocalCached (2516 ms)
[ OK ] FetcherCacheTest.CachedFallback (957 ms)
[ OK ] FetcherCacheTest.LocalUncachedExtract (2519 ms)
[ OK ] FetcherCacheTest.LocalCachedExtract (2594 ms)
[ OK ] FetcherCacheTest.SimpleEviction (4532 ms)
[ OK ] FetcherCacheTest.FallbackFromEviction (2579 ms)
[ OK ] FetcherCacheTest.RemoveLRUCacheEntries (3528 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedSerialized (2520 ms)
[ OK ] FetcherCacheHttpTest.HttpCachedConcurrent (1534 ms)
[ OK ] FetcherCacheHttpTest.HttpMixed (1345 ms)
```
And test the recovery test manually:
```
./bin/mesos-tests.sh --gtest_filter="FetcherCacheHttpRecoveryTest.HttpCachedRecovery"
```
Thanks,
haosdent huang