You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Benjamin Mahler <bm...@apache.org> on 2018/07/10 20:51:08 UTC

Review Request 67877: Added a metrics scalability benchmark in libprocess.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67877/
-----------------------------------------------------------

Review request for mesos and Greg Mann.


Bugs: MESOS-9068
    https://issues.apache.org/jira/browse/MESOS-9068


Repository: mesos


Description
-------

This initial benchmark focuses on adding a large number of counters
and measuring how long the add/remove and snapshot takes.


Diffs
-----

  3rdparty/libprocess/src/tests/metrics_tests.cpp a5b41ed3f4491244656222681f443e7ed55393c2 


Diff: https://reviews.apache.org/r/67877/diff/1/


Testing
-------

Before https://reviews.apache.org/r/67871/:

```
[==========] Running 5 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
Added 1 counters in 168443ns
Snapshot of 1 counters in 339047ns
HTTP /snapshot of 1 counters in 1.999797ms
Removed 1 counters in 94595ns
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (3 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
Added 100 counters in 3.542703ms
Snapshot of 100 counters in 1.119107ms
HTTP /snapshot of 100 counters in 500.982997ms
Removed 100 counters in 3.949413ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (510 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
Added 1000 counters in 27.697086ms
Snapshot of 1000 counters in 9.232065ms
HTTP /snapshot of 1000 counters in 473.66825ms
Removed 1000 counters in 31.632763ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (544 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
Added 10000 counters in 277.567781ms
Snapshot of 10000 counters in 81.45371ms
HTTP /snapshot of 10000 counters in 219.1157ms
Removed 10000 counters in 309.334783ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (902 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
Added 100000 counters in 3.753352484secs
Snapshot of 100000 counters in 1.033072727secs
HTTP /snapshot of 100000 counters in 2.226777961secs
Removed 100000 counters in 4.851697589secs
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (11996 ms)
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (13955 ms total)

[----------] Global test environment tear-down
[==========] 5 tests from 1 test case ran. (13955 ms total)
[  PASSED  ] 5 tests.
```

After https://reviews.apache.org/r/67871/:

```
[==========] Running 5 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
Added 1 counters in 102994ns
Snapshot of 1 counters in 266999ns
HTTP /snapshot of 1 counters in 1.83628ms
Removed 1 counters in 27402ns
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (2 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
Added 100 counters in 3.049965ms
Snapshot of 100 counters in 930319ns
HTTP /snapshot of 100 counters in 501.558737ms
Removed 100 counters in 4.996015ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (511 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
Added 1000 counters in 28.623101ms
Snapshot of 1000 counters in 7.496592ms
HTTP /snapshot of 1000 counters in 470.870567ms
Removed 1000 counters in 39.663126ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (548 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
Added 10000 counters in 335.506147ms
Snapshot of 10000 counters in 67.963256ms
HTTP /snapshot of 10000 counters in 154.661098ms
Removed 10000 counters in 333.06148ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (904 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
Added 100000 counters in 3.396691566secs
Snapshot of 100000 counters in 640.134635ms
HTTP /snapshot of 100000 counters in 1.404702464secs
Removed 100000 counters in 3.526395358secs
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (9119 ms)
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (11084 ms total)

[----------] Global test environment tear-down
[==========] 5 tests from 1 test case ran. (11084 ms total)
[  PASSED  ] 5 tests.
```


Thanks,

Benjamin Mahler

Re: Review Request 67877: Added a metrics scalability benchmark in libprocess.

Posted by Mesos Reviewbot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67877/#review205982
-----------------------------------------------------------



Patch looks great!

Reviews applied: [67877]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; ./support/docker-build.sh

- Mesos Reviewbot


On July 11, 2018, 3:32 p.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67877/
> -----------------------------------------------------------
> 
> (Updated July 11, 2018, 3:32 p.m.)
> 
> 
> Review request for mesos and Greg Mann.
> 
> 
> Bugs: MESOS-9068
>     https://issues.apache.org/jira/browse/MESOS-9068
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This initial benchmark focuses on adding a large number of counters
> and measuring how long the add/remove and snapshot takes.
> 
> 
> Diffs
> -----
> 
>   3rdparty/libprocess/src/tests/benchmarks.cpp 604122a2f33d7abfab92631d9eac33b6753e846b 
> 
> 
> Diff: https://reviews.apache.org/r/67877/diff/2/
> 
> 
> Testing
> -------
> 
> Before https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 168443ns
> Snapshot of 1 counters in 339047ns
> HTTP /snapshot of 1 counters in 1.999797ms
> Removed 1 counters in 94595ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (3 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.542703ms
> Snapshot of 100 counters in 1.119107ms
> HTTP /snapshot of 100 counters in 500.982997ms
> Removed 100 counters in 3.949413ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (510 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 27.697086ms
> Snapshot of 1000 counters in 9.232065ms
> HTTP /snapshot of 1000 counters in 473.66825ms
> Removed 1000 counters in 31.632763ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (544 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 277.567781ms
> Snapshot of 10000 counters in 81.45371ms
> HTTP /snapshot of 10000 counters in 219.1157ms
> Removed 10000 counters in 309.334783ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (902 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.753352484secs
> Snapshot of 100000 counters in 1.033072727secs
> HTTP /snapshot of 100000 counters in 2.226777961secs
> Removed 100000 counters in 4.851697589secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (11996 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (13955 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (13955 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> After https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 102994ns
> Snapshot of 1 counters in 266999ns
> HTTP /snapshot of 1 counters in 1.83628ms
> Removed 1 counters in 27402ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (2 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.049965ms
> Snapshot of 100 counters in 930319ns
> HTTP /snapshot of 100 counters in 501.558737ms
> Removed 100 counters in 4.996015ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (511 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 28.623101ms
> Snapshot of 1000 counters in 7.496592ms
> HTTP /snapshot of 1000 counters in 470.870567ms
> Removed 1000 counters in 39.663126ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (548 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 335.506147ms
> Snapshot of 10000 counters in 67.963256ms
> HTTP /snapshot of 10000 counters in 154.661098ms
> Removed 10000 counters in 333.06148ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (904 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.396691566secs
> Snapshot of 100000 counters in 640.134635ms
> HTTP /snapshot of 100000 counters in 1.404702464secs
> Removed 100000 counters in 3.526395358secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (9119 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (11084 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (11084 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>

Re: Review Request 67877: Added a metrics scalability benchmark in libprocess.

Posted by Benjamin Mahler <bm...@apache.org>.

> On July 25, 2018, 5:48 p.m., Greg Mann wrote:
> > 3rdparty/libprocess/src/tests/benchmarks.cpp
> > Lines 704 (patched)
> > <https://reviews.apache.org/r/67877/diff/3/?file=2058569#file2058569line704>
> >
> >     s/are/is/

Hm.. looks like there's no clear choice in this case:

https://www.grammarly.com/blog/there-is-there-are/

I assume this isn't an issue for you even though this was marked as one?


> On July 25, 2018, 5:48 p.m., Greg Mann wrote:
> > 3rdparty/libprocess/src/tests/benchmarks.cpp
> > Lines 707 (patched)
> > <https://reviews.apache.org/r/67877/diff/3/?file=2058569#file2058569line707>
> >
> >     Nit: seems like we use a mix of snake case and camel case in this file, but camel case is a bit more common? I'm fine either way here, but wanted to bring it up.

Yeah, we're in a weird territory right now unfortunately. My understanding is that libprocess started as snake_case, but later it took on some camelCase from mesos cross-pollination. We can't change mesos to snake_case easily, but we can try to stick to snake_case in libprocess. That being said, it's confusing for folks. I prefer snake_case and if tooling can support that inconsistency across mesos and libprocess then we're set.


- Benjamin


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67877/#review206458
-----------------------------------------------------------


On July 11, 2018, 10:51 p.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67877/
> -----------------------------------------------------------
> 
> (Updated July 11, 2018, 10:51 p.m.)
> 
> 
> Review request for mesos and Greg Mann.
> 
> 
> Bugs: MESOS-9068
>     https://issues.apache.org/jira/browse/MESOS-9068
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This initial benchmark focuses on adding a large number of counters
> and measuring how long the add/remove and snapshot takes.
> 
> 
> Diffs
> -----
> 
>   3rdparty/libprocess/src/tests/benchmarks.cpp 604122a2f33d7abfab92631d9eac33b6753e846b 
> 
> 
> Diff: https://reviews.apache.org/r/67877/diff/3/
> 
> 
> Testing
> -------
> 
> Before https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 168443ns
> Snapshot of 1 counters in 339047ns
> HTTP /snapshot of 1 counters in 1.999797ms
> Removed 1 counters in 94595ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (3 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.542703ms
> Snapshot of 100 counters in 1.119107ms
> HTTP /snapshot of 100 counters in 500.982997ms
> Removed 100 counters in 3.949413ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (510 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 27.697086ms
> Snapshot of 1000 counters in 9.232065ms
> HTTP /snapshot of 1000 counters in 473.66825ms
> Removed 1000 counters in 31.632763ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (544 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 277.567781ms
> Snapshot of 10000 counters in 81.45371ms
> HTTP /snapshot of 10000 counters in 219.1157ms
> Removed 10000 counters in 309.334783ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (902 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.753352484secs
> Snapshot of 100000 counters in 1.033072727secs
> HTTP /snapshot of 100000 counters in 2.226777961secs
> Removed 100000 counters in 4.851697589secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (11996 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (13955 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (13955 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> After https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 102994ns
> Snapshot of 1 counters in 266999ns
> HTTP /snapshot of 1 counters in 1.83628ms
> Removed 1 counters in 27402ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (2 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.049965ms
> Snapshot of 100 counters in 930319ns
> HTTP /snapshot of 100 counters in 501.558737ms
> Removed 100 counters in 4.996015ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (511 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 28.623101ms
> Snapshot of 1000 counters in 7.496592ms
> HTTP /snapshot of 1000 counters in 470.870567ms
> Removed 1000 counters in 39.663126ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (548 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 335.506147ms
> Snapshot of 10000 counters in 67.963256ms
> HTTP /snapshot of 10000 counters in 154.661098ms
> Removed 10000 counters in 333.06148ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (904 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.396691566secs
> Snapshot of 100000 counters in 640.134635ms
> HTTP /snapshot of 100000 counters in 1.404702464secs
> Removed 100000 counters in 3.526395358secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (9119 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (11084 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (11084 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>

Re: Review Request 67877: Added a metrics scalability benchmark in libprocess.

Posted by Greg Mann <gr...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67877/#review206458
-----------------------------------------------------------


Fix it, then Ship it!





3rdparty/libprocess/src/tests/benchmarks.cpp
Lines 695-703 (patched)
<https://reviews.apache.org/r/67877/#comment289422>

    Nit: two newlines before/after the test case instantiation?



3rdparty/libprocess/src/tests/benchmarks.cpp
Lines 704 (patched)
<https://reviews.apache.org/r/67877/#comment289423>

    s/are/is/



3rdparty/libprocess/src/tests/benchmarks.cpp
Lines 707 (patched)
<https://reviews.apache.org/r/67877/#comment289424>

    Nit: seems like we use a mix of snake case and camel case in this file, but camel case is a bit more common? I'm fine either way here, but wanted to bring it up.


- Greg Mann


On July 11, 2018, 10:51 p.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67877/
> -----------------------------------------------------------
> 
> (Updated July 11, 2018, 10:51 p.m.)
> 
> 
> Review request for mesos and Greg Mann.
> 
> 
> Bugs: MESOS-9068
>     https://issues.apache.org/jira/browse/MESOS-9068
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This initial benchmark focuses on adding a large number of counters
> and measuring how long the add/remove and snapshot takes.
> 
> 
> Diffs
> -----
> 
>   3rdparty/libprocess/src/tests/benchmarks.cpp 604122a2f33d7abfab92631d9eac33b6753e846b 
> 
> 
> Diff: https://reviews.apache.org/r/67877/diff/3/
> 
> 
> Testing
> -------
> 
> Before https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 168443ns
> Snapshot of 1 counters in 339047ns
> HTTP /snapshot of 1 counters in 1.999797ms
> Removed 1 counters in 94595ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (3 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.542703ms
> Snapshot of 100 counters in 1.119107ms
> HTTP /snapshot of 100 counters in 500.982997ms
> Removed 100 counters in 3.949413ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (510 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 27.697086ms
> Snapshot of 1000 counters in 9.232065ms
> HTTP /snapshot of 1000 counters in 473.66825ms
> Removed 1000 counters in 31.632763ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (544 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 277.567781ms
> Snapshot of 10000 counters in 81.45371ms
> HTTP /snapshot of 10000 counters in 219.1157ms
> Removed 10000 counters in 309.334783ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (902 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.753352484secs
> Snapshot of 100000 counters in 1.033072727secs
> HTTP /snapshot of 100000 counters in 2.226777961secs
> Removed 100000 counters in 4.851697589secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (11996 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (13955 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (13955 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> After https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 102994ns
> Snapshot of 1 counters in 266999ns
> HTTP /snapshot of 1 counters in 1.83628ms
> Removed 1 counters in 27402ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (2 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.049965ms
> Snapshot of 100 counters in 930319ns
> HTTP /snapshot of 100 counters in 501.558737ms
> Removed 100 counters in 4.996015ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (511 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 28.623101ms
> Snapshot of 1000 counters in 7.496592ms
> HTTP /snapshot of 1000 counters in 470.870567ms
> Removed 1000 counters in 39.663126ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (548 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 335.506147ms
> Snapshot of 10000 counters in 67.963256ms
> HTTP /snapshot of 10000 counters in 154.661098ms
> Removed 10000 counters in 333.06148ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (904 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.396691566secs
> Snapshot of 100000 counters in 640.134635ms
> HTTP /snapshot of 100000 counters in 1.404702464secs
> Removed 100000 counters in 3.526395358secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (9119 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (11084 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (11084 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>

Re: Review Request 67877: Added a metrics scalability benchmark in libprocess.

Posted by Mesos Reviewbot Windows <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67877/#review205987
-----------------------------------------------------------



PASS: Mesos patch 67877 was successfully built and tested.

Reviews applied: `['67877']`

All the build artifacts available at: http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/1902/mesos-review-67877

- Mesos Reviewbot Windows


On July 11, 2018, 10:51 p.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67877/
> -----------------------------------------------------------
> 
> (Updated July 11, 2018, 10:51 p.m.)
> 
> 
> Review request for mesos and Greg Mann.
> 
> 
> Bugs: MESOS-9068
>     https://issues.apache.org/jira/browse/MESOS-9068
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This initial benchmark focuses on adding a large number of counters
> and measuring how long the add/remove and snapshot takes.
> 
> 
> Diffs
> -----
> 
>   3rdparty/libprocess/src/tests/benchmarks.cpp 604122a2f33d7abfab92631d9eac33b6753e846b 
> 
> 
> Diff: https://reviews.apache.org/r/67877/diff/3/
> 
> 
> Testing
> -------
> 
> Before https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 168443ns
> Snapshot of 1 counters in 339047ns
> HTTP /snapshot of 1 counters in 1.999797ms
> Removed 1 counters in 94595ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (3 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.542703ms
> Snapshot of 100 counters in 1.119107ms
> HTTP /snapshot of 100 counters in 500.982997ms
> Removed 100 counters in 3.949413ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (510 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 27.697086ms
> Snapshot of 1000 counters in 9.232065ms
> HTTP /snapshot of 1000 counters in 473.66825ms
> Removed 1000 counters in 31.632763ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (544 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 277.567781ms
> Snapshot of 10000 counters in 81.45371ms
> HTTP /snapshot of 10000 counters in 219.1157ms
> Removed 10000 counters in 309.334783ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (902 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.753352484secs
> Snapshot of 100000 counters in 1.033072727secs
> HTTP /snapshot of 100000 counters in 2.226777961secs
> Removed 100000 counters in 4.851697589secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (11996 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (13955 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (13955 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> After https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 102994ns
> Snapshot of 1 counters in 266999ns
> HTTP /snapshot of 1 counters in 1.83628ms
> Removed 1 counters in 27402ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (2 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.049965ms
> Snapshot of 100 counters in 930319ns
> HTTP /snapshot of 100 counters in 501.558737ms
> Removed 100 counters in 4.996015ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (511 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 28.623101ms
> Snapshot of 1000 counters in 7.496592ms
> HTTP /snapshot of 1000 counters in 470.870567ms
> Removed 1000 counters in 39.663126ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (548 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 335.506147ms
> Snapshot of 10000 counters in 67.963256ms
> HTTP /snapshot of 10000 counters in 154.661098ms
> Removed 10000 counters in 333.06148ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (904 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.396691566secs
> Snapshot of 100000 counters in 640.134635ms
> HTTP /snapshot of 100000 counters in 1.404702464secs
> Removed 100000 counters in 3.526395358secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (9119 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (11084 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (11084 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>

Re: Review Request 67877: Added a metrics scalability benchmark in libprocess.

Posted by Benjamin Mahler <bm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67877/
-----------------------------------------------------------

(Updated July 11, 2018, 10:51 p.m.)


Review request for mesos and Greg Mann.


Changes
-------

Avoid AWAIT timeouts.


Bugs: MESOS-9068
    https://issues.apache.org/jira/browse/MESOS-9068


Repository: mesos


Description
-------

This initial benchmark focuses on adding a large number of counters
and measuring how long the add/remove and snapshot takes.


Diffs (updated)
-----

  3rdparty/libprocess/src/tests/benchmarks.cpp 604122a2f33d7abfab92631d9eac33b6753e846b 


Diff: https://reviews.apache.org/r/67877/diff/3/

Changes: https://reviews.apache.org/r/67877/diff/2-3/


Testing
-------

Before https://reviews.apache.org/r/67871/:

```
[==========] Running 5 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
Added 1 counters in 168443ns
Snapshot of 1 counters in 339047ns
HTTP /snapshot of 1 counters in 1.999797ms
Removed 1 counters in 94595ns
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (3 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
Added 100 counters in 3.542703ms
Snapshot of 100 counters in 1.119107ms
HTTP /snapshot of 100 counters in 500.982997ms
Removed 100 counters in 3.949413ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (510 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
Added 1000 counters in 27.697086ms
Snapshot of 1000 counters in 9.232065ms
HTTP /snapshot of 1000 counters in 473.66825ms
Removed 1000 counters in 31.632763ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (544 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
Added 10000 counters in 277.567781ms
Snapshot of 10000 counters in 81.45371ms
HTTP /snapshot of 10000 counters in 219.1157ms
Removed 10000 counters in 309.334783ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (902 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
Added 100000 counters in 3.753352484secs
Snapshot of 100000 counters in 1.033072727secs
HTTP /snapshot of 100000 counters in 2.226777961secs
Removed 100000 counters in 4.851697589secs
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (11996 ms)
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (13955 ms total)

[----------] Global test environment tear-down
[==========] 5 tests from 1 test case ran. (13955 ms total)
[  PASSED  ] 5 tests.
```

After https://reviews.apache.org/r/67871/:

```
[==========] Running 5 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
Added 1 counters in 102994ns
Snapshot of 1 counters in 266999ns
HTTP /snapshot of 1 counters in 1.83628ms
Removed 1 counters in 27402ns
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (2 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
Added 100 counters in 3.049965ms
Snapshot of 100 counters in 930319ns
HTTP /snapshot of 100 counters in 501.558737ms
Removed 100 counters in 4.996015ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (511 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
Added 1000 counters in 28.623101ms
Snapshot of 1000 counters in 7.496592ms
HTTP /snapshot of 1000 counters in 470.870567ms
Removed 1000 counters in 39.663126ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (548 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
Added 10000 counters in 335.506147ms
Snapshot of 10000 counters in 67.963256ms
HTTP /snapshot of 10000 counters in 154.661098ms
Removed 10000 counters in 333.06148ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (904 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
Added 100000 counters in 3.396691566secs
Snapshot of 100000 counters in 640.134635ms
HTTP /snapshot of 100000 counters in 1.404702464secs
Removed 100000 counters in 3.526395358secs
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (9119 ms)
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (11084 ms total)

[----------] Global test environment tear-down
[==========] 5 tests from 1 test case ran. (11084 ms total)
[  PASSED  ] 5 tests.
```


Thanks,

Benjamin Mahler

Re: Review Request 67877: Added a metrics scalability benchmark in libprocess.

Posted by Benjamin Mahler <bm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67877/
-----------------------------------------------------------

(Updated July 11, 2018, 10:32 p.m.)


Review request for mesos and Greg Mann.


Changes
-------

Moved to benchmarks.cpp to follow existing convention.


Bugs: MESOS-9068
    https://issues.apache.org/jira/browse/MESOS-9068


Repository: mesos


Description
-------

This initial benchmark focuses on adding a large number of counters
and measuring how long the add/remove and snapshot takes.


Diffs (updated)
-----

  3rdparty/libprocess/src/tests/benchmarks.cpp 604122a2f33d7abfab92631d9eac33b6753e846b 


Diff: https://reviews.apache.org/r/67877/diff/2/

Changes: https://reviews.apache.org/r/67877/diff/1-2/


Testing
-------

Before https://reviews.apache.org/r/67871/:

```
[==========] Running 5 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
Added 1 counters in 168443ns
Snapshot of 1 counters in 339047ns
HTTP /snapshot of 1 counters in 1.999797ms
Removed 1 counters in 94595ns
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (3 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
Added 100 counters in 3.542703ms
Snapshot of 100 counters in 1.119107ms
HTTP /snapshot of 100 counters in 500.982997ms
Removed 100 counters in 3.949413ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (510 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
Added 1000 counters in 27.697086ms
Snapshot of 1000 counters in 9.232065ms
HTTP /snapshot of 1000 counters in 473.66825ms
Removed 1000 counters in 31.632763ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (544 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
Added 10000 counters in 277.567781ms
Snapshot of 10000 counters in 81.45371ms
HTTP /snapshot of 10000 counters in 219.1157ms
Removed 10000 counters in 309.334783ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (902 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
Added 100000 counters in 3.753352484secs
Snapshot of 100000 counters in 1.033072727secs
HTTP /snapshot of 100000 counters in 2.226777961secs
Removed 100000 counters in 4.851697589secs
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (11996 ms)
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (13955 ms total)

[----------] Global test environment tear-down
[==========] 5 tests from 1 test case ran. (13955 ms total)
[  PASSED  ] 5 tests.
```

After https://reviews.apache.org/r/67871/:

```
[==========] Running 5 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
Added 1 counters in 102994ns
Snapshot of 1 counters in 266999ns
HTTP /snapshot of 1 counters in 1.83628ms
Removed 1 counters in 27402ns
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (2 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
Added 100 counters in 3.049965ms
Snapshot of 100 counters in 930319ns
HTTP /snapshot of 100 counters in 501.558737ms
Removed 100 counters in 4.996015ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (511 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
Added 1000 counters in 28.623101ms
Snapshot of 1000 counters in 7.496592ms
HTTP /snapshot of 1000 counters in 470.870567ms
Removed 1000 counters in 39.663126ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (548 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
Added 10000 counters in 335.506147ms
Snapshot of 10000 counters in 67.963256ms
HTTP /snapshot of 10000 counters in 154.661098ms
Removed 10000 counters in 333.06148ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (904 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
Added 100000 counters in 3.396691566secs
Snapshot of 100000 counters in 640.134635ms
HTTP /snapshot of 100000 counters in 1.404702464secs
Removed 100000 counters in 3.526395358secs
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (9119 ms)
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (11084 ms total)

[----------] Global test environment tear-down
[==========] 5 tests from 1 test case ran. (11084 ms total)
[  PASSED  ] 5 tests.
```


Thanks,

Benjamin Mahler

Re: Review Request 67877: Added a metrics scalability benchmark in libprocess.

Posted by Benjamin Mahler <bm...@apache.org>.

> On July 10, 2018, 9:56 p.m., Mesos Reviewbot Windows wrote:
> > FAIL: Some of the unit tests failed. Please check the relevant logs.
> > 
> > Reviews applied: `['67877']`
> > 
> > Failed command: `Start-MesosCITesting`
> > 
> > All the build artifacts available at: http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/1897/mesos-review-67877
> > 
> > Relevant logs:
> > 
> > - [libprocess-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/1897/mesos-review-67877/logs/libprocess-tests-stdout.log):
> > 
> > ```
> > Added 1000 counters in 296.9324ms
> > Snapshot of 1000 counters in 100.9292ms
> > HTTP /snapshot of 1000 counters in 168.9975ms
> > Removed 1000 counters in 331.9743ms
> > [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (969 ms)
> > [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> > Added 10000 counters in 2.629906secs
> > Snapshot of 10000 counters in 1.0159752secs
> > HTTP /snapshot of 10000 counters in 1.654987secs
> > Removed 10000 counters in 3.1659297secs
> > [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (8635 ms)
> > [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> > Added 100000 counters in 26.8674348secs
> > Snapshot of 100000 counters in 10.963786secs
> > d:\dcos\mesos\mesos\3rdparty\libprocess\src\tests\metrics_tests.cpp(677): error: Failed to wait 15secs for http::get(upid, "snapshot")
> > HTTP /snapshot of 100000 counters in 15.0038049secs
> > Removed 100000 counters in 33.9055092secs
> > [  FAILED  ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4, where GetParam() = 100000 (88327 ms)
> > [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (98498 ms total)
> > 
> > [----------] Global test environment tear-down
> > [==========] 211 tests from 34 test cases ran. (101009 ms total)
> > [  PASSED  ] 210 tests.
> > [  FAILED  ] 1 test, listed below:
> > [  FAILED  ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4, where GetParam() = 100000
> > 
> >  1 FAILED TEST
> >   YOU HAVE 14 DISABLED TESTS
> > 
> > ```
> > 
> > - [libprocess-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/1897/mesos-review-67877/logs/libprocess-tests-stderr.log):
> > 
> > ```
> > WARNING: Logging before InitGoogleLogging() is written to STDERR
> > I0710 21:19:21.486577   212 process.cpp:926] Stopped the socket accept loop
> > ```

Hm.. I guess libprocess' benchmark pattern is actually to put all benchmarks in benchmarks.cpp for now which has a separate main. That doesn't seem too scalable but I'll think about whether to move this there or keep it in metrics_tests.cpp but add similar benchmark filtering to what we do in mesos tests.


- Benjamin


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67877/#review205941
-----------------------------------------------------------


On July 10, 2018, 8:51 p.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67877/
> -----------------------------------------------------------
> 
> (Updated July 10, 2018, 8:51 p.m.)
> 
> 
> Review request for mesos and Greg Mann.
> 
> 
> Bugs: MESOS-9068
>     https://issues.apache.org/jira/browse/MESOS-9068
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This initial benchmark focuses on adding a large number of counters
> and measuring how long the add/remove and snapshot takes.
> 
> 
> Diffs
> -----
> 
>   3rdparty/libprocess/src/tests/metrics_tests.cpp a5b41ed3f4491244656222681f443e7ed55393c2 
> 
> 
> Diff: https://reviews.apache.org/r/67877/diff/1/
> 
> 
> Testing
> -------
> 
> Before https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 168443ns
> Snapshot of 1 counters in 339047ns
> HTTP /snapshot of 1 counters in 1.999797ms
> Removed 1 counters in 94595ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (3 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.542703ms
> Snapshot of 100 counters in 1.119107ms
> HTTP /snapshot of 100 counters in 500.982997ms
> Removed 100 counters in 3.949413ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (510 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 27.697086ms
> Snapshot of 1000 counters in 9.232065ms
> HTTP /snapshot of 1000 counters in 473.66825ms
> Removed 1000 counters in 31.632763ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (544 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 277.567781ms
> Snapshot of 10000 counters in 81.45371ms
> HTTP /snapshot of 10000 counters in 219.1157ms
> Removed 10000 counters in 309.334783ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (902 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.753352484secs
> Snapshot of 100000 counters in 1.033072727secs
> HTTP /snapshot of 100000 counters in 2.226777961secs
> Removed 100000 counters in 4.851697589secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (11996 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (13955 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (13955 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> After https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 102994ns
> Snapshot of 1 counters in 266999ns
> HTTP /snapshot of 1 counters in 1.83628ms
> Removed 1 counters in 27402ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (2 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.049965ms
> Snapshot of 100 counters in 930319ns
> HTTP /snapshot of 100 counters in 501.558737ms
> Removed 100 counters in 4.996015ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (511 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 28.623101ms
> Snapshot of 1000 counters in 7.496592ms
> HTTP /snapshot of 1000 counters in 470.870567ms
> Removed 1000 counters in 39.663126ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (548 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 335.506147ms
> Snapshot of 10000 counters in 67.963256ms
> HTTP /snapshot of 10000 counters in 154.661098ms
> Removed 10000 counters in 333.06148ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (904 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.396691566secs
> Snapshot of 100000 counters in 640.134635ms
> HTTP /snapshot of 100000 counters in 1.404702464secs
> Removed 100000 counters in 3.526395358secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (9119 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (11084 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (11084 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>

Re: Review Request 67877: Added a metrics scalability benchmark in libprocess.

Posted by Mesos Reviewbot Windows <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67877/#review205941
-----------------------------------------------------------



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67877']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/1897/mesos-review-67877

Relevant logs:

- [libprocess-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/1897/mesos-review-67877/logs/libprocess-tests-stdout.log):

```
Added 1000 counters in 296.9324ms
Snapshot of 1000 counters in 100.9292ms
HTTP /snapshot of 1000 counters in 168.9975ms
Removed 1000 counters in 331.9743ms
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (969 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
Added 10000 counters in 2.629906secs
Snapshot of 10000 counters in 1.0159752secs
HTTP /snapshot of 10000 counters in 1.654987secs
Removed 10000 counters in 3.1659297secs
[       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (8635 ms)
[ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
Added 100000 counters in 26.8674348secs
Snapshot of 100000 counters in 10.963786secs
d:\dcos\mesos\mesos\3rdparty\libprocess\src\tests\metrics_tests.cpp(677): error: Failed to wait 15secs for http::get(upid, "snapshot")
HTTP /snapshot of 100000 counters in 15.0038049secs
Removed 100000 counters in 33.9055092secs
[  FAILED  ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4, where GetParam() = 100000 (88327 ms)
[----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (98498 ms total)

[----------] Global test environment tear-down
[==========] 211 tests from 34 test cases ran. (101009 ms total)
[  PASSED  ] 210 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4, where GetParam() = 100000

 1 FAILED TEST
  YOU HAVE 14 DISABLED TESTS

```

- [libprocess-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/1897/mesos-review-67877/logs/libprocess-tests-stderr.log):

```
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0710 21:19:21.486577   212 process.cpp:926] Stopped the socket accept loop
```

- Mesos Reviewbot Windows


On July 10, 2018, 8:51 p.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67877/
> -----------------------------------------------------------
> 
> (Updated July 10, 2018, 8:51 p.m.)
> 
> 
> Review request for mesos and Greg Mann.
> 
> 
> Bugs: MESOS-9068
>     https://issues.apache.org/jira/browse/MESOS-9068
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This initial benchmark focuses on adding a large number of counters
> and measuring how long the add/remove and snapshot takes.
> 
> 
> Diffs
> -----
> 
>   3rdparty/libprocess/src/tests/metrics_tests.cpp a5b41ed3f4491244656222681f443e7ed55393c2 
> 
> 
> Diff: https://reviews.apache.org/r/67877/diff/1/
> 
> 
> Testing
> -------
> 
> Before https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 168443ns
> Snapshot of 1 counters in 339047ns
> HTTP /snapshot of 1 counters in 1.999797ms
> Removed 1 counters in 94595ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (3 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.542703ms
> Snapshot of 100 counters in 1.119107ms
> HTTP /snapshot of 100 counters in 500.982997ms
> Removed 100 counters in 3.949413ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (510 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 27.697086ms
> Snapshot of 1000 counters in 9.232065ms
> HTTP /snapshot of 1000 counters in 473.66825ms
> Removed 1000 counters in 31.632763ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (544 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 277.567781ms
> Snapshot of 10000 counters in 81.45371ms
> HTTP /snapshot of 10000 counters in 219.1157ms
> Removed 10000 counters in 309.334783ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (902 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.753352484secs
> Snapshot of 100000 counters in 1.033072727secs
> HTTP /snapshot of 100000 counters in 2.226777961secs
> Removed 100000 counters in 4.851697589secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (11996 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (13955 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (13955 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> After https://reviews.apache.org/r/67871/:
> 
> ```
> [==========] Running 5 tests from 1 test case.
> [----------] Global test environment set-up.
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0
> Added 1 counters in 102994ns
> Snapshot of 1 counters in 266999ns
> HTTP /snapshot of 1 counters in 1.83628ms
> Removed 1 counters in 27402ns
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/0 (2 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1
> Added 100 counters in 3.049965ms
> Snapshot of 100 counters in 930319ns
> HTTP /snapshot of 100 counters in 501.558737ms
> Removed 100 counters in 4.996015ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/1 (511 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2
> Added 1000 counters in 28.623101ms
> Snapshot of 1000 counters in 7.496592ms
> HTTP /snapshot of 1000 counters in 470.870567ms
> Removed 1000 counters in 39.663126ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/2 (548 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3
> Added 10000 counters in 335.506147ms
> Snapshot of 10000 counters in 67.963256ms
> HTTP /snapshot of 10000 counters in 154.661098ms
> Removed 10000 counters in 333.06148ms
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/3 (904 ms)
> [ RUN      ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4
> Added 100000 counters in 3.396691566secs
> Snapshot of 100000 counters in 640.134635ms
> HTTP /snapshot of 100000 counters in 1.404702464secs
> Removed 100000 counters in 3.526395358secs
> [       OK ] MetricsCount/Metrics_BENCHMARK_Test.Scalability/4 (9119 ms)
> [----------] 5 tests from MetricsCount/Metrics_BENCHMARK_Test (11084 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 5 tests from 1 test case ran. (11084 ms total)
> [  PASSED  ] 5 tests.
> ```
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>