You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Dmitry Zhuk <dz...@twopensource.com> on 2017/10/11 17:34:40 UTC
Review Request 62899: Added protobuf message passing benchmark.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62899/
-----------------------------------------------------------
Review request for mesos and Benjamin Mahler.
Bugs: MESOS-6971
https://issues.apache.org/jira/browse/MESOS-6971
Repository: mesos
Description
-------
This measures performance of delivering protobuf message from receiving
`MessageEvent` till calling handler. Used to evaluate performance gains
from using protobuf arenas.
Diffs
-----
3rdparty/libprocess/Makefile.am 6b2fdff375514da1285cc614c3504c4d0892e464
3rdparty/libprocess/src/tests/benchmarks.cpp 8195193fbcc1c22b72cd95b09ad17f7529d1952d
3rdparty/libprocess/src/tests/benchmarks.proto PRE-CREATION
Diff: https://reviews.apache.org/r/62899/diff/1/
Testing
-------
make check
./benchmarks --gtest_filter=ProcessTest.Process_BENCHMARK_MessagePassing
Thanks,
Dmitry Zhuk
Re: Review Request 62899: Added protobuf message passing benchmark.
Posted by Benjamin Mahler <bm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62899/#review187855
-----------------------------------------------------------
3rdparty/libprocess/src/tests/benchmarks.cpp
Lines 583 (patched)
<https://reviews.apache.org/r/62899/#comment264889>
How about `ProtobufInstallHandlerBenchmarkProcess`?
3rdparty/libprocess/src/tests/benchmarks.cpp
Lines 592-596 (patched)
<https://reviews.apache.org/r/62899/#comment264922>
Maybe a TODO for also benchmarking the version that takes individual fields, especially because of the copying induced by the RepeatedPtrField -> vector conversion.
3rdparty/libprocess/src/tests/benchmarks.cpp
Lines 598-621 (patched)
<https://reviews.apache.org/r/62899/#comment264926>
It would be nice to have a time-based benchmark instead of iteration based (e.g. see how many operations can be done in 1 second).
That way, we have a better handle on how long the benchmarks take. As it stands, the existing benchmarks take so long that it's impossible to run them in CI or run them all locally.
3rdparty/libprocess/src/tests/benchmarks.cpp
Lines 624-644 (patched)
<https://reviews.apache.org/r/62899/#comment264925>
Can you describe the structure of the messages being benchmarked? I'm still a little puzzled at what the overall message tree looks like here given the queue logic. :S
Maybe let's pass the parameters? E.g.
repeated_count, tree_depth?
3rdparty/libprocess/src/tests/benchmarks.cpp
Lines 648 (patched)
<https://reviews.apache.org/r/62899/#comment264890>
How about `Process_BENCHMARK_ProtobufInstallHandler`?
- Benjamin Mahler
On Oct. 12, 2017, 1:09 p.m., Dmitry Zhuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62899/
> -----------------------------------------------------------
>
> (Updated Oct. 12, 2017, 1:09 p.m.)
>
>
> Review request for mesos and Benjamin Mahler.
>
>
> Bugs: MESOS-6971
> https://issues.apache.org/jira/browse/MESOS-6971
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This measures performance of delivering protobuf message from receiving
> `MessageEvent` till calling handler. Used to evaluate performance gains
> from using protobuf arenas.
>
>
> Diffs
> -----
>
> 3rdparty/libprocess/Makefile.am 6b2fdff375514da1285cc614c3504c4d0892e464
> 3rdparty/libprocess/src/tests/benchmarks.cpp 8195193fbcc1c22b72cd95b09ad17f7529d1952d
> 3rdparty/libprocess/src/tests/benchmarks.proto PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/62899/diff/2/
>
>
> Testing
> -------
>
> make check
> ./benchmarks --gtest_filter=ProcessTest.Process_BENCHMARK_MessagePassing
>
>
> Thanks,
>
> Dmitry Zhuk
>
>
Re: Review Request 62899: Added protobuf message passing benchmark.
Posted by Benjamin Mahler <bm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62899/#review188225
-----------------------------------------------------------
Ship it!
Looks good! I made some tweaks to the code and added some comments to show the tree structure.
- Benjamin Mahler
On Oct. 13, 2017, 12:55 p.m., Dmitry Zhuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62899/
> -----------------------------------------------------------
>
> (Updated Oct. 13, 2017, 12:55 p.m.)
>
>
> Review request for mesos and Benjamin Mahler.
>
>
> Bugs: MESOS-6971
> https://issues.apache.org/jira/browse/MESOS-6971
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This measures performance of delivering protobuf message from receiving
> `MessageEvent` till calling handler. Used to evaluate performance gains
> from using protobuf arenas.
>
>
> Diffs
> -----
>
> 3rdparty/libprocess/Makefile.am 6b2fdff375514da1285cc614c3504c4d0892e464
> 3rdparty/libprocess/src/tests/benchmarks.cpp 8195193fbcc1c22b72cd95b09ad17f7529d1952d
> 3rdparty/libprocess/src/tests/benchmarks.proto PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/62899/diff/3/
>
>
> Testing
> -------
>
> make check
> ./benchmarks --gtest_filter=ProcessTest.Process_BENCHMARK_ProtobufInstallHandler
> [ RUN ] ProcessTest.Process_BENCHMARK_ProtobufInstallHandler
> Size: 4, throughput: 633872.07 messages/s
> Size: 10, throughput: 403461.32 messages/s
> Size: 34, throughput: 216181.66 messages/s
> Size: 64, throughput: 132159.29 messages/s
> Size: 154, throughput: 64097.07 messages/s
> Size: 304, throughput: 34928.53 messages/s
> Size: 605, throughput: 18249.50 messages/s
> Size: 1507, throughput: 7521.48 messages/s
> Size: 3011, throughput: 3765.40 messages/s
> Size: 4515, throughput: 2526.66 messages/s
> Size: 6019, throughput: 1889.99 messages/s
> Size: 9027, throughput: 1192.44 messages/s
> Size: 12034, throughput: 887.66 messages/s
> Size: 15042, throughput: 713.76 messages/s
> Size: 30081, throughput: 349.73 messages/s
> [ OK ] ProcessTest.Process_BENCHMARK_ProtobufInstallHandler (15937 ms)
>
>
> Thanks,
>
> Dmitry Zhuk
>
>
Re: Review Request 62899: Added protobuf message passing benchmark.
Posted by Dmitry Zhuk <dz...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62899/
-----------------------------------------------------------
(Updated Oct. 13, 2017, 12:55 p.m.)
Review request for mesos and Benjamin Mahler.
Bugs: MESOS-6971
https://issues.apache.org/jira/browse/MESOS-6971
Repository: mesos
Description
-------
This measures performance of delivering protobuf message from receiving
`MessageEvent` till calling handler. Used to evaluate performance gains
from using protobuf arenas.
Diffs (updated)
-----
3rdparty/libprocess/Makefile.am 6b2fdff375514da1285cc614c3504c4d0892e464
3rdparty/libprocess/src/tests/benchmarks.cpp 8195193fbcc1c22b72cd95b09ad17f7529d1952d
3rdparty/libprocess/src/tests/benchmarks.proto PRE-CREATION
Diff: https://reviews.apache.org/r/62899/diff/3/
Changes: https://reviews.apache.org/r/62899/diff/2-3/
Testing (updated)
-------
make check
./benchmarks --gtest_filter=ProcessTest.Process_BENCHMARK_ProtobufInstallHandler
[ RUN ] ProcessTest.Process_BENCHMARK_ProtobufInstallHandler
Size: 4, throughput: 633872.07 messages/s
Size: 10, throughput: 403461.32 messages/s
Size: 34, throughput: 216181.66 messages/s
Size: 64, throughput: 132159.29 messages/s
Size: 154, throughput: 64097.07 messages/s
Size: 304, throughput: 34928.53 messages/s
Size: 605, throughput: 18249.50 messages/s
Size: 1507, throughput: 7521.48 messages/s
Size: 3011, throughput: 3765.40 messages/s
Size: 4515, throughput: 2526.66 messages/s
Size: 6019, throughput: 1889.99 messages/s
Size: 9027, throughput: 1192.44 messages/s
Size: 12034, throughput: 887.66 messages/s
Size: 15042, throughput: 713.76 messages/s
Size: 30081, throughput: 349.73 messages/s
[ OK ] ProcessTest.Process_BENCHMARK_ProtobufInstallHandler (15937 ms)
Thanks,
Dmitry Zhuk
Re: Review Request 62899: Added protobuf message passing benchmark.
Posted by Dmitry Zhuk <dz...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62899/
-----------------------------------------------------------
(Updated Oct. 12, 2017, 1:09 p.m.)
Review request for mesos and Benjamin Mahler.
Changes
-------
Changed benchmark to cover more messages sizes.
Bugs: MESOS-6971
https://issues.apache.org/jira/browse/MESOS-6971
Repository: mesos
Description
-------
This measures performance of delivering protobuf message from receiving
`MessageEvent` till calling handler. Used to evaluate performance gains
from using protobuf arenas.
Diffs (updated)
-----
3rdparty/libprocess/Makefile.am 6b2fdff375514da1285cc614c3504c4d0892e464
3rdparty/libprocess/src/tests/benchmarks.cpp 8195193fbcc1c22b72cd95b09ad17f7529d1952d
3rdparty/libprocess/src/tests/benchmarks.proto PRE-CREATION
Diff: https://reviews.apache.org/r/62899/diff/2/
Changes: https://reviews.apache.org/r/62899/diff/1-2/
Testing
-------
make check
./benchmarks --gtest_filter=ProcessTest.Process_BENCHMARK_MessagePassing
Thanks,
Dmitry Zhuk