You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2020/07/24 17:59:41 UTC
[kudu-CR] table locations-itest: reuse protobufs in the benchmark client
Hello Alexey Serbin,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/16236
to review the following change.
Change subject: table_locations-itest: reuse protobufs in the benchmark client
......................................................................
table_locations-itest: reuse protobufs in the benchmark client
The previous client code would allocate a new request/response protobuf
for each call, which, while maybe realistic from a client perspective,
ended up causing a lot of allocator contention which also slowed down
the server's ability to allocate, making for a less realistic
benchmark/stress test of the server.
This patch simply reuses the protobufs to reduce allocator pressure,
speeding up the benchmark significantly. Benchmarked with:
$ KUDU_ALLOW_SLOW_TESTS=1 ./build/latest/bin/table_locations-itest \
--gtest_filter=TableLocationsTest.GetTableLocationsBenchmark \
--rpc_num_service_threads=32 \
--benchmark_num_threads=48
RPC benchmark:
==================
before:
RPC queue overflows: 0
GetTableLocations RPC: 43240.8 req/sec
Stats on GetTableLocations RPC (times in microseconds): Count: 216204
Mean: 308.099
Percentiles:
0% (min) = 15
25% = 146
50% (med) = 243
75% = 392
95% = 744
99% = 1120
99.9% = 2080
99.99% = 4576
100% (max) = 42197
after:
GetTableLocations RPC: 57670 req/sec
Stats on GetTableLocations RPC (times in microseconds): Count: 288350
Mean: 248.583
Percentiles:
0% (min) = 19
25% = 152
50% (med) = 208
75% = 296
95% = 520
99% = 788
99.9% = 1736
99.99% = 4320
100% (max) = 8313
Function benchmark:
=====================
Before:
GetTableLocations function call: 264899.2 req/sec
After:
GetTableLocations function call: 327872.2 req/sec
Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
---
M src/kudu/integration-tests/table_locations-itest.cc
1 file changed, 8 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/36/16236/1
--
To view, visit http://gerrit.cloudera.org:8080/16236
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
Gerrit-Change-Number: 16236
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
[kudu-CR] table locations-itest: reuse protobufs in the benchmark client
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16236 )
Change subject: table_locations-itest: reuse protobufs in the benchmark client
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16236
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
Gerrit-Change-Number: 16236
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 24 Jul 2020 18:01:15 +0000
Gerrit-HasComments: No
[kudu-CR] table locations-itest: reuse protobufs in the benchmark client
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16236 )
Change subject: table_locations-itest: reuse protobufs in the benchmark client
......................................................................
table_locations-itest: reuse protobufs in the benchmark client
The previous client code would allocate a new request/response protobuf
for each call, which, while maybe realistic from a client perspective,
ended up causing a lot of allocator contention which also slowed down
the server's ability to allocate, making for a less realistic
benchmark/stress test of the server.
This patch simply reuses the protobufs to reduce allocator pressure,
speeding up the benchmark significantly. Benchmarked with:
$ KUDU_ALLOW_SLOW_TESTS=1 ./build/latest/bin/table_locations-itest \
--gtest_filter=TableLocationsTest.GetTableLocationsBenchmark \
--rpc_num_service_threads=32 \
--benchmark_num_threads=48
RPC benchmark:
==================
before:
RPC queue overflows: 0
GetTableLocations RPC: 43240.8 req/sec
Stats on GetTableLocations RPC (times in microseconds): Count: 216204
Mean: 308.099
Percentiles:
0% (min) = 15
25% = 146
50% (med) = 243
75% = 392
95% = 744
99% = 1120
99.9% = 2080
99.99% = 4576
100% (max) = 42197
after:
GetTableLocations RPC: 57670 req/sec
Stats on GetTableLocations RPC (times in microseconds): Count: 288350
Mean: 248.583
Percentiles:
0% (min) = 19
25% = 152
50% (med) = 208
75% = 296
95% = 520
99% = 788
99.9% = 1736
99.99% = 4320
100% (max) = 8313
Function benchmark:
=====================
Before:
GetTableLocations function call: 264899.2 req/sec
After:
GetTableLocations function call: 327872.2 req/sec
Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
Reviewed-on: http://gerrit.cloudera.org:8080/16236
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/integration-tests/table_locations-itest.cc
1 file changed, 8 insertions(+), 4 deletions(-)
Approvals:
Alexey Serbin: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/16236
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
Gerrit-Change-Number: 16236
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>