You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2020/07/24 17:59:41 UTC

[kudu-CR] table locations-itest: reuse protobufs in the benchmark client

Hello Alexey Serbin,

I'd like you to do a code review. Please visit

    http://gerrit.cloudera.org:8080/16236

to review the following change.


Change subject: table_locations-itest: reuse protobufs in the benchmark client
......................................................................

table_locations-itest: reuse protobufs in the benchmark client

The previous client code would allocate a new request/response protobuf
for each call, which, while maybe realistic from a client perspective,
ended up causing a lot of allocator contention which also slowed down
the server's ability to allocate, making for a less realistic
benchmark/stress test of the server.

This patch simply reuses the protobufs to reduce allocator pressure,
speeding up the benchmark significantly. Benchmarked with:

$ KUDU_ALLOW_SLOW_TESTS=1  ./build/latest/bin/table_locations-itest \
  --gtest_filter=TableLocationsTest.GetTableLocationsBenchmark \
  --rpc_num_service_threads=32 \
  --benchmark_num_threads=48

RPC benchmark:
==================
before:
  RPC queue overflows: 0
  GetTableLocations RPC: 43240.8 req/sec
  Stats on GetTableLocations RPC (times in microseconds): Count: 216204
  Mean: 308.099
  Percentiles:
     0%  (min) = 15
    25%        = 146
    50%  (med) = 243
    75%        = 392
    95%        = 744
    99%        = 1120
    99.9%      = 2080
    99.99%     = 4576
    100% (max) = 42197

after:
 GetTableLocations RPC: 57670 req/sec
 Stats on GetTableLocations RPC (times in microseconds): Count: 288350
 Mean: 248.583
 Percentiles:
    0%  (min) = 19
   25%        = 152
   50%  (med) = 208
   75%        = 296
   95%        = 520
   99%        = 788
   99.9%      = 1736
   99.99%     = 4320
   100% (max) = 8313

Function benchmark:
=====================
Before:
 GetTableLocations function call: 264899.2 req/sec

After:
 GetTableLocations function call: 327872.2 req/sec

Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
---
M src/kudu/integration-tests/table_locations-itest.cc
1 file changed, 8 insertions(+), 4 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/36/16236/1
-- 
To view, visit http://gerrit.cloudera.org:8080/16236
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
Gerrit-Change-Number: 16236
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>

[kudu-CR] table locations-itest: reuse protobufs in the benchmark client

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/16236 )

Change subject: table_locations-itest: reuse protobufs in the benchmark client
......................................................................


Patch Set 1: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/16236
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
Gerrit-Change-Number: 16236
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Fri, 24 Jul 2020 18:01:15 +0000
Gerrit-HasComments: No

[kudu-CR] table locations-itest: reuse protobufs in the benchmark client

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16236 )

Change subject: table_locations-itest: reuse protobufs in the benchmark client
......................................................................

table_locations-itest: reuse protobufs in the benchmark client

The previous client code would allocate a new request/response protobuf
for each call, which, while maybe realistic from a client perspective,
ended up causing a lot of allocator contention which also slowed down
the server's ability to allocate, making for a less realistic
benchmark/stress test of the server.

This patch simply reuses the protobufs to reduce allocator pressure,
speeding up the benchmark significantly. Benchmarked with:

$ KUDU_ALLOW_SLOW_TESTS=1  ./build/latest/bin/table_locations-itest \
  --gtest_filter=TableLocationsTest.GetTableLocationsBenchmark \
  --rpc_num_service_threads=32 \
  --benchmark_num_threads=48

RPC benchmark:
==================
before:
  RPC queue overflows: 0
  GetTableLocations RPC: 43240.8 req/sec
  Stats on GetTableLocations RPC (times in microseconds): Count: 216204
  Mean: 308.099
  Percentiles:
     0%  (min) = 15
    25%        = 146
    50%  (med) = 243
    75%        = 392
    95%        = 744
    99%        = 1120
    99.9%      = 2080
    99.99%     = 4576
    100% (max) = 42197

after:
 GetTableLocations RPC: 57670 req/sec
 Stats on GetTableLocations RPC (times in microseconds): Count: 288350
 Mean: 248.583
 Percentiles:
    0%  (min) = 19
   25%        = 152
   50%  (med) = 208
   75%        = 296
   95%        = 520
   99%        = 788
   99.9%      = 1736
   99.99%     = 4320
   100% (max) = 8313

Function benchmark:
=====================
Before:
 GetTableLocations function call: 264899.2 req/sec

After:
 GetTableLocations function call: 327872.2 req/sec

Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
Reviewed-on: http://gerrit.cloudera.org:8080/16236
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Tested-by: Kudu Jenkins
---
M src/kudu/integration-tests/table_locations-itest.cc
1 file changed, 8 insertions(+), 4 deletions(-)

Approvals:
  Alexey Serbin: Looks good to me, approved
  Kudu Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/16236
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
Gerrit-Change-Number: 16236
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>