You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2016/05/03 20:56:37 UTC
[kudu-CR] rpc: reduce the number of recv syscalls
Hello Binglin Chang, Henry Robinson, Mike Percy,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/2943
to review the following change.
Change subject: rpc: reduce the number of recv syscalls
......................................................................
rpc: reduce the number of recv syscalls
This changes InboundTransfer to try to read an extra length header
following each call. If the header is available, the next
InboundTransfer is pre-seeded with that known length. This avoids an
extra recv() syscall for every length header in the case that the sender
has sent multiple RPCs in the span of a single event loop iteration.
This doesn't have an effect on the sync RPC benchmark because each
client only has a single RPC outstanding at a time. In the async
benchmark, though, it reduces the syscall count and system CPU time
substantially.
I measured with perf stat -e syscalls:sys_enter_recvfrom -I1000:
before:
[ RUN ] RpcBench.BenchmarkCallAsync
11.001645188 3,276,190 syscalls:sys_enter_recvfrom
12.001805098 3,342,820 syscalls:sys_enter_recvfrom
13.001961246 3,340,416 syscalls:sys_enter_recvfrom
14.002122984 3,346,229 syscalls:sys_enter_recvfrom
15.002286260 3,341,074 syscalls:sys_enter_recvfrom
16.002444642 3,334,740 syscalls:sys_enter_recvfrom
17.002604965 3,345,492 syscalls:sys_enter_recvfrom
18.002755159 3,343,448 syscalls:sys_enter_recvfrom
19.002907373 3,343,502 syscalls:sys_enter_recvfrom
20.003066162 3,345,778 syscalls:sys_enter_recvfrom
I0503 11:28:28.534939 17406 rpc-bench.cc:81] Mode: Async
I0503 11:28:28.534947 17406 rpc-bench.cc:85] Client reactors: 24
I0503 11:28:28.534950 17406 rpc-bench.cc:86] Call concurrency: 100
I0503 11:28:28.534952 17406 rpc-bench.cc:89] Worker threads: 40
I0503 11:28:28.534955 17406 rpc-bench.cc:90] Server reactors: 24
I0503 11:28:28.534956 17406 rpc-bench.cc:91] ----------------------------------
I0503 11:28:28.534958 17406 rpc-bench.cc:92] Reqs/sec: 835032
I0503 11:28:28.534966 17406 rpc-bench.cc:93] User CPU per req: 24.8712us
I0503 11:28:28.534970 17406 rpc-bench.cc:94] Sys CPU per req: 20.057us
I0503 11:28:28.534973 17406 rpc-bench.cc:95] Ctx Sw. per req: 0.691264
after:
11.001644614 2,176,488 syscalls:sys_enter_recvfrom
12.001805542 2,207,104 syscalls:sys_enter_recvfrom
13.001960757 2,213,293 syscalls:sys_enter_recvfrom
14.002110852 2,212,539 syscalls:sys_enter_recvfrom
15.002265130 2,210,270 syscalls:sys_enter_recvfrom
16.002430423 2,218,725 syscalls:sys_enter_recvfrom
17.002577149 2,213,841 syscalls:sys_enter_recvfrom
18.002728841 2,214,574 syscalls:sys_enter_recvfrom
19.002876367 2,210,987 syscalls:sys_enter_recvfrom
20.002988496 2,210,339 syscalls:sys_enter_recvfrom
I0503 11:27:38.645000 16813 rpc-bench.cc:81] Mode: Async
I0503 11:27:38.645009 16813 rpc-bench.cc:85] Client reactors: 24
I0503 11:27:38.645012 16813 rpc-bench.cc:86] Call concurrency: 100
I0503 11:27:38.645015 16813 rpc-bench.cc:89] Worker threads: 40
I0503 11:27:38.645016 16813 rpc-bench.cc:90] Server reactors: 24
I0503 11:27:38.645018 16813 rpc-bench.cc:91] ----------------------------------
I0503 11:27:38.645020 16813 rpc-bench.cc:92] Reqs/sec: 834166
I0503 11:27:38.645028 16813 rpc-bench.cc:93] User CPU per req: 22.2478us
I0503 11:27:38.645032 16813 rpc-bench.cc:94] Sys CPU per req: 16.1264us
I0503 11:27:38.645035 16813 rpc-bench.cc:95] Ctx Sw. per req: 0.873741
The syscall count is reduced by about one third, with similar reduction
in CPU usage.
Change-Id: I2adbe51b51267deb43b837c8ffae09cf8f54d251
---
M src/kudu/rpc/connection.cc
M src/kudu/rpc/transfer.cc
M src/kudu/rpc/transfer.h
3 files changed, 43 insertions(+), 18 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/43/2943/1
--
To view, visit http://gerrit.cloudera.org:8080/2943
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2adbe51b51267deb43b837c8ffae09cf8f54d251
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Binglin Chang <de...@gmail.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>