You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2016/05/03 20:56:37 UTC
[kudu-CR] rpc: LIFO service queue
Hello Binglin Chang, Henry Robinson, Mike Percy,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/2938
to review the following change.
Change subject: rpc: LIFO service queue
......................................................................
rpc: LIFO service queue
This adds a service queue implementation which differs from the previous
one in a few ways:
1) when the worker threads want to dequeue work and there is none available,
they push themselves onto a stack and wait for work on their own independent
condition variable rather than a central one. Because it uses a stack,
the wake-up behavior is LIFO rather than FIFO, which is preferable to
reduce context switches and increase cache locality (hot threads stay
hot).
2) when enqueueing a call, if there is already a waiting worker thread,
then the call is assigned directly to that thread without going
through the queue implementation. So, even if the queue
implementation is heavy-weight (such as our priority queue) it does
not affect the performance of workloads which have enough worker
threads allocated.
This improves rpc-bench throughput fairly substantially (~47% for sync,
66% for async):
./bin/rpc-bench -server_reactors 24 -async_call_concurrency 100
-worker_threads 40 -client_threads 24 -run_seconds 10
original:
------------------------
Mode: Sync
Client threads: 24
Worker threads: 40
Server reactors: 24
----------------------------------
Reqs/sec: 189006
User CPU per req: 21.9328us
Sys CPU per req: 31.633us
Ctx Sw. per req: 5.08355
Mode: Async
Client reactors: 24
Call concurrency: 100
Worker threads: 40
Server reactors: 24
----------------------------------
Reqs/sec: 202874
User CPU per req: 14.2749us
Sys CPU per req: 23.538us
Ctx Sw. per req: 2.7318
lifo queue without hash change:
-----------------------
Reqs/sec: 278730
User CPU per req: 19.8051us
Sys CPU per req: 21.9387us
Ctx Sw. per req: 4.04089
Mode: Async
Client reactors: 24
Call concurrency: 100
Worker threads: 40
Server reactors: 24
----------------------------------
Reqs/sec: 336000
User CPU per req: 11.0189us
Sys CPU per req: 13.6012us
Ctx Sw. per req: 1.48036
Change-Id: Ibd1eb677dd52f89683eb648b42918fcf51437215
---
M src/kudu/rpc/CMakeLists.txt
M src/kudu/rpc/mt-rpc-test.cc
M src/kudu/rpc/rpc-test-base.h
M src/kudu/rpc/service_pool.cc
M src/kudu/rpc/service_pool.h
A src/kudu/rpc/service_queue-test.cc
A src/kudu/rpc/service_queue.cc
M src/kudu/rpc/service_queue.h
M src/kudu/server/rpc_server.cc
9 files changed, 495 insertions(+), 19 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/38/2938/1
--
To view, visit http://gerrit.cloudera.org:8080/2938
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibd1eb677dd52f89683eb648b42918fcf51437215
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Binglin Chang <de...@gmail.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>