[kudu-CR] KUDU-1587 part 2: reject write ops if apply queue is overloaded

["Alexey Serbin (Code Review)" <ge...@cloudera.org>]

Hello Kudu Jenkins, Andrew Wong, Todd Lipcon, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16343

to look at the new patch set (#2).

Change subject: KUDU-1587 part 2: reject write ops if apply queue is overloaded
......................................................................

KUDU-1587 part 2: reject write ops if apply queue is overloaded

This patch implements control admission for write requests in tablet
servers based on the load status of their apply queue. With this change,
the recently introduced OpApplyQueueTest.ApplyQueueBackpressure scenario
successfully passes.

By default, the overload threshold for the tablet server's apply queue
is set to 500 milliseconds. If the queue times of the tasks in the apply
queue become higher than the specified threshold, the apply queue enters
overloaded state.  When the queue is overloaded, the tablet server
rejects incoming write requests with some probability.  The longer
the queue stays overloaded, the greater the probability of rejections.
In addition, the more row operations a write request has, the greater
the probability of such request to be the rejected.  The apply queue
exits the overloaded state when queue times drop below the specified
threshold.

This new behavior is now enabled by default. To disable it and return
to the legacy one, set --tablet_apply_pool_overload_threshold_ms to 0.

Change-Id: I6d7688d6fa832e606b8efc4549568fa52dfa1931
---
M src/kudu/kserver/kserver.cc
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/util/threadpool-test.cc
M src/kudu/util/threadpool.cc
M src/kudu/util/threadpool.h
7 files changed, 352 insertions(+), 299 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/43/16343/2
-- 
To view, visit http://gerrit.cloudera.org:8080/16343
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6d7688d6fa832e606b8efc4549568fa52dfa1931
Gerrit-Change-Number: 16343
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>