[kudu-CR] Add per tablet write RPC throttling

["Binglin Chang (Code Review)" <ge...@cloudera.org>]

Hello Kudu Jenkins,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/2327

to look at the new patch set (#2).

Change subject: Add per tablet write RPC throttling
......................................................................

Add per tablet write RPC throttling

This commit adds per tablet write RPC throttling, a simple way to
provide some degree of isolation in multi-user environment.

Token bucket is used for throttling, and three config options are
added to specify throttling behavior:

  1. tablet_throttler_rpc_per_sec
     Maximum write RPC allowed per second,
     default 0, which means no limit.
  2. tablet_throttler_bytes_per_sec
     Maximum write RPC request bytes allowed per second,
     defualt 0, which means no limit.
  3. tablet_throttler_burst_factor
     Burst factor, user can accumulate tokens and use them in a
     short time period to archive burst throughput.

By default throttling is not enabled.

The assumptions are:
  1. Data ingestion rates(write RPC rates) are mostly stable,
     e.g. log and time series.
  2. We reserve a stable share of resoures for data ingestion per
     TS, e.g. 20000 writes per TS per sec.
  3. Each TS can hold roughly same number of tablets/leaders,
     e.g. 120 tablets and 40 leaders. We would like write RPC spread
     evenly to those 40 leaders, each have 20000 / 40 = 500 capacity.

This capacity also become a simple indicator for user and cluster
admin to do provision and planning:

  A workload require 200K op/s write throughput, so the table should
  have 200K / 500 = 400 tablets.

  The cluster have capacity for 4000 tablets, and currently there are
  3000 tablets running, so we can only allow 1000 new tablets or more
  nodes is needed.

Change-Id: I33cb6934d27b883a783682cef1e0723100637d45
---
M src/kudu/integration-tests/CMakeLists.txt
A src/kudu/integration-tests/write_throttling-itest.cc
M src/kudu/tablet/CMakeLists.txt
M src/kudu/tablet/tablet.cc
M src/kudu/tablet/tablet.h
A src/kudu/tablet/tablet_throttle-test.cc
M src/kudu/tserver/tablet_service.cc
M src/kudu/util/CMakeLists.txt
A src/kudu/util/throttler-test.cc
A src/kudu/util/throttler.h
10 files changed, 452 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/2327/2
-- 
To view, visit http://gerrit.cloudera.org:8080/2327
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I33cb6934d27b883a783682cef1e0723100637d45
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Binglin Chang <de...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>