[kudu-CR] KUDU-1728 parallelize download blocks in tablet-copy-client

["wangning (Code Review)" <ge...@cloudera.org>]

Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Bankim Bhavsar, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16274

to look at the new patch set (#7).

Change subject: KUDU-1728 parallelize download blocks in tablet-copy-client
......................................................................

KUDU-1728 parallelize download blocks in tablet-copy-client

Parallelize the action of 'Download blocks' in tablet-copy-client.

Previsouly downloading blocks from tablet server is executed sequentially,
thus actions which related to 'DownloadBlocks' like
'recover from other tserver', 'cluster rebalance' may be slow sometimes.
And sometimes downloading the blocks is slow when only one thread is used
while bandwidth isn't the bottleneck.

This commit introduce FLAGS_num_threads_blocks_download to control the
number of threads for download blocks within a tablet-copy-client.

Here I attached the simple benchmark, the result are averaged by 8 epoch
result. Metrics of result is seconds with accuracy of 2 decimal points.

The experiment is download all data from remote tserver to local tserver.

Settings:
Remote machine: 8 cores 32g tserver limited to 8g, tserver version 1.10
Local machine: 8 cores 12g memory
Tserver has 3 x 500M tablets + 3 x 430M tablets + 3 x 420M tablets and
259 x 8M tablets.
All tablets were compacted well with diskrowset height 1.0.

Here I use two variable to controll the experiment.
numbers of tablet-copy-client thread, write as tc.
numbers of blocks-download thread within each tablet-copy-client thread,
(FLAGS_num_threads_blocks_download), write as bd.

The result 37.58 correspond to column 'tc-2', row 'bd-4' can be explained as,
It takes 37.58s to download all data from tserver with 2
tablet-copy-client threads and each tablet-copy-client thread has 4 threads to
download blocks.

All result in column tc-1 refer to behave before this patch.

time     tc-1      tc-2      tc-4      tc-8

bd-1     76.91     48.50     34.27     32.39
bd-2     54.45     43.26     33.27     32.37
bd-4     48.36     37.58     33.31     32.30
bd-8     47.95     38.36     33.98     32.60

Change-Id: Id83abca7a38cf183d9c27d82bb8a022699079e0e
---
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_client.cc
M src/kudu/tserver/tablet_copy_client.h
3 files changed, 183 insertions(+), 63 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/74/16274/7
-- 
To view, visit http://gerrit.cloudera.org:8080/16274
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id83abca7a38cf183d9c27d82bb8a022699079e0e
Gerrit-Change-Number: 16274
Gerrit-PatchSet: 7
Gerrit-Owner: wangning <19...@gmail.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Bankim Bhavsar <ba...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: wangning <19...@gmail.com>