[kudu-CR] KUDU-2977 Sharding block map to speed up tserver startup

["Yingchun Lai (Code Review)" <ge...@cloudera.org>]

Hello Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14555

to look at the new patch set (#3).

Change subject: KUDU-2977 Sharding block map to speed up tserver startup
......................................................................

KUDU-2977 Sharding block map to speed up tserver startup

Separate LogBlockManager's block lock and container lock,
and sharding block map.

Tablet server with multiple data directories cost much time to
startup, there is a thread for each data directory, each of them
should aquire a global lock to add blocks found in its own data
directory to the global block map, so there is a queue to aquire
lock that may cost much time. The same when remove a batch of blocks.

After bunch of benchmarks, we found pretty stable improvment of this
patch, time saving much more when block count increasing (reduce about
20% startup time when 10,000,000 blocks on 8 data directories).

Sharding count is 8, which has the best performance after comparing
with 1, 2, 4, 8, 16, 32, 64 and 128.

Change-Id: If0d5c13e051a2c1d6cfd1c9ad7db8a3cd195459d
---
M src/kudu/fs/log_block_manager-test.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
3 files changed, 153 insertions(+), 103 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/55/14555/3
-- 
To view, visit http://gerrit.cloudera.org:8080/14555
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If0d5c13e051a2c1d6cfd1c9ad7db8a3cd195459d
Gerrit-Change-Number: 14555
Gerrit-PatchSet: 3
Gerrit-Owner: Yingchun Lai <40...@qq.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <40...@qq.com>