[kudu-CR] [Tool] Compact metadata file for old versions of Kudu

["Wang Xixu (Code Review)" <ge...@cloudera.org>]

Hello Tidy Bot, Kudu Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19327

to look at the new patch set (#2).

Change subject: [Tool] Compact metadata file for old versions of Kudu
......................................................................

[Tool] Compact metadata file for old versions of Kudu

Som old versions of Kudu do not support compacting metadata
file in runtime. A long time later, metadata file becomes very
big, for example, more than 100G. When restarting Kudu tablet
server, it will read all metadata files to check whick blocks
are still alive and compact metadata files into new small
metadata files which only record alive blocks. As the metadata
files are very big, compacting metadata files will cost a lot of
time, maybe 2-3 hours. That will cause the service unavailabe
for a long time, which is unacceptable.

Runtime metadata file compacting has already been supported
above version of 1.16.0, see jira: #KUDU-3318. Upgrading old
version to above Kudu 1.16.0 still needs to restart Kudu.

This tool helps to compact metdata file for old versions of
Kudu. It supports 4 operations:
--print: Statistic the total blocks and alive blocks for
every container.
--compact: Compact metadata files when Kudu is still running
--merge: Stop Kudu and write incremental records into compacted
metadata files. Then backup old metadata files.
--clear: Delete metadata backup files.

Restarting old version of Kudu maybe cost 2-3 hours, but using
compact tool, service stopped only lasts 3 minutes.

Change-Id: Id61cb6c88d7a093a8448e2497639f5c5c094efa4
---
M src/kudu/fs/log_block_manager-test.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
M src/kudu/tools/tool_action_fs.cc
M src/kudu/util/pb_util.cc
M src/kudu/util/pb_util.h
6 files changed, 592 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/19327/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19327
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id61cb6c88d7a093a8448e2497639f5c5c094efa4
Gerrit-Change-Number: 19327
Gerrit-PatchSet: 2
Gerrit-Owner: Wang Xixu <14...@qq.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)