[kudu-CR] KUDU-1489: move tablet metadata

["Andrew Wong (Code Review)" <ge...@cloudera.org>]

Hello Kudu Jenkins,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/7617

to look at the new patch set (#2).

Change subject: KUDU-1489: move tablet metadata
......................................................................

KUDU-1489: move tablet metadata

Currently tablet metadata are placed in the first specified data root.
This patch allows users to specify that metadata be spread across all
data roots.

To accomplish this, servers will scan meta and cmeta directories across
all roots and note where tablets currently are. When the meta or cmeta
directory is needed, if a more appropriate directory exists, it is
provided as an  alternate. With striping, an appropriate directory for
a tablet's metadata is any directory in the tablet's directory group.

Changes to the tablet lifecycle with striping:
- During DataDirManager::Open(), scan through all the directories and
  map all the tablets' current meta/cmeta directories.
- During calls to ListTabletIds(), scan through all directories and
  update the tablets' current directories.
- If there are duplicates (not an expected case, but may happen in the
  case of an unexpected shutdown), only one is kept.
- When writing the metas (e.g. ReplaceSuperBlockUnlocked()), check to
  see whether the metadata dir is appropriate, if not, write it to a
  different dir.
- The DataDirManager is now in charge of creating the metadata
  directories.

Testing is done in fs_manager-test to ensure that tablet metadata gets
striped and removed as needed.

Change-Id: Ia5308f8d4b4b738f353f511e2f1a2634b675a60a
---
M src/kudu/consensus/consensus_meta-test.cc
M src/kudu/consensus/consensus_meta.cc
M src/kudu/fs/data_dirs.cc
M src/kudu/fs/data_dirs.h
M src/kudu/fs/fs_manager-test.cc
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/integration-tests/external_mini_cluster_fs_inspector.cc
M src/kudu/tablet/tablet_metadata.cc
M src/kudu/tablet/tablet_metadata.h
10 files changed, 654 insertions(+), 168 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/7617/2
-- 
To view, visit http://gerrit.cloudera.org:8080/7617
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia5308f8d4b4b738f353f511e2f1a2634b675a60a
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Tidy Bot