[kudu-CR] minicluster: facilitate creating multidir clusters

["Andrew Wong (Code Review)" <ge...@cloudera.org>]

Hello Kudu Jenkins,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/6845

to look at the new patch set (#10).

Change subject: minicluster: facilitate creating multidir clusters
......................................................................

minicluster: facilitate creating multidir clusters

In order to test different disk configurations, it is becoming
increasingly important to have end-to-end testing with nodes backed
by multiple directories.

External miniclusters by default use a single directory for each daemon's
data (i.e. wals, data dirs, logs fall under a single
/cluster/daemon/data directory). This patch adds multi-directory via a
new optional ExternalDaemonPathSpec struct.

Since miniclusters generate directories under a single cluster root,
specifying full path names via additional flags is tedious to do at
runtime for multiple daemons, only made worse by the fact that
miniclusters also create default directories based on the daemon's path.

Instead, the ExternalDaemonPathSpec specifies subdirectories as a list
of numeric postfixes to add to the default path. On top of specifying
data dirs, the PathSpec also allows specification of the wal and log
dirs.

E.g. miniclusters that would have used the path /cluster/data can
now specify multiple directories /cluster/data-1, /cluster/data-2,
/cluster/data-3, etc.

/test_path
+-cluster
  +-daemon (master, ts-0, ts-1)
    +-datadir (data-0, data-1, ...; before, this would be a single dir)
      +-data, consensus-meta, etc.

The new test multidir-cluster-itest demonstrates this.

Change-Id: Id2f5def6980ad394c8558ad97ba830f1b0257332
---
M src/kudu/integration-tests/CMakeLists.txt
M src/kudu/integration-tests/external_mini_cluster.cc
M src/kudu/integration-tests/external_mini_cluster.h
A src/kudu/integration-tests/multidir-cluster-itest.cc
M src/kudu/integration-tests/ts_itest-base.h
M src/kudu/util/path_util.cc
M src/kudu/util/path_util.h
7 files changed, 232 insertions(+), 30 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/45/6845/10
-- 
To view, visit http://gerrit.cloudera.org:8080/6845
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id2f5def6980ad394c8558ad97ba830f1b0257332
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: David Ribeiro Alves <da...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Tidy Bot