[23/32] incubator-impala git commit: IMPALA-4299: add buildall.sh option to start test cluster

[ta...@apache.org]

IMPALA-4299: add buildall.sh option to start test cluster

A previous commit "IMPALA-4259: build Impala without any test
cluster setup" altered some undocumented side-effects of
buildall.sh.

Previously the following commands reconfigured and restarted the test
cluster. It worked because buildall.sh unconditionally regenerated
the test cluster configs.

  ./buildall.sh -notests && ./testdata/bin/run-all.sh
  ./buildall.sh -noclean -notests && ./testdata/bin/run-all.sh

Instead of restoring the old behaviour and continuing to encourage
mixing use of low and high level scripts like testdata/bin/run-all.sh
as part of the "standard" workflow, this commit adds another
high-level option to buildall.sh, -start_minicluster, that
accomplishes the high-level task of restarting a minicluster with
fresh configs. The above commands can be replaced with:

  ./buildall.sh -notests -start_minicluster
  ./buildall.sh -notests -noclean -start_minicluster

Change-Id: I0ab3461f8ff3de49b3f28a0dc22fa0a6d5569da5
Reviewed-on: http://gerrit.cloudera.org:8080/4734
Reviewed-by: Alex Behm <al...@cloudera.com>
Tested-by: Internal Jenkins


Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/ef762b73
Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/ef762b73
Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/ef762b73

Branch: refs/heads/hadoop-next
Commit: ef762b73a1ddb8842bb5bac5bfed733b9a71fc1b
Parents: 0480253
Author: Tim Armstrong <ta...@cloudera.com>
Authored: Mon Oct 17 10:04:39 2016 -0700
Committer: Internal Jenkins <cl...@gerrit.cloudera.org>
Committed: Mon Oct 17 22:19:06 2016 +0000

----------------------------------------------------------------------
 buildall.sh | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/ef762b73/buildall.sh
----------------------------------------------------------------------
diff --git a/buildall.sh b/buildall.sh
index ebd3eb3..a7858a3 100755
--- a/buildall.sh
+++ b/buildall.sh
@@ -46,6 +46,7 @@ TESTS_ACTION=1
 FORMAT_CLUSTER=0
 FORMAT_METASTORE=0
 FORMAT_SENTRY_POLICY_DB=0
+NEED_MINICLUSTER=0
 START_IMPALA_CLUSTER=0
 IMPALA_KERBERIZE=0
 SNAPSHOT_FILE=
@@ -133,6 +134,9 @@ do
       METASTORE_SNAPSHOT_FILE="$(readlink -f "$METASTORE_SNAPSHOT_FILE")"
       shift;
       ;;
+    -start_minicluster)
+      NEED_MINICLUSTER=1
+      ;;
     -start_impala_cluster)
       START_IMPALA_CLUSTER=1
       ;;
@@ -169,6 +173,10 @@ do
       echo "[-asan] : Address sanitizer build [Default: False]"
       echo "[-skiptests] : Skips execution of all tests"
       echo "[-notests] : Skips building and execution of all tests"
+      echo "[-start_minicluster] : Start test cluster including Impala and all"\
+            " its dependencies. If already running, all services are restarted."\
+            " Regenerates test cluster config files. [Default: True if running "\
+            " tests or loading data, False otherwise]"
       echo "[-start_impala_cluster] : Start Impala minicluster after build"\
            " [Default: False]"
       echo "[-testpairwise] : Run tests in 'pairwise' mode (increases"\
@@ -191,8 +199,12 @@ Examples of common tasks:
   # Build and skip tests
   ./buildall.sh -skiptests
 
-  # Incrementally rebuild and skip tests. Keeps existing Hadoop services running.
-  ./buildall.sh -skiptests -noclean
+  # Build, then restart the minicluster and Impala with fresh configs.
+  ./buildall.sh -notests -start_minicluster -start_impala_cluster
+
+  # Incrementally rebuild and skip tests. Keeps existing minicluster services running
+  # and restart Impala.
+  ./buildall.sh -skiptests -noclean -start_impala_cluster
 
   # Build, load a snapshot file, run tests
   ./buildall.sh -snapshot_file <file>
@@ -256,7 +268,6 @@ if [[ -z "$METASTORE_SNAPSHOT_FILE" && "${TARGET_FILESYSTEM}" != "hdfs" &&
   exit 1
 fi
 
-NEED_MINICLUSTER=0
 if [[ $TESTS_ACTION -eq 1 || $TESTDATA_ACTION -eq 1 || $FORMAT_CLUSTER -eq 1 ||
       $FORMAT_METASTORE -eq 1 || $FORMAT_SENTRY_POLICY_DB -eq 1 || -n "$SNAPSHOT_FILE" ||
       -n "$METASTORE_SNAPSHOT_FILE" ]]; then