[06/32] incubator-impala git commit: IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment

[ta...@apache.org]

IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment

This script bootstraps an Impala dev environment on Ubuntu 14.04. It
is not hermetic -- it changes some config files for the user and for
the OS.

It is green on Jenkins, and it runs in about 6.5 hours. The intention
is to have this script run in a CI tool for post-commit testing, with
the hope that this will make it easier for new developers to get a
working development environment. Previously, the new developer
workflow lived on wiki pages and tended to bit-rot.

Still left to do: migrating the install script into the official
Impala repo.

Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e
Reviewed-on: http://gerrit.cloudera.org:8080/4674
Reviewed-by: Jim Apple <jb...@cloudera.com>
Tested-by: Internal Jenkins


Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/784716f7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/784716f7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/784716f7

Branch: refs/heads/hadoop-next
Commit: 784716f776492ed648833f2717b3784900b6f090
Parents: 2a04b0e
Author: Jim Apple <jb...@cloudera.com>
Authored: Fri Oct 7 20:34:28 2016 -0700
Committer: Internal Jenkins <cl...@gerrit.cloudera.org>
Committed: Fri Oct 14 06:07:46 2016 +0000

----------------------------------------------------------------------
 bin/bootstrap_development.sh | 80 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 80 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/784716f7/bin/bootstrap_development.sh
----------------------------------------------------------------------
diff --git a/bin/bootstrap_development.sh b/bin/bootstrap_development.sh
new file mode 100755
index 0000000..8c4f742
--- /dev/null
+++ b/bin/bootstrap_development.sh
@@ -0,0 +1,80 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script bootstraps a development environment from almost nothing; it is known to
+# work on Ubuntu 14.04, and it definitely clobbers some local environment, so it's best to
+# run this in a sandbox first, like a VM or docker.
+#
+# The intended user is a person who wants to start contributing code to Impala. This
+# script serves as an executable reference point for how to get started.
+#
+# At this time, it completes in about 6.5 hours. It generates and loads the test data and
+# metadata without using a snapshot (which takes about 3 hours) and it then runs the full
+# testsuite (frontend, backend, end-to-end, JDBC, and custom cluster) in "core"
+# exploration mode.
+
+set -eux -o pipefail
+
+HOMEDIR="/home/$(whoami)/"
+
+if [[ ! -d "${HOMEDIR}" ]]
+then
+    echo "${HOMEDIR} is needed for installing Impala dependencies"
+    exit 1
+fi
+
+if [[ -z "${JAVA_HOME}" ]]
+then
+    echo "JAVA_HOME must be set to install Impala dependencies"
+    exit 1
+fi
+
+if ! sudo true
+then
+    echo "Passwordless sudo is needed for this script"
+    exit 1
+fi
+
+IMPALA_SETUP_REPO_URL="https://github.com/awleblang/impala-setup"
+
+# Place to download setup scripts
+TMPDIR=$(mktemp -d)
+function cleanup {
+    rm -rf "${TMPDIR}"
+}
+trap cleanup EXIT
+
+# Install build and test pre-reqs
+pushd "${TMPDIR}"
+git clone "${IMPALA_SETUP_REPO_URL}" impala-setup
+cd impala-setup
+chmod +x ./install.sh
+sudo ./install.sh
+popd
+
+# HDFS bug workaround
+echo "127.0.0.1 $(hostname -s) $(hostname)" | sudo tee -a /etc/hosts
+echo "NoHostAuthenticationForLocalhost yes" > ~/.ssh/config
+
+pushd "$(dirname $0)/.."
+export IMPALA_HOME="$(pwd)"
+export MAX_PYTEST_FAILURES=0
+source bin/impala-config.sh
+./buildall.sh -noclean -format -testdata
+popd