[Impala-ASF-CR] IMPALA-7988: support loading data with dockerized Impalas

["Tim Armstrong (Code Review)" <ge...@cloudera.org>]

Tim Armstrong has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/12189 )

Change subject: IMPALA-7988: support loading data with dockerized Impalas
......................................................................

IMPALA-7988: support loading data with dockerized Impalas

This patch does the work to load data and run some end-to-end
query tests on a dockerised cluster. Changes were required
in start-impala-cluster.py/ImpalaCluster and in some configuration
files.

ImpalaCluster is used for various things, including discovering
service ports and testing for cluster readiness. This patch adds
basic support and uses it from start-impala-cluster.py to check
for cluster readiness. Some logic is moved from
start-impala-cluster.py to ImpalaCluster.

Limitations:
* We're fairly inconsistent about whether services listen only on
  a single interface (e.g. loopback, traditionally) or whether it
  listens on all interfaces. This doesn't fix all of those issues.
  E.g. HDFS datanodes listen on all interfaces to work around
  some issues.
* Many tests don't pass yet, particularly those using
  ImpalaCluster(), which isn't initialised with the appropriate
  docker arguments.

Testing:
Did a full data load locally using a dockerised Impala cluster:

  START_CLUSTER_ARGS="--docker_network=impala-cluster" \
  TEST_START_CLUSTER_ARGS="--docker_network=impala-cluster" \
  ./buildall.sh -format -testdata -ninja -notests -skiptests -noclean

Ran a selection of end-to-end tests touching HDFS, Kudu and HBase
tables after I loaded data locally.

Ran exhaustive tests with non-dockerised impala cluster.

Change-Id: I98fb9c4f5a3a3bb15c7809eab28ec8e5f63ff517
---
M bin/impala-config.sh
M bin/start-impala-cluster.py
M fe/src/test/resources/hbase-site.xml.template
M fe/src/test/resources/hive-default.xml
M fe/src/test/resources/mysql-hive-site.xml.template
M fe/src/test/resources/postgresql-hive-site.xml.template
M testdata/bin/create-load-data.sh
M testdata/bin/run-all.sh
M testdata/cluster/node_templates/common/etc/hadoop/conf/hdfs-site.xml.tmpl
M testdata/cluster/node_templates/common/etc/kudu/tserver.conf.tmpl
M tests/common/impala_cluster.py
11 files changed, 358 insertions(+), 229 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/12189/5
-- 
To view, visit http://gerrit.cloudera.org:8080/12189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I98fb9c4f5a3a3bb15c7809eab28ec8e5f63ff517
Gerrit-Change-Number: 12189
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>