[impala] 01/02: Allow data cache to be enabled optionally when running tests

[ta...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

tarmstrong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git

commit 8f533e334c660dcdeaa4322b8755d7c0dce55f69
Author: Michael Ho <kw...@cloudera.com>
AuthorDate: Sun May 12 17:19:23 2019 -0700

    Allow data cache to be enabled optionally when running tests
    
    This change adds two environment variables ${DATA_CACHE_DIR}
    and ${DATA_CACHE_SIZE} which specify the root directory path
    of the data cache and the data cache size respectively. If both
    are set, Impala cluster will be started with data cache enabled
    when running E2E tests. When running with HDFS, it will also
    set the config option --always_use_data_cache to true to force
    use of the data cache all the time.
    
    Change-Id: I09117ab289c2355408212a5fc6493ab751f4853b
    Reviewed-on: http://gerrit.cloudera.org:8080/13330
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 bin/run-all-tests.sh | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/bin/run-all-tests.sh b/bin/run-all-tests.sh
index c77d062..b74d03c 100755
--- a/bin/run-all-tests.sh
+++ b/bin/run-all-tests.sh
@@ -59,6 +59,10 @@ fi
 : ${RUN_TESTS_ARGS:=}
 # Extra args to pass to run-custom-cluster-tests.sh
 : ${RUN_CUSTOM_CLUSTER_TESTS_ARGS:=}
+# Data cache root directory location.
+: ${DATA_CACHE_DIR:=}
+# Data cache size.
+: ${DATA_CACHE_SIZE:=}
 if [[ "${TARGET_FILESYSTEM}" == "local" ]]; then
   # TODO: Remove abort_on_config_error flag from here and create-load-data.sh once
   # checkConfiguration() accepts the local filesystem (see IMPALA-1850).
@@ -69,6 +73,17 @@ else
   TEST_START_CLUSTER_ARGS="${TEST_START_CLUSTER_ARGS} --cluster_size=3"
 fi
 
+# Enable data cache if configured.
+if [[ -n "${DATA_CACHE_DIR}" && -n "${DATA_CACHE_SIZE}" ]]; then
+   TEST_START_CLUSTER_ARGS="${TEST_START_CLUSTER_ARGS} "`
+       `"--data_cache_dir=${DATA_CACHE_DIR} --data_cache_size=${DATA_CACHE_SIZE}"
+   # Force use of data cache for HDFS. Data cache is only enabled for remote reads.
+   if [[ "${TARGET_FILESYSTEM}" == "hdfs" ]]; then
+      TEST_START_CLUSTER_ARGS="${TEST_START_CLUSTER_ARGS} "`
+          `"--impalad_args=--always_use_data_cache=true"
+   fi
+fi
+
 if [[ "${ERASURE_CODING}" = true ]]; then
   # We do not run FE tests when erasure coding is enabled because planner tests
   # would fail.