[impala] 02/04: IMPALA-11610: Pass environment variables into dockerized-impala-run-tests.sh

[jo...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git

commit 72812c5955a23c9270008e8371d8585d5165a27d
Author: Joe McDonnell <jo...@cloudera.com>
AuthorDate: Fri Sep 30 20:39:10 2022 -0700

    IMPALA-11610: Pass environment variables into dockerized-impala-run-tests.sh
    
    Because dockerized-impala-bootstrap-test.sh does a relogin while
    calling dockerized-impala-run-tests.sh, the environment is not
    preserved.
    
    This adds a script dockerized-impala-preserve-vars.py that takes
    a list of environment variables to preserve and appends
    export statements to bin/impala-config-local.sh. Since
    dockerized-impala-run-tests.sh sources bin/impala-config.sh, these
    variables will be carried into the test execution.
    
    This starts by adding environment variables used by upstream
    Jenkin's ubuntu-16.04-dockerized-tests. Jenkins jobs can also
    call dockerized-impala-preserve-vars.py directly.
    
    Testing:
     - Hand tested the preservation script
     - Verified ubuntu-16.04-dockerized-tests now respected EE_TEST
       argument.
    
    Change-Id: I325217c731883c087c724194b45d50b790c7c280
    Reviewed-on: http://gerrit.cloudera.org:8080/19088
    Reviewed-by: Joe McDonnell <jo...@cloudera.com>
    Tested-by: Joe McDonnell <jo...@cloudera.com>
---
 .../dockerized-impala-bootstrap-and-test.sh        |  7 +++
 bin/jenkins/dockerized-impala-preserve-vars.py     | 56 ++++++++++++++++++++++
 2 files changed, 63 insertions(+)

diff --git a/bin/jenkins/dockerized-impala-bootstrap-and-test.sh b/bin/jenkins/dockerized-impala-bootstrap-and-test.sh
index e2171da3a..7317ba93c 100755
--- a/bin/jenkins/dockerized-impala-bootstrap-and-test.sh
+++ b/bin/jenkins/dockerized-impala-bootstrap-and-test.sh
@@ -30,6 +30,13 @@ source ./bin/bootstrap_system.sh
 # Install docker
 ./bin/jenkins/install_docker.sh
 
+# Preserve some important environment variables so that they are available to
+# dockerized-impala-run-tests.sh.
+# NOTE: A Jenkins job can also call dockerized-impala-preserve-vars.py directly
+# to preserve additional variables.
+./bin/jenkins/dockerized-impala-preserve-vars.py \
+    EE_TEST EE_TEST_FILES JDBC_TEST EXPLORATION_STRATEGY CMAKE_BUILD_TYPE
+
 # Execute the tests using su to re-login so that group change made above
 # setup_docker takes effect. This does a full re-login and does not stay
 # in the current directory, so change back to $IMPALA_HOME (resolved in
diff --git a/bin/jenkins/dockerized-impala-preserve-vars.py b/bin/jenkins/dockerized-impala-preserve-vars.py
new file mode 100755
index 000000000..f36a45dab
--- /dev/null
+++ b/bin/jenkins/dockerized-impala-preserve-vars.py
@@ -0,0 +1,56 @@
+#!/usr/bin/env python
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Since dockerized-impala-bootstrap-and-test.sh does a full re-login
+# as part of calling dockerized-impala-run-tests.sh, it loses
+# environment variables. Preserving the environment variables are important
+# for parameterized Jenkins jobs. This script takes a list of environment
+# variables and preserves them by adding export statements to
+# bin/impala-config-local.sh.
+#
+# Usage: dockerized-impala-preserve-vars.py [env var1] [env var2] ...
+# If an environment variable is not defined in the current environment,
+# it is omitted with a warning.
+
+import sys
+import os
+
+
+def main():
+  if len(sys.argv) <= 1:
+    print("Usage: {0} [env vars]".format(sys.argv[0]))
+    sys.exit(1)
+
+  if "IMPALA_HOME" not in os.environ:
+    print("ERROR: IMPALA_HOME must be defined")
+    sys.exit(1)
+
+  impala_home = os.environ["IMPALA_HOME"]
+  # Append to the end of bin/impala-config-local.sh
+  with open("{0}/bin/impala-config-local.sh".format(impala_home), "a") as f:
+    for env_var in sys.argv[1:]:
+      if env_var not in os.environ:
+        print("{0} is not defined in the environment, skipping...".format(env_var))
+        continue
+      new_export = "export {0}=\"{1}\"".format(env_var, os.environ[env_var])
+      print("Adding '{0}' to bin/impala-config-local.sh".format(new_export))
+      f.write("{0}\n".format(new_export))
+
+
+if __name__ == "__main__": main()