(impala) 02/02: IMPALA-13326: Prefer python3 for tarball packaged impala-shell

[st...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

stigahuang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git

commit cbb35ebccdb50c0a032fe2b39c2b3ebb496479a9
Author: Joe McDonnell <jo...@cloudera.com>
AuthorDate: Tue May 27 14:12:07 2025 -0700

    IMPALA-13326: Prefer python3 for tarball packaged impala-shell
    
    The tarball packaging for impala-shell ships support for
    multiple Python versions (including both Python 2 and Python 3).
    In the impala-shell script, it determines the python to use
    and uses the corresponding installation. Historically, impala-shell
    has preferred the "python" executable (which can be Python 2) to
    the "python3" executable. Since Python 2 is deprecated, this flips
    the preference to prefer "python3" to "python".
    
    This continues to respect IMPALA_PYTHON_EXECUTABLE as before, but
    it adds an IMPALA_SHELL_PYTHON_FALLBACK variable to determine
    whether to fall back to the regular logic. This defaults to
    true, allowing fallback, to maintain existing behavior. The
    shell end-to-end tests set this to false to lock in the
    Python version.
    
    Testing:
     - Ran shell tests
    
    Change-Id: If0e32e8eee672e4dc66e725722f5150cd1e4c9a6
    Reviewed-on: http://gerrit.cloudera.org:8080/22953
    Reviewed-by: Riza Suminto <ri...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
    Reviewed-by: Michael Smith <mi...@cloudera.com>
---
 shell/packaging/impala-shell | 34 ++++++++++++++++++++++++++++++----
 tests/shell/util.py          |  6 ++++--
 2 files changed, 34 insertions(+), 6 deletions(-)

diff --git a/shell/packaging/impala-shell b/shell/packaging/impala-shell
index b666b69cf..38013a725 100755
--- a/shell/packaging/impala-shell
+++ b/shell/packaging/impala-shell
@@ -29,10 +29,36 @@ SHELL_HOME=${IMPALA_SHELL_HOME:-${SCRIPT_DIR}}
 # Set the envrionment's locale settings to allow for utf-8 compatibility
 export LC_CTYPE=${LC_CTYPE:-en_US.UTF-8}
 
-# Select python version; prefer 2, use 3 if 2's absent. Allow override with envvar
-PYTHON_EXE="${IMPALA_PYTHON_EXECUTABLE:-python}"
-if ! command -v "${PYTHON_EXE}" > /dev/null; then
-  PYTHON_EXE=python3
+# If the user specified IMPALA_PYTHON_EXECUTABLE and it exists, use it.
+# If it doesn't exist and IMPALA_SHELL_PYTHON_FALLBACK is false, exit with an error
+# message. Otherwise, fall back to the regular path. IMPALA_SHELL_PYTHON_FALLBACK
+# defaults to true to match historical behavior.
+IMPALA_SHELL_PYTHON_FALLBACK=${IMPALA_SHELL_PYTHON_FALLBACK:-true}
+PYTHON_EXE=""
+if [[ -n "${IMPALA_PYTHON_EXECUTABLE}" ]]; then
+  if command -v "${IMPALA_PYTHON_EXECUTABLE}" > /dev/null; then
+    PYTHON_EXE="${IMPALA_PYTHON_EXECUTABLE}"
+  elif ! ${IMPALA_SHELL_PYTHON_FALLBACK} ; then
+    echo "Couldn't find IMPALA_PYTHON_EXECUTABLE=${IMPALA_PYTHON_EXECUTABLE}."
+    exit 1
+  fi
+fi
+
+# If IMPALA_PYTHON_EXECUTABLE isn't specified or doesn't exist, try the regular
+# "python3" and "python" executables. Since Python 2 is deprecated, prefer "python3"
+# to "python" (though "python" may actually point to Python 3).
+if [[ -z "${PYTHON_EXE}" ]]; then
+  if command -v python3 > /dev/null; then
+    PYTHON_EXE=python3
+  elif command -v python > /dev/null; then
+    PYTHON_EXE=python
+  else
+    if [[ -n "${IMPALA_PYTHON_EXECUTABLE}" ]]; then
+      echo "Couldn't find IMPALA_PYTHON_EXECUTABLE=${IMPALA_PYTHON_EXECUTABLE}."
+    fi
+    echo "Couldn't find python3 or python on the PATH"
+    exit 1
+  fi
 fi
 
 # impala-shell is installed in /install_py${PYTHON_VERSION}
diff --git a/tests/shell/util.py b/tests/shell/util.py
index e6637af2d..191f24849 100755
--- a/tests/shell/util.py
+++ b/tests/shell/util.py
@@ -397,8 +397,10 @@ def get_impala_shell_executable(vector):
   # use 'dev' as the default.
   impala_shell_executable, _ = get_dev_impala_shell_executable()
   return {
-    'dev': [impala_shell_executable],
-    'dev3': ['env', 'IMPALA_PYTHON_EXECUTABLE=python3', impala_shell_executable],
+    'dev': ['env', 'IMPALA_PYTHON_EXECUTABLE=python',
+            'IMPALA_SHELL_PYTHON_FALLBACK=false', impala_shell_executable],
+    'dev3': ['env', 'IMPALA_PYTHON_EXECUTABLE=python3',
+             'IMPALA_SHELL_PYTHON_FALLBACK=false', impala_shell_executable],
     'python2': [os.path.join(IMPALA_HOME, 'shell/build/python2_venv/bin/impala-shell')],
     'python3': [os.path.join(IMPALA_HOME, 'shell/build/python3_venv/bin/impala-shell')]
   }[vector.get_value_with_default('impala_shell', 'dev')]