[impala] 04/04: IMPALA-11317/IMPALA-11316/IMPALA-11315: impala-shell Python 3 fixes

[st...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

stigahuang pushed a commit to branch branch-4.1.1
in repository https://gitbox.apache.org/repos/asf/impala.git

commit f6ee249ac96fb09293cd3f9f1c0adeafb923cd5d
Author: Joe McDonnell <jo...@cloudera.com>
AuthorDate: Sat May 21 19:04:45 2022 -0700

    IMPALA-11317/IMPALA-11316/IMPALA-11315: impala-shell Python 3 fixes
    
    This fixes a few impala-shell Python 3 issues:
    1. In ImpalaShell's do_history(), the decode() call needs to be
       avoided in Python 3, because in Python 3 the cmd is already
       a string and doesn't need further decoding. (IMPALA-11315)
    2. TestImpalaShell.test_http_socket_timeout() gets a different
       error message in Python 3. It throws the "BlockingIOError"
       rather than "socker.error". (IMPALA-11316)
    3. ImpalaHttpClient.py's code to retrieve the body when
       handling an HTTP error needs to have a decode() call
       for the body. Otherwise, the body remains bytes and
       causes TestImpalaShellInteractive.test_http_interactions_extra()
       to fail. (IMPALA-11317)
    
    Testing:
     - Ran shell tests in the standard way
     - Ran shell tests with the impala-shell executable coming from
       a Python 3 virtualenv using the PyPi package
    
    Change-Id: Ie58380a17d7e011f4ce96b27d34717509a0b80a6
    Reviewed-on: http://gerrit.cloudera.org:8080/18556
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Reviewed-by: Wenzhe Zhou <wz...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
    Reviewed-on: http://gerrit.cloudera.org:8080/18885
    Reviewed-by: Quanlong Huang <hu...@gmail.com>
---
 shell/ImpalaHttpClient.py             |  2 +-
 shell/impala_shell.py                 |  5 ++++-
 tests/shell/test_shell_commandline.py | 11 ++++++++---
 3 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/shell/ImpalaHttpClient.py b/shell/ImpalaHttpClient.py
index 947cecfc1..bbf1e4d6f 100644
--- a/shell/ImpalaHttpClient.py
+++ b/shell/ImpalaHttpClient.py
@@ -271,5 +271,5 @@ class ImpalaHttpClient(TTransportBase):
     if self.code >= 300:
       # Report any http response code that is not 1XX (informational response) or
       # 2XX (successful).
-      body = self.readBody()
+      body = self.readBody().decode('utf-8')
       raise HttpError(self.code, self.message, body, self.headers)
diff --git a/shell/impala_shell.py b/shell/impala_shell.py
index acaa02b88..8a061c736 100755
--- a/shell/impala_shell.py
+++ b/shell/impala_shell.py
@@ -1508,7 +1508,10 @@ class ImpalaShell(cmd.Cmd, object):
     if self.readline and self.readline.get_current_history_length() > 0:
       for index in xrange(1, self.readline.get_current_history_length() + 1):
         cmd = self.readline.get_history_item(index)
-        print('[%d]: %s' % (index, cmd.decode('utf-8', 'replace')), file=sys.stderr)
+        if sys.version_info.major == 2:
+          print('[%d]: %s' % (index, cmd.decode('utf-8', 'replace')), file=sys.stderr)
+        else:
+          print('[%d]: %s' % (index, cmd), file=sys.stderr)
     else:
       print(READLINE_UNAVAILABLE_ERROR, file=sys.stderr)
 
diff --git a/tests/shell/test_shell_commandline.py b/tests/shell/test_shell_commandline.py
index ef6536e26..cd53404c2 100644
--- a/tests/shell/test_shell_commandline.py
+++ b/tests/shell/test_shell_commandline.py
@@ -1220,9 +1220,14 @@ class TestImpalaShell(ImpalaTestSuite):
     args = ['--quiet', '-B', '--query', 'select 0;']
     result = run_impala_shell_cmd(vector, args + ['--http_socket_timeout_s=0'],
                                   expect_success=False)
-    expected_err = ("Caught exception [Errno 115] Operation now in progress, "
-                   "type=<class 'socket.error'> in OpenSession. Num remaining tries: 3")
-    assert result.stderr.splitlines()[0] == expected_err
+    expected_err_py2 = (
+      "Caught exception [Errno 115] Operation now in progress, "
+      "type=<class 'socket.error'> in OpenSession. Num remaining tries: 3")
+    expected_err_py3 = (
+      "Caught exception [Errno 115] Operation now in progress, "
+      "type=<class 'BlockingIOError'> in OpenSession. Num remaining tries: 3")
+    actual_err = result.stderr.splitlines()[0]
+    assert actual_err == expected_err_py2 or actual_err == expected_err_py3
 
     # Test http_socket_timeout_s=-1, expect errors
     result = run_impala_shell_cmd(vector, args + ['--http_socket_timeout_s=-1'],