[impala] branch master updated: IMPALA-11313: Use Thrift 0.14.2 for impala-shell PyPi package

[jo...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git


The following commit(s) were added to refs/heads/master by this push:
     new a11450db8 IMPALA-11313: Use Thrift 0.14.2 for impala-shell PyPi package
a11450db8 is described below

commit a11450db86e69b25345baf74c52a1592a27c42df
Author: Joe McDonnell <jo...@cloudera.com>
AuthorDate: Sat May 21 19:17:11 2022 -0700

    IMPALA-11313: Use Thrift 0.14.2 for impala-shell PyPi package
    
    Thrift 0.11.0 has known issues where Unicode errors are
    not handler properly, including one case where the client
    can hang. The traditional form factor for impala-shell
    uses a patched Thrift that fixes those issues, but the
    PyPi package uses the unpatched Thrift 0.11.0.
    
    This modifies the requirements.txt file to use Thrift 0.14.2,
    which has fixes for these Unicode issues. Thrift 0.14.2 has
    a slightly different error message, so this amends the
    allowed error messages in test_utf8_decoding_error_handling().
    
    This is a bit awkward, given that the Python code generation
    continues to happen with Thrift 0.11.0. Comparing the
    Python code for Thrift 0.11 vs Thrift 0.14, I didn't see
    noticeable differences. Given that the client can hang,
    this seems worth fixing ahead of the full conversion to
    Thrift 0.14 for all of Impala.
    
    Testing:
     - Ran the Unicode error handling tests with a PyPi
       impala-shell
     - Ran the shell tests normally
    
    Change-Id: I63e0a5dda98df20c9184a347397118b1f3529603
    Reviewed-on: http://gerrit.cloudera.org:8080/18560
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 shell/packaging/requirements.txt      | 2 +-
 tests/shell/test_shell_commandline.py | 7 +++++--
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/shell/packaging/requirements.txt b/shell/packaging/requirements.txt
index 73e09f157..40b93246f 100644
--- a/shell/packaging/requirements.txt
+++ b/shell/packaging/requirements.txt
@@ -6,5 +6,5 @@ sasl==0.2.1
 setuptools>=36.8.0
 six==1.14.0
 sqlparse==0.3.1
-thrift==0.11.0
+thrift==0.14.2
 thrift_sasl==0.4.3
diff --git a/tests/shell/test_shell_commandline.py b/tests/shell/test_shell_commandline.py
index 14a6fdcfb..65a96515d 100644
--- a/tests/shell/test_shell_commandline.py
+++ b/tests/shell/test_shell_commandline.py
@@ -533,8 +533,11 @@ class TestImpalaShell(ImpalaTestSuite):
     result = run_impala_shell_cmd(vector, ['-B', '-q', "select unhex('aa')"])
     assert 'UnicodeDecodeError' not in result.stderr
     # Same as above, the result using thrift <0.10.0 is '\xaa'. The result using
-    # thrift >=0.10.0 is '\xef\xbf\xbd'.
-    assert '\xef\xbf\xbd' in result.stdout or '\xaa' in result.stdout
+    # thrift >=0.10.0 is '\xef\xbf\xbd'. When testing with strict_hs2_protocol, this
+    # is running against Hive which is another variable. On thrift 0.14 and higher,
+    # talking to Hive, the result is b'\\xaa', so allow this as another possibility.
+    assert '\xef\xbf\xbd' in result.stdout or '\xaa' in result.stdout or \
+           '\\xaa' in result.stdout
 
   def test_global_config_file(self, vector):
     """Test global and user configuration files."""