[jira] [Commented] (IMPALA-11785) impala-shell should produce a warning if Thrift's fastbinary package isn't working

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725403#comment-17725403 ] 

ASF subversion and git services commented on IMPALA-11785:
----------------------------------------------------------

Commit 451543a2e56cc6a5830c7beca74d2153315bb15d in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=451543a2e ]

IMPALA-11785: Warn if Thrift fastbinary is not working for impala-shell

Thrift's fastbinary module provides native code that
accelerations the BinaryProtocol. It can make a large
performance difference when using the Hiveserver2
protocol with impala-shell. If the fastbinary is not
working, it silently falls back to interpreted code.
This can happen because the fastbinary couldn't load
a particular library, etc.

This adds a warning on impala-shell startup when
it detects that Thrift's fastbinary is not working.

When bin/impala-shell.sh is modified to use python3,
impala-shell outputs this error (shortened for legibility):
WARNING: Failed to load Thrift's fastbinary module. Thrift's
BinaryProtocol will not be accelerated, which can reduce performance.
Error was '{path to Python2 thrift fastbinary.so}: undefined symbol: _Py_ZeroStruct'

Testing:
 - Added a simple test that verifies the impala-shell
   does not output the warning
 - Outputs warning when Python 2 thrift used for Python 3 shell

Change-Id: Id5d0e5db5cfdf1db4521b00f912b4697a7f646e8
Reviewed-on: http://gerrit.cloudera.org:8080/19806
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> impala-shell should produce a warning if Thrift's fastbinary package isn't working
> ----------------------------------------------------------------------------------
>
>                 Key: IMPALA-11785
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11785
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Clients
>    Affects Versions: Impala 4.3.0
>            Reporter: Joe McDonnell
>            Assignee: Joe McDonnell
>            Priority: Major
>
> Thrift's fastbinary package accelerates the binary protocol for Thrift with native code, and it can make a large difference in performance for impala-shell (and other clients). Differences in compilers or other build-time discrepancies can cause it to fail to load. When that happens, it falls back to the non-accelerated Python code.
> Currently, there is no warning when it falls back to the non-accelerated code. We should change impala-shell to issue a warning in this case. One way to do that is to try to import thrift.protocol.fastbinary and issue a warning if it fails to import.
> {noformat}
> try:
>   import thrift.protocol.fastbinary
>   print("Successfully imported fastbinary")
> except:
>   print("Could not import fastbinary"){noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org