You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/06/21 05:22:00 UTC
[jira] [Commented] (IMPALA-12219) impala-shell displays DeprecationWarning when using Python >=3.10
[ https://issues.apache.org/jira/browse/IMPALA-12219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17735539#comment-17735539 ]
ASF subversion and git services commented on IMPALA-12219:
----------------------------------------------------------
Commit 234d641d7bba4c2cab8e7c2b5fb72d2861bea23f in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=234d641d7 ]
IMPALA-11961/IMPALA-12207: Add Redhat 9 / Ubuntu 22 support
This adds support for Redhat 9 / Ubuntu 22. It updates
to a newer toolchain that has those builds, and it adds
supporting code in bootstrap_system.sh.
Redhat 9 and Ubuntu 22 use python = python3, which requires
various changes to build scripts and tests. Ubuntu 22 uses
Python 3.10, which deprecates certain ssl.PROTOCOL_TLS, so
this adapts test_client_ssl.py to that change until it
can be fully addressed in IMPALA-12219.
Various OpenSSL methods have been deprecated. As a workaround
until these can be addressed properly, this specifies
-Wno-deprecated-declarations. This can be removed once the
code is adapted to the non-deprecated APIs in IMPALA-12226.
Impala crashes with tcmalloc errors unless we update to a newer
gperftools, so this moves to gperftools 2.10. gperftools changed
the default for tcmalloc.aggressive_memory_decommit to off, so
this adapts our code to set it for backend tests. The gperftools
upgrade does not show any performance regression:
+----------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+-----------------------+---------+------------+------------+----------------+
| TPCH(42) | parquet / none / none | 3.08 | -0.64% | 2.20 | -0.37% |
+----------+-----------------------+---------+------------+------------+----------------+
With newer Python versions, the impala-virtualenv command
fails to create a Python 3 virtualenv. This switches to
using Python 3's builtin venv command for Python >=3.6.
Kudu needed a newer version and LLVM required a couple patches.
Testing:
- Ran a core job on Ubuntu 22 and Redhat 9. The tests run
to completion without crashing. There are test failures
that will be addressed in follow-up JIRAs.
- Ran dockerised tests on Ubuntu 22.
- Ran dockerised tests on Ubuntu 20 and Rocky 8.5.
Change-Id: If1fcdb2f8c635ecd6dc7a8a1db81f5f389c78b86
Reviewed-on: http://gerrit.cloudera.org:8080/20073
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Joe McDonnell <jo...@cloudera.com>
> impala-shell displays DeprecationWarning when using Python >=3.10
> -----------------------------------------------------------------
>
> Key: IMPALA-12219
> URL: https://issues.apache.org/jira/browse/IMPALA-12219
> Project: IMPALA
> Issue Type: Bug
> Components: Clients
> Affects Versions: Impala 4.3.0
> Reporter: Joe McDonnell
> Priority: Major
>
> Ubuntu 22 uses Python 3.10, which deprecated ssl.PROTOCOL_TLS. When impala-shell starts up, it produces this message:
> {noformat}
> Starting Impala Shell with no authentication using Python 3.10.6
> SSL is enabled. Impala server certificates will NOT be verified (set --ca_cert to change)
> Warning: live_progress only applies to interactive shell sessions, and is being skipped for now.
> /home/impdev/Impala/shell/build/impala-shell-4.3.0-SNAPSHOT/ext-py3/thrift-0.16.0-py3.10-linux-x86_64.egg/thrift/transport/TSSLSocket.py:53: DeprecationWarning: ssl.PROTOCOL_TLS is deprecated{noformat}
> This doesn't seem to be coming from impala-shell code, so we will need to track down what is using this.
> This is somewhat harmless, except that it causes test failures for custom_cluster/test_client_ssl.py, which asserts that DeprecationWarning is not in the output.
> {noformat}
> def _validate_positive_cases(self, vector, ca_cert=""):
> shell_options = ["--ssl", "-q", "select 1 + 2"]
> result = run_impala_shell_cmd(vector, shell_options, wait_until_connected=False)
> for msg in [self.SSL_ENABLED, self.CONNECTED, self.FETCHED]:
> assert msg in result.stderr
> assert self.DEPRECATION_WARNING not in result.stderr <-----{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org