[jira] [Commented] (IMPALA-11941) Support Java 17 in Impala

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-11941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736712#comment-17736712 ] 

ASF subversion and git services commented on IMPALA-11941:
----------------------------------------------------------

Commit 3b0705ba63c7eb6910829b5a7a7feb8f6d8e5607 in impala's branch refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=3b0705ba6 ]

IMPALA-11941: Support Java 17 in Impala

Enables building for Java 17 - and particularly using Java 17 in
containers - but won't run a minicluster fully with Java 17 as some
projects (Hadoop) don't yet support it.

Starting with Java 15, ehcache.sizeof encounters
UnsupportedOperationException: can't get field offset on a hidden class
in class members pointing to capturing lambda functions. Java 17 also
introduces new modules that need to be added to add-opens. Both of these
pose problems for continued use of ehcache.

Adds https://github.com/jbellis/jamm as a new cache weigher for Java
15+. We build from HEAD as an external project until Java 17 support is
released (https://github.com/jbellis/jamm/issues/44). Adds the
'java_weigher' option to select 'sizeof' or 'jamm'; defaults to 'auto',
which uses jamm for Java 15+ and sizeof for everything else. Also adds
metrics for viewing cache weight results.

Adds JAVA_HOME/lib/server to LD_LIBRARY_PATH in run-jvm-binary to
simplify switching between JDK versions for testing. You can now
- export IMPALA_JDK_VERSION=11
- source bin/impala-config.sh
- start-impala-cluster.py
and have Impala running a different JDK (11) version.

Retains add-opens calls that are still necessary due to dependencies'
use of lambdas for jamm, and all others for ehcache. Add-opens are still
required as a fallback, as noted in
https://github.com/jbellis/jamm#object-graph-crawling. We catch the
exceptions jamm and ehcache throw - CannotAccessFieldException,
UnsupportedOperationException - to avoid crashing Impala, and add it to
the list of banned log messages (as we should add-opens when we find
them).

Testing:
- container test run with Java 11 and 17 (excludes custom cluster)
- manual custom_cluster/test_local_catalog.py +
  test_banned_log_messages.py run with Java 11 and 17 (Java 8 build)
- full Java 11 build (passed except IMPALA-12184)
- add test catalog cache entry size metrics fit reasonable bounds
- add unit test for utility to find jamm jar file in classpath

Change-Id: Ic378896f572e030a3a019646a96a32a07866a737
Reviewed-on: http://gerrit.cloudera.org:8080/19863
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Support Java 17 in Impala
> -------------------------
>
>                 Key: IMPALA-11941
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11941
>             Project: IMPALA
>          Issue Type: Improvement
>            Reporter: Manish Maheshwari
>            Assignee: Michael Smith
>            Priority: Major
>
> Support Java 17 in Impala.
> ----
> jdeprscan identifies one new deprecation to address
> {code}
> $ /usr/lib/jvm/java-17-openjdk-amd64/bin/jdeprscan --release 17 --class-path "$CLASSPATH" fe/target/impala-frontend-4.3.0-SNAPSHOT.jar
> Jar file fe/target/impala-frontend-4.3.0-SNAPSHOT.jar:
> class org/apache/impala/util/UnsafeUtil uses deprecated class java/security/AccessController (forRemoval=true)
> {code}
> This is related to our use of sun.misc.Unsafe. However an appropriate replacement - https://openjdk.org/jeps/193 - was only added in Java 9.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org