[I] Make and keep production and test code working with "up to date" Java versions [polaris]

["snazy (via GitHub)" <gi...@apache.org>]

snazy opened a new issue, #727:
URL: https://github.com/apache/polaris/issues/727

   ### Describe the bug
   
   Polaris, as a server, should be able run run recent Java versions for various reasons (GC improvements, heap usage improvements, OS integration improvements, etc etc etc).
   
   "Client" libraries (think: Iceberg, Spark, Hadoop) however are known to cause problems with "newest" Java versions and/or recent library versions for various reasons.
   
   Polaris relies on Hadoop and Spark for its tests and we are therefore limited to only run on Java versions that those libraries support / work with. It usually takes a lot of time until all these libraries are all compatible with "newer" Java versions - often lagging behind a few Java versions.
   
   ## Goals
   
   1. Tests should not rely on Hadoop or Spark, at best not even Iceberg, but only on Polaris code
   2. If tests (think: integration tests) really need Hadoop and/or Spark, those should run separately using a "compatible" Java version to allow Polaris to leverage modern Java versions.
   
   ## Polaris tests using Hadoop fail with Java 23
   
   Background: The whole Java `SecurityManager` stuff was deprecated in Java 17 via [JEP 411](https://openjdk.org/jeps/411) (April 2021). Alternatives (`Subject.getSubject()` --> `Subject.current()`) have been [introduced with Java 18](https://bugs.openjdk.org/browse/JDK-8267108).
   
   [JEP 486](https://openjdk.org/jeps/486) is about to _permanently_ disable the security manager, even the workaround to set `java.security.manager=allow` will not work beginning with Java 24 (the JVM won't start).
   
   There's [HADOOP-19212](https://issues.apache.org/jira/browse/HADOOP-19212) (opened June 2024) with sadly rather [stale PR](https://github.com/apache/hadoop/pull/7081).
   
   Workaround for Java 23: #723
   
   
   ### To Reproduce
   
   _No response_
   
   ### Actual Behavior
   
   _No response_
   
   ### Expected Behavior
   
   _No response_
   
   ### Additional context
   
   _No response_
   
   ### System information
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org