Java 11 exception

["Vararu, Vadim" <va...@adswizz.com>]

Hi,

Is there any known issue of not supporting Java 11?

I’m trying to migrate my AWS EMR Spark/Hadoop/Livy environment from Java 8 to 11 but after configuring that, I get such exception coming from Livy deeps:

org.apache.spark.SparkException: Exception thrown in awaitResult:
                at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:301) ~[spark-core_2.12-3.3.0-amzn-1.jar:3.3.0-amzn-1]
                at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:514) ~[spark-yarn_2.12-3.3.0-amzn-1.jar:3.3.0-amzn-1]
                at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:278) ~[spark-yarn_2.12-3.3.0-amzn-1.jar:3.3.0-amzn-1]
                at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:929) ~[spark-yarn_2.12-3.3.0-amzn-1.jar:3.3.0-amzn-1]
                at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:928) ~[spark-yarn_2.12-3.3.0-amzn-1.jar:3.3.0-amzn-1]
                at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
                at javax.security.auth.Subject.doAs(Subject.java:423) ~[?:?]
                at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) ~[hadoop-client-api-3.3.3-amzn-0.jar:?]
                at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:928) ~[spark-yarn_2.12-3.3.0-amzn-1.jar:3.3.0-amzn-1]
                at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala) ~[spark-yarn_2.12-3.3.0-amzn-1.jar:3.3.0-amzn-1]
Caused by: java.util.concurrent.ExecutionException: Boxed Error
                at scala.concurrent.impl.Promise$.resolver(Promise.scala:87) ~[scala-library-2.12.15.jar:?]
                at scala.concurrent.impl.Promise$.scala$concurrent$impl$Promise$$resolveTry(Promise.scala:79) ~[scala-library-2.12.15.jar:?]
                at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:284) ~[scala-library-2.12.15.jar:?]
                at scala.concurrent.Promise.tryFailure(Promise.scala:112) ~[scala-library-2.12.15.jar:?]
                at scala.concurrent.Promise.tryFailure$(Promise.scala:112) ~[scala-library-2.12.15.jar:?]
                at scala.concurrent.impl.Promise$DefaultPromise.tryFailure(Promise.scala:187) ~[scala-library-2.12.15.jar:?]
                at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:761) ~[spark-yarn_2.12-3.3.0-amzn-1.jar:3.3.0-amzn-1]
Caused by: java.lang.IncompatibleClassChangeError: Inconsistent constant pool data in classfile for class org/apache/livy/shaded/json4s/DefaultFormats. Method 'java.text.SimpleDateFormat $anonfun$df$1(org.apache.livy.shaded.json4s.DefaultFormats)' at index 156 is CONSTANT_MethodRef and should be CONSTANT_InterfaceMethodRef
                at org.apache.livy.shaded.json4s.DefaultFormats.$init$(Formats.scala:318) ~[?:?]
                at org.apache.livy.shaded.json4s.DefaultFormats$.<init>(Formats.scala:296) ~[?:?]
                at org.apache.livy.shaded.json4s.DefaultFormats$.<clinit>(Formats.scala) ~[?:?]
                at org.apache.livy.repl.Session.<init>(Session.scala:66) ~[?:?]
                at org.apache.livy.repl.ReplDriver.initializeSparkEntries(ReplDriver.scala:43) ~[?:?]
                at org.apache.livy.rsc.driver.RSCDriver.run(RSCDriver.java:337) ~[?:?]
                at org.apache.livy.rsc.driver.RSCDriverBootstrapper.main(RSCDriverBootstrapper.java:93) ~[?:?]
                at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
                at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
                at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
                at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
                at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:742) ~[spark-yarn_2.12-3.3.0-amzn-1.jar:3.3.0-amzn-1]