[GitHub] [incubator-livy] onedigit edited a comment on pull request #170: [LiVY-590] Add dependency to jersey-core

[GitBox <gi...@apache.org>]

onedigit edited a comment on pull request #170:
URL: https://github.com/apache/incubator-livy/pull/170#issuecomment-703122226


   This problem persists in the current master.
   ```
           at org.apache.livy.utils.SparkYarnApp$.yarnClient(SparkYarnApp.scala:52)
           at org.apache.livy.utils.SparkYarnApp$$anon$1.run(SparkYarnApp.scala:78)
   Caused by: java.lang.ClassNotFoundException: javax.ws.rs.ext.MessageBodyReader
           at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
           at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
           at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
           at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
           ... 54 more
   java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.yarn.util.timeline.TimelineUtils
           at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:200)
           at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
           at org.apache.livy.utils.SparkYarnApp$.yarnClient$lzycompute(SparkYarnApp.scala:54)
   ```
   And then I see the exact same issue that @akitanaka describes, i.e. the Livy session remains in `starting` state.
   
   Even if, `jersey-core-1.19.jar` is added, the problem won't be solved, since that jar no longer includes the missing class. That is
   ```
   jar tvf jersey-core-1.19.jar | grep javax.ws.rs.ext.MessageBodyReader
     1763 Thu Nov 21 07:17:18 UTC 2013 META-INF/services/javax.ws.rs.ext.MessageBodyReader
   ```
   However, If we look inside the `jersey-core-1.9.jar`, we see that the missing class is there:
   ```
   jar tvf jersey-core-1.9.jar  | grep javax.ws.rs.ext.MessageBodyReader
     1763 Fri Sep 02 11:16:04 UTC 2011 META-INF/services/javax.ws.rs.ext.MessageBodyReader
      950 Fri Sep 02 11:16:40 UTC 2011 javax/ws/rs/ext/MessageBodyReader.class
   ```
   In order to keep `jersey-core-1.19.jar` that's required by the thriftserver and get Livy server working, we need to add the right version of `jsr311-api` jar.  For example, Hadoop 3.3.0 now includes `jsr311-api-1.1.1.jar`.  This is the jar that now contains the required class:
   ```
   jar tvf jsr311-api-1.1.1.jar | grep javax.ws.rs.ext.MessageBodyReader
      950 Mon Nov 09 13:45:50 UTC 2009 javax/ws/rs/ext/MessageBodyReader.class
   ```
   If I manually add this jar to the class path of the Livy server, then it works as expected.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org