JAR upload required?

[Patrick Kasarski <pa...@semanticbits.com>]

Hi,

Is it possible to submit Scala code without uploading a JAR? I have tried
the following:

object LivySubmission {
  def main(args: Array[String]): Unit = {
    val client = new LivyClientBuilder(false)
      .setURI(new URI("http://localhost:8998/"))
      .build()
      .asScalaClient

    val job = client.run { context =>
      context.createStreamingContext(15000)
      1 + 1
    }

    job.onSuccess { case r => Console.println("1 + 1 = " + r) }
  }
}

Result is:
 "log": [
   "Caused by: java.lang.ClassNotFoundException:
org.apache.livy.scalaapi.LivyScalaClient$$anon$2",
   "\tat java.net.URLClassLoader.findClass(URLClassLoader.java:382)",
   "\tat java.lang.ClassLoader.loadClass(ClassLoader.java:418)",
   "\tat java.lang.ClassLoader.loadClass(ClassLoader.java:351)",
   "\tat java.lang.Class.forName0(Native Method)",
   "\tat java.lang.Class.forName(Class.java:348)",
   "\tat
org.apache.livy.shaded.kryo.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:136)",
   "\t... 40 more",
   "21/03/22 23:21:30 INFO ExecutorAllocationManager: Request to remove
executorIds: 1",
   "\nYARN Diagnostics: "
 ]

It looks like it's trying to load the anonymous function submitted to
client.run and can't find it, is that correct?

Regards,
Patrick