You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@livy.apache.org by "TaeYoung Kim (JIRA)" <ji...@apache.org> on 2018/11/05 08:33:00 UTC

[jira] [Created] (LIVY-532) Fail create session with spark,yarn-client mode

TaeYoung Kim created LIVY-532:
---------------------------------

             Summary: Fail create session with spark,yarn-client mode
                 Key: LIVY-532
                 URL: https://issues.apache.org/jira/browse/LIVY-532
             Project: Livy
          Issue Type: Bug
          Components: API, Interpreter, REPL, Server
    Affects Versions: 0.6.0
            Reporter: TaeYoung Kim


I'm using
 * spark2.3
 * livy 0.6.0 snapshot (current master, last commit: 0442eb01aa293d540fdf23ddd86b86c4278b83b9)
 * yarn client mode

I tried add jar in create session params.


```
18/11/05 08:17:16 WARN rsc.RSCClient: Client RPC channel closed unexpectedly.
18/11/05 08:17:16 WARN rsc.RSCClient: Error stopping RPC.
io.netty.util.concurrent.BlockingOperationException: DefaultChannelPromise@638f9c7b(uncancellable)
at io.netty.util.concurrent.DefaultPromise.checkDeadLock(DefaultPromise.java:394)
at io.netty.channel.DefaultChannelPromise.checkDeadLock(DefaultChannelPromise.java:157)
at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:230)
at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:129)
at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:28)
at io.netty.util.concurrent.DefaultPromise.sync(DefaultPromise.java:336)
at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:117)
at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:28)
at org.apache.livy.rsc.rpc.Rpc.close(Rpc.java:310)
at org.apache.livy.rsc.RSCClient.stop(RSCClient.java:232)
at org.apache.livy.rsc.RSCClient$2$1.onSuccess(RSCClient.java:129)
at org.apache.livy.rsc.RSCClient$2$1.onSuccess(RSCClient.java:123)
at org.apache.livy.rsc.Utils$2.operationComplete(Utils.java:108)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:518)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.notifyListenersWithStackOverFlowProtection(DefaultPromise.java:431)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:108)
at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
at io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:995)
at io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:621)
at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:599)
at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:543)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.closeOnRead(AbstractNioByteChannel.java:71)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:158)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:564)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:505)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:419)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:391)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
at java.lang.Thread.run(Thread.java:748)
18/11/05 08:17:16 INFO utils.LineBufferedStream: Exception in thread "main" scala.reflect.internal.FatalError: object Predef does not have a member classOf
18/11/05 08:17:16 INFO rsc.RSCClient: Failing pending job 3091ce65-e328-4c0f-9585-a861b90706d4 due to shutdown.
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass.scala$reflect$internal$Definitions$DefinitionsClass$$fatalMissingSymbol(Definitions.scala:1186)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass.getMember(Definitions.scala:1203)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass.getMemberMethod(Definitions.scala:1238)
18/11/05 08:17:16 INFO interactive.InteractiveSession: Stopping InteractiveSession 2...
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.Predef_classOf$lzycompute(Definitions.scala:1469)
18/11/05 08:17:16 INFO interactive.InteractiveSession: Failed to ping RSC driver for session 2. Killing application.
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.Predef_classOf(Definitions.scala:1469)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.isPredefClassOf(Definitions.scala:1459)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedIdent$2(Typers.scala:4885)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedIdentOrWildcard$1(Typers.scala:4908)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5340)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5360)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5396)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5423)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5370)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5374)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.ReplGlobal$$anon$1$$anon$2.typed(ReplGlobal.scala:36)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5472)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5480)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5012)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5312)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5359)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5396)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5423)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5370)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5374)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.ReplGlobal$$anon$1$$anon$2.typed(ReplGlobal.scala:36)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5448)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.apply(Analyzer.scala:102)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$GlobalPhase$$anonfun$applyPhase$1.apply$mcV$sp(Global.scala:440)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$GlobalPhase.withCurrentUnit(Global.scala:431)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:440)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:94)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:93)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.collection.Iterator$class.foreach(Iterator.scala:893)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.run(Analyzer.scala:93)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1501)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1486)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$Run.compileSources(Global.scala:1481)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.IMain.compileSourcesKeepingRun(IMain.scala:435)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compileAndSaveRun(IMain.scala:855)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compile(IMain.scala:813)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.IMain.bind(IMain.scala:675)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter$$anonfun$bind$1.apply(SparkInterpreter.scala:133)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter$$anonfun$bind$1.apply(SparkInterpreter.scala:133)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter.bind(SparkInterpreter.scala:132)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.AbstractSparkInterpreter.postStart(AbstractSparkInterpreter.scala:72)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter$$anonfun$start$1.apply$mcV$sp(SparkInterpreter.scala:89)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter$$anonfun$start$1.apply(SparkInterpreter.scala:63)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter$$anonfun$start$1.apply(SparkInterpreter.scala:63)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.AbstractSparkInterpreter.restoreContextClassLoader(AbstractSparkInterpreter.scala:340)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter.start(SparkInterpreter.scala:63)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.Session$$anonfun$1.apply(Session.scala:128)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.Session$$anonfun$1.apply(Session.scala:122)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
18/11/05 08:17:16 INFO utils.LineBufferedStream: at java.lang.Thread.run(Thread.java:748)

```

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)