[GitHub] [incubator-livy] jahstreet commented on a change in pull request #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes

[GitBox <gi...@apache.org>]

jahstreet commented on a change in pull request #167: [LIVY-588][WIP]: Full support for Spark on Kubernetes
URL: https://github.com/apache/incubator-livy/pull/167#discussion_r302875798
 
 

 ##########
 File path: server/src/main/scala/org/apache/livy/server/interactive/InteractiveSession.scala
 ##########
 @@ -467,6 +478,7 @@ class InteractiveSession(
       transition(SessionState.ShuttingDown)
       sessionStore.remove(RECOVERY_SESSION_TYPE, id)
       client.foreach { _.stop(true) }
+      if(livyConf.isRunningOnKubernetes()) app.foreach(_.kill())
 
 Review comment:
   From the high level - yes `it is to clean up the completed driver pod when the interactive session terminates`.
   Here could be nice to make clear what does it mean `interactive session terminates`:
   - User stops session
   - GC stops session
   - Failure occurs
   In each of these cases stopSession is called. And you don't see Interactive Session Driver pod in a Completed state cause it is always Running and waiting for statements to execute until it actually `terminates`. And that line is the only place where Interactive Session Driver pod gets deleted from the cluster.
   
   For Batch Sessions it is already done so: https://github.com/apache/incubator-livy/blob/master/server/src/main/scala/org/apache/livy/server/batch/BatchSession.scala#L174
   Batch Session Driver goes to completed state cause its entrypoint job gets finished. And stopSession here gets triggered on GC.
   
   Hope it helps.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services