You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "juliuszsompolski (via GitHub)" <gi...@apache.org> on 2023/05/02 11:22:38 UTC

[GitHub] [spark] juliuszsompolski commented on a diff in pull request #41005: [SPARK-43267][CONNECT] Add Spark Connect SparkSession.interruptAll

juliuszsompolski commented on code in PR #41005:
URL: https://github.com/apache/spark/pull/41005#discussion_r1182417710


##########
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/SparkSession.scala:
##########
@@ -525,6 +525,19 @@ class SparkSession private[sql] (
     planIdGenerator.set(0)
   }
 
+  /**
+   * Interrupt all operations of this session currently running on the connected server.
+   *
+   * TODO/WIP: Currently it will interrupt the Spark Jobs running on the server, triggered from
+   * ExecutePlan requests. If an operation is not running a Spark Job, it becomes an noop and the

Review Comment:
   For example if a query just started and is still doing analysis/optimization/planning when the interrupt hits, the interrupt will be a noop and the query will still launch jobs once it gets to execution.
   For example, Delta DML commands often run several stages, each of them runs jobs, with some driver local processing in between.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org