[GitHub] [spark] juliuszsompolski commented on a diff in pull request #30481: [SPARK-33526][SQL] Add config to control if cancel invoke interrupt task on thriftserver

["juliuszsompolski (via GitHub)" <gi...@apache.org>]

juliuszsompolski commented on code in PR #30481:
URL: https://github.com/apache/spark/pull/30481#discussion_r1181684437


##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -926,13 +926,23 @@ object SQLConf {
       .booleanConf
       .createWithDefault(false)
 
+  val THRIFTSERVER_FORCE_CANCEL =
+    buildConf("spark.sql.thriftServer.interruptOnCancel")
+      .doc("When true, all running tasks will be interrupted if one cancels a query. " +
+        "When false, all running tasks will remain until finished.")
+      .version("3.2.0")
+      .booleanConf
+      .createWithDefault(false)

Review Comment:
   https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala#L134 and https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala#L270 unconditionally set interruptOnCancel=true for broadcasts and for streaming queries. https://issues.apache.org/jira/browse/HDFS-1208 remains open, but it seems there's not been issues resulting from using true in the other places.
   
   Given that, do we need to bother protecting it with a config for Thriftserver, or at least this config could be flipped to be enabled by default? Having it default to false limits the default cancellation behaviour.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org