Re: [PR] [WIP][SPARK-50357][PYTHON] Support Interrupt(Tag|All) APIs for PySpark [spark]

["xupefei (via GitHub)" <gi...@apache.org>]

xupefei commented on code in PR #49014:
URL: https://github.com/apache/spark/pull/49014#discussion_r1865906758


##########
sql/core/src/test/scala/org/apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala:
##########
@@ -330,4 +330,109 @@ class SparkSessionJobTaggingAndCancellationSuite
       threadPool.shutdownNow()
     }
   }
+
+  test("Interrupt jobs by tag") {

Review Comment:
   After [this PR](https://github.com/apache/spark/pull/48906) tags are isolated per session: tags added my the main thread can be seen from child threads.
   
   In this test case, `session.interruptTag("tag_A")` (which runs in the main thread) won't see `session.addTag(tag)` (runs in a child* thread).
   
   *) Actually when working on the aforementioned PR, I found that threads from a Pool are not considered a "child" thread and won't see tags. I have to manually create threads instead: https://github.com/apache/spark/pull/48906/files#diff-647cd00e281c3cccb76a42646a692abaef6738c8157daf6642bc8fa16e233394R168
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org