Re: [PR] [WIP][SPARK-50357][PYTHON] Support Interrupt(Tag|All) APIs for PySpark [spark]

["itholic (via GitHub)" <gi...@apache.org>]

itholic commented on code in PR #49014:
URL: https://github.com/apache/spark/pull/49014#discussion_r1867208405


##########
sql/core/src/test/scala/org/apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala:
##########
@@ -330,4 +330,109 @@ class SparkSessionJobTaggingAndCancellationSuite
       threadPool.shutdownNow()
     }
   }
+
+  test("Interrupt jobs by tag") {

Review Comment:
   Thanks for the context @xupefei !
   
   Is it normal? or should we made an additional fix from Scala side(or Python side) to make this test work? (because the corresponding test is working on Python side so anyway I think we should match the behavior by fixing one side)



##########
sql/core/src/test/scala/org/apache/spark/sql/SparkSessionJobTaggingAndCancellationSuite.scala:
##########
@@ -330,4 +330,109 @@ class SparkSessionJobTaggingAndCancellationSuite
       threadPool.shutdownNow()
     }
   }
+
+  test("Interrupt jobs by tag") {

Review Comment:
   Hi, @xupefei . I add two test cases here to ensure consistency with testing on the PySpark side (which are defined `python/pyspark/sql/tests/test_job_cancellation.py` in this PR) but seems like they are not working as I expected.
   
   Could you take a look at these new tests to confirm if I missed something or should we actually made a fix for core side first?? Thanks!
   
   Also cc @HyukjinKwon 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org