Re: [PR] [SPARK-48900] Add `reason` field for all internal calls for job/stage cancellation [spark]

["JoshRosen (via GitHub)" <gi...@apache.org>]

JoshRosen commented on code in PR #47374:
URL: https://github.com/apache/spark/pull/47374#discussion_r1683615195


##########
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala:
##########
@@ -83,10 +83,23 @@ trait ShuffleExchangeLike extends Exchange {
 
   protected def mapOutputStatisticsFuture: Future[MapOutputStatistics]
 
+  /**
+   * Cancels the shuffle job with a reason.
+   */
+  final def cancelShuffleJob(reason: String): Unit = {

Review Comment:
   I'm not a fan of the copy-paste code duplication between the "with reason" and "without reason" overloads and think that we should instead refactor the common code to a `private final def doCancelShuffleJob(reason: Option(reason))`. Within that method, we can do something like
   
   ```scala
   reason match {
     case Some(reason) => action.cancel(reason()
     case None => action.cancel()
   }
   ```
   
   or similar.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org