[GitHub] [spark] JoshRosen commented on a diff in pull request #42742: [SPARK-45025][CORE] Allow block manager memory store iterator to handle thread interrupt and perform task completion gracefully

["JoshRosen (via GitHub)" <gi...@apache.org>]

JoshRosen commented on code in PR #42742:
URL: https://github.com/apache/spark/pull/42742#discussion_r1313513495


##########
core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala:
##########
@@ -220,7 +220,8 @@ private[spark] class MemoryStore(
     }
 
     // Unroll this block safely, checking whether we have exceeded our threshold periodically
-    while (values.hasNext && keepUnrolling) {
+    // and if no thread interrupts have been received.
+    while (values.hasNext && keepUnrolling && !Thread.currentThread().isInterrupted) {

Review Comment:
   > IIRC isInterrupted is a couple of orders more expensive than a boolean check.
   
   In older JDKs it is implemented via an intrinsic and in newer ones (14+) it's just a volatile boolean check (see https://bugs.openjdk.org/browse/JDK-8229516), so I don't think it's too expensive. There's an old StackOverflow answer at https://stackoverflow.com/a/5158441/590203 offering a plausible explanation for why the intrinsic should be cheap.
   
   On JDK 11 on my laptop, I ran a toy JMH benchmark and measured < 1ns per call. 
   
   I think these costs are very small in comparison to the per-element serialization costs.
   
   Given this, I'm not too worried about performance regressions due to this change and think it's probably okay to check on every element rather than every 16th.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org