[GitHub] [spark] Ngone51 commented on a change in pull request #30131: [WIP][SPARK-33220][CORE]Use `scheduleWithFixedDelay` to avoid repeated unnecessary scheduling for a short time

[GitBox <gi...@apache.org>]

Ngone51 commented on a change in pull request #30131:
URL: https://github.com/apache/spark/pull/30131#discussion_r510054542



##########
File path: core/src/main/scala/org/apache/spark/ContextCleaner.scala
##########
@@ -126,7 +126,7 @@ private[spark] class ContextCleaner(
     cleaningThread.setDaemon(true)
     cleaningThread.setName("Spark Context Cleaner")
     cleaningThread.start()
-    periodicGCService.scheduleAtFixedRate(() => System.gc(),
+    periodicGCService.scheduleWithFixedDelay(() => System.gc(),

Review comment:
       This looks like a reasonable change to me.

##########
File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
##########
@@ -134,7 +134,7 @@ class CoarseGrainedSchedulerBackend(scheduler: TaskSchedulerImpl, val rpcEnv: Rp
       // Periodically revive offers to allow delay scheduling to work
       val reviveIntervalMs = conf.get(SCHEDULER_REVIVE_INTERVAL).getOrElse(1000L)
 
-      reviveThread.scheduleAtFixedRate(() => Utils.tryLogNonFatalError {
+      reviveThread.scheduleWithFixedDelay(() => Utils.tryLogNonFatalError {
         Option(self).foreach(_.send(ReviveOffers))

Review comment:
       I think it's meaningless to replace for this kind of task whose intentional action happens asynchronically. For example, in this case, sending the message `ReviveOffers` should finish quickly but the intentional action `reviveOffers` may take a longer time. I guess the *delay* here you want to deal with is from the former one rather than the latter.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org