[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #42403: [SPARK-44726][CORE] Improve `HeartbeatReceiver` config validation error message

["dongjoon-hyun (via GitHub)" <gi...@apache.org>]

dongjoon-hyun commented on code in PR #42403:
URL: https://github.com/apache/spark/pull/42403#discussion_r1287917895


##########
core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala:
##########
@@ -88,9 +88,15 @@ private[spark] class HeartbeatReceiver(sc: SparkContext, clock: Clock)
 
   private val executorHeartbeatIntervalMs = sc.conf.get(config.EXECUTOR_HEARTBEAT_INTERVAL)
 
-  require(checkTimeoutIntervalMs <= executorTimeoutMs,
-    s"${Network.NETWORK_TIMEOUT_INTERVAL.key} should be less than or " +
-      s"equal to ${config.STORAGE_BLOCKMANAGER_HEARTBEAT_TIMEOUT.key}.")
+  if (sc.conf.get(config.STORAGE_BLOCKMANAGER_HEARTBEAT_TIMEOUT).isEmpty) {
+    require(checkTimeoutIntervalMs <= executorTimeoutMs,
+      s"${Network.NETWORK_TIMEOUT_INTERVAL.key} should be less than or " +
+        s"equal to ${Network.NETWORK_TIMEOUT.key}.")
+  } else {
+    require(checkTimeoutIntervalMs <= executorTimeoutMs,
+      s"${Network.NETWORK_TIMEOUT_INTERVAL.key} should be less than or " +
+        s"equal to ${config.STORAGE_BLOCKMANAGER_HEARTBEAT_TIMEOUT.key}.")
+  }
   require(executorHeartbeatIntervalMs <= executorTimeoutMs,
     s"${config.EXECUTOR_HEARTBEAT_INTERVAL.key} should be less than or " +
       s"equal to ${config.STORAGE_BLOCKMANAGER_HEARTBEAT_TIMEOUT.key}")

Review Comment:
   Thank you for review. No, we don't need it because it's already checked `SparkConf.validateSettings`.
   
   https://github.com/apache/spark/blob/eccc045a250818db5e0cfe7d00ac6dcae1ac3d7e/core/src/main/scala/org/apache/spark/SparkConf.scala#L566-L573
   
   Here is the existing Spark error message.
   ```
   $ bin/spark-shell -c spark.executor.heartbeatInterval=1000s
   ...
   java.lang.IllegalArgumentException: requirement failed: The value of spark.network.timeout=120000ms must be greater than the value of spark.executor.heartbeatInterval=1000000ms.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org