[GitHub] [spark] dongjoon-hyun commented on a diff in pull request #40383: [SPARK-42762][K8S][MINOR] Improve logging in K8s on disconnect when using statefulsets.

["dongjoon-hyun (via GitHub)" <gi...@apache.org>]

dongjoon-hyun commented on code in PR #40383:
URL: https://github.com/apache/spark/pull/40383#discussion_r1133168958


##########
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala:
##########
@@ -294,10 +295,15 @@ private[spark] class KubernetesClusterSchedulerBackend(
   }
 
   private class KubernetesDriverEndpoint extends DriverEndpoint {
+
+    protected val execIDRequester = new HashMap[RpcAddress, String]

Review Comment:
   This looks like monotonically increasing and will cause OOM in JVM in case of dynamic allocation or executor rolling case. What do you think about that, @holdenk ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org