[GitHub] [spark] mridulm commented on a change in pull request #35504: [SPARK-38194][YARN] Make yarn memory overhead factor configurable

[GitBox <gi...@apache.org>]

mridulm commented on a change in pull request #35504:
URL: https://github.com/apache/spark/pull/35504#discussion_r805496558



##########
File path: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala
##########
@@ -325,6 +325,17 @@ package object config extends Logging {
     .bytesConf(ByteUnit.MiB)
     .createWithDefaultString("512m")
 
+  /* Shared driver and executor configuration. */
+
+  private[spark] val MEMORY_OVERHEAD_FACTOR =

Review comment:
       Memory overhead is not specific to yarn, but applies to all resource managers.
   Let us add this to core's `org.apache.spark.internal.package.scala`
   Also, split it by driver and executor as well, to independently control them.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org