[GitHub] [spark] tgravescs commented on a change in pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

[GitBox <gi...@apache.org>]

tgravescs commented on a change in pull request #28972:
URL: https://github.com/apache/spark/pull/28972#discussion_r450363633



##########
File path: core/src/main/scala/org/apache/spark/resource/ExecutorResourceRequests.scala
##########
@@ -54,6 +54,19 @@ class ExecutorResourceRequests() extends Serializable {
     this
   }
 
+  /**
+   * Specify off heap memory. The value specified will be converted to MiB.

Review comment:
       that is only in the getDefaultExecutorResources function, which creates the default profile from the standard set of configs.  If the user creates a new ResourceProfile and called this function directly to set off heap memory, there is nothing checking the config MEMORY_OFFHEAP_ENABLED and on the yarn side your change doesn't check ti either, it just picks up whatever value is set here.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org