[GitHub] [spark] srowen commented on a change in pull request #34846: [SPARK-37593][CORE] Optimize HeapMemoryAllocator to avoid memory waste when using G1GC

[GitBox <gi...@apache.org>]

srowen commented on a change in pull request #34846:
URL: https://github.com/apache/spark/pull/34846#discussion_r769744747



##########
File path: core/src/main/scala/org/apache/spark/memory/MemoryManager.scala
##########
@@ -244,17 +245,26 @@ private[spark] abstract class MemoryManager(
    * and then divide it by a factor of safety.
    */
   val pageSizeBytes: Long = {
-    val minPageSize = 1L * 1024 * 1024   // 1MB
-    val maxPageSize = 64L * minPageSize  // 64MB
-    val cores = if (numCores > 0) numCores else Runtime.getRuntime.availableProcessors()
-    // Because of rounding to next power of 2, we may have safetyFactor as 8 in worst case
-    val safetyFactor = 16
-    val maxTungstenMemory: Long = tungstenMemoryMode match {
-      case MemoryMode.ON_HEAP => onHeapExecutionMemoryPool.poolSize
-      case MemoryMode.OFF_HEAP => offHeapExecutionMemoryPool.poolSize
+    val default = Utils.G1HeapRegionSize match {

Review comment:
       Would lazy do anything here? it's a local inside what's evaluated for a val. I could see declaring it lazily outside this block instead. Then even being lazy doesn't matter




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org