[GitHub] spark pull request #19135: [SPARK-21923][CORE]Avoid calling reserveUnrollMem...

[cloud-fan <gi...@git.apache.org>]

Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19135#discussion_r138822554
  
    --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala ---
    @@ -325,6 +325,10 @@ private[spark] class MemoryStore(
     
         // Whether there is still enough memory for us to continue unrolling this block
         var keepUnrolling = true
    +    // Number of elements unrolled so far
    +    var elementsUnrolled = 0L
    +    // How often to check whether we need to request more memory
    +    val memoryCheckPeriod = 16
    --- End diff --
    
    This is the same approach we used in `putIteratorAsValues`. One thing still missing is `memoryGrowthFactor`, can we add that too?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org