[GitHub] spark pull request #20862: [SPARK-23744][CORE]Fix memory leak in ReadableCha...

[kiszk <gi...@git.apache.org>]

Github user kiszk commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20862#discussion_r178623333
  
    --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala ---
    @@ -301,7 +301,10 @@ private class ReadableChannelFileRegion(source: ReadableByteChannel, blockSize:
         written
       }
     
    -  override def deallocate(): Unit = source.close()
    +  override def deallocate() {
    +    source.close()
    +    StorageUtils.dispose(buffer)
    --- End diff --
    
    Sorry for being late since I overlooked.
    
    While `ByteBuffer.allocateDirect` allocates memory in off-heap memory, the memory will be also reclaimed when the ByteBuffer object will be reclaimed.
    Regarding `-XX:+DisableExplicitGC`, as described [here](http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html), this option just disables `System.gc()`. Other GC operations may reclaim the memory in off-heap allocated by `ByteBuffer.allocateDirect`.
    >By default calls to System.gc() are enabled (-XX:-DisableExplicitGC). Use -XX:+DisableExplicitGC to disable calls to System.gc(). Note that the JVM still performs garbage collection when necessary.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org