[GitHub] spark pull request #23083: [SPARK-26114][CORE] ExternalSorter's readingItera...

[advancedxy <gi...@git.apache.org>]

Github user advancedxy commented on a diff in the pull request:

    https://github.com/apache/spark/pull/23083#discussion_r236060023
  
    --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala ---
    @@ -727,9 +727,10 @@ private[spark] class ExternalSorter[K, V, C](
         spills.clear()
         forceSpillFiles.foreach(s => s.file.delete())
         forceSpillFiles.clear()
    -    if (map != null || buffer != null) {
    +    if (map != null || buffer != null || readingIterator != null) {
           map = null // So that the memory can be garbage-collected
           buffer = null // So that the memory can be garbage-collected
    +      readingIterator = null // So that the memory can be garbage-collected
    --- End diff --
    
    Nice. Case well explained.
    
    But I think you need to add corresponding test cases for `CompletionIterator` and `ExternalSorter`.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org