[jira] [Created] (SPARK-5423) ExternalAppendOnlyMap won't delete temp spilled file if some exception happens during using it

["Shixiong Zhu (JIRA)" <ji...@apache.org>]

Shixiong Zhu created SPARK-5423:
-----------------------------------

             Summary: ExternalAppendOnlyMap won't delete temp spilled file if some exception happens during using it 
                 Key: SPARK-5423
                 URL: https://issues.apache.org/jira/browse/SPARK-5423
             Project: Spark
          Issue Type: Improvement
            Reporter: Shixiong Zhu
            Priority: Minor


ExternalAppendOnlyMap won't delete temp spilled file if some exception happens during using it.

There is already a TODO in the comment:
{code}
    // TODO: Ensure this gets called even if the iterator isn't drained.
    private def cleanup() {
      batchIndex = batchOffsets.length  // Prevent reading any other batch
      val ds = deserializeStream
      deserializeStream = null
      fileStream = null
      ds.close()
      file.delete()
    }
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org