You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "David Lewis (JIRA)" <ji...@apache.org> on 2018/06/18 22:08:00 UTC

[jira] [Created] (SPARK-24585) Adding ability to audit file system before and after test to ensure all files are cleaned up.

David Lewis created SPARK-24585:
-----------------------------------

             Summary: Adding ability to audit file system before and after test to ensure all files are cleaned up.
                 Key: SPARK-24585
                 URL: https://issues.apache.org/jira/browse/SPARK-24585
             Project: Spark
          Issue Type: Test
          Components: Build
    Affects Versions: 2.3.1
            Reporter: David Lewis


Some spark tests use temporary files and folders on the file system. This proposal is to audit the file system before and after to ensure that all files created are cleaned up.

This proposal is to add two flags to SparkFunSuite. One to enable the auditing, which will log a warning if there is a discrepancy, and one to throw an exception if there is a discrepancy. 

This is intended to help prevent file leakage in spark.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org