You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by an...@apache.org on 2015/05/08 23:09:06 UTC
spark git commit: [MINOR] Defeat early garbage collection of test
suite variable
Repository: spark
Updated Branches:
refs/heads/master 4e7360e12 -> 31da40dfe
[MINOR] Defeat early garbage collection of test suite variable
The JVM is free to collect references to variables that no longer participate in a computation. This simple patch adds an operation to the variable 'rdd' to ensure it is not collected early in the test suite's explicit calls to GC.
ref: http://bugs.java.com/view_bug.do?bug_id=6721588
Author: Tim Ellison <t....@gmail.com>
Closes #6010 from tellison/master and squashes the following commits:
77d1c8f [Tim Ellison] Defeat early garbage collection of test suite variable by aggressive JVMs
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/31da40df
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/31da40df
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/31da40df
Branch: refs/heads/master
Commit: 31da40dfeeeab69ee7974992328e3f67046ad3da
Parents: 4e7360e
Author: Tim Ellison <t....@gmail.com>
Authored: Fri May 8 14:08:52 2015 -0700
Committer: Andrew Or <an...@databricks.com>
Committed: Fri May 8 14:08:58 2015 -0700
----------------------------------------------------------------------
core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/31da40df/core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala
----------------------------------------------------------------------
diff --git a/core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala b/core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala
index c7868dd..cb30e1f 100644
--- a/core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala
+++ b/core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala
@@ -183,6 +183,7 @@ class ContextCleanerSuite extends ContextCleanerSuiteBase {
}
// Test that GC causes shuffle cleanup after dereferencing the RDD
+ rdd.count() // Defeat any early collection of rdd variable by the JVM
val postGCTester = new CleanerTester(sc, shuffleIds = Seq(0))
rdd = null // Make RDD out of scope, so that corresponding shuffle goes out of scope
runGC()
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org