[spark] branch master updated: [SPARK-39869][SQL][TESTS] Fix flaky hive - slow tests because of out-of-memory

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new c4c623a3a89 [SPARK-39869][SQL][TESTS] Fix flaky hive - slow tests because of out-of-memory
c4c623a3a89 is described below

commit c4c623a3a890267b2f9f8d472c8be30fc5db53e1
Author: Hyukjin Kwon <gu...@apache.org>
AuthorDate: Tue Jul 26 18:37:20 2022 +0900

    [SPARK-39869][SQL][TESTS] Fix flaky hive - slow tests because of out-of-memory
    
    ### What changes were proposed in this pull request?
    
    This PR adds some manual `System.gc`. I know enough that this doesn't guarantee the garbage collection and sounds somewhat funny but it works in my experience so far, and I did such hack in some places before.
    
    ### Why are the changes needed?
    
    To deflake the tests.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No, dev and test-only.
    
    ### How was this patch tested?
    
    CI in this PR should test it out.
    
    Closes #37291 from HyukjinKwon/SPARK-39869.
    
    Authored-by: Hyukjin Kwon <gu...@apache.org>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 .../apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala | 5 +++++
 .../test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala  | 1 +
 2 files changed, 6 insertions(+)

diff --git a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala
index ecbe87b163d..debae3ad520 100644
--- a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala
+++ b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala
@@ -1388,6 +1388,11 @@ abstract class HiveThriftServer2TestBase extends SparkFunSuite with BeforeAndAft
        """.stripMargin)
   }
 
+  override def beforeEach(): Unit = {
+    super.beforeEach()
+    System.gc()
+  }
+
   override protected def beforeAll(): Unit = {
     super.beforeAll()
     diagnosisBuffer.clear()
diff --git a/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala b/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala
index 170cf4898f3..18202307fc4 100644
--- a/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala
+++ b/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala
@@ -63,6 +63,7 @@ class HiveSparkSubmitSuite
 
   override def beforeEach(): Unit = {
     super.beforeEach()
+    System.gc()
   }
 
   test("temporary Hive UDF: define a UDF and use it") {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org