[spark] branch master updated: [SPARK-37985][SQL] Fix flaky test for SPARK-37578

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 3eca309  [SPARK-37985][SQL] Fix flaky test for SPARK-37578
3eca309 is described below

commit 3eca309fa8b70f65b18ceeac75c4e613c18368af
Author: Angerszhuuuu <an...@gmail.com>
AuthorDate: Mon Jan 24 08:52:07 2022 +0900

    [SPARK-37985][SQL] Fix flaky test for SPARK-37578
    
    ### What changes were proposed in this pull request?
    In SQL test `SPARK-37578: Update output metrics from Datasource v2`
    Execution's metric value is not null seem only can promise the execution is not still running.
    But can't promise bytesWrittenListener have received all TaskEnd event, cause test failed.
    
    ### Why are the changes needed?
    Fix flaky test
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    Existed UT
    
    Closes #35277 from AngersZhuuuu/SPARK-37985.
    
    Authored-by: Angerszhuuuu <an...@gmail.com>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 .../org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala    | 1 +
 1 file changed, 1 insertion(+)

diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala
index 8eaeefc..ad74469 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala
@@ -933,6 +933,7 @@ class SQLAppStatusListenerSuite extends SharedSparkSession with JsonTestUtils
             statusStore.executionsList().last.metricValues != null)
         }
 
+        spark.sparkContext.listenerBus.waitUntilEmpty()
         assert(bytesWritten.sum == 246)
         assert(recordsWritten.sum == 20)
       } finally {

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org