You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2024/10/11 03:29:57 UTC

(spark) branch master updated: [SPARK-48567][PYTHON][TESTS][FOLLOW-UP] Make the query scope higher so finally can access to it

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 4666972a3190 [SPARK-48567][PYTHON][TESTS][FOLLOW-UP] Make the query scope higher so finally can access to it
4666972a3190 is described below

commit 4666972a31907393390a6bc353e87dfedc6d6445
Author: Hyukjin Kwon <gu...@apache.org>
AuthorDate: Fri Oct 11 12:29:45 2024 +0900

    [SPARK-48567][PYTHON][TESTS][FOLLOW-UP] Make the query scope higher so finally can access to it
    
    ### What changes were proposed in this pull request?
    
    This PR is a followup that fixes the test to recover the build.
    
    ### Why are the changes needed?
    
    To fix up the build.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No, test-only.
    
    ### How was this patch tested?
    
    CI in this PR.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    No.
    
    Closes #48417 from HyukjinKwon/SPARK-48567-followup.
    
    Authored-by: Hyukjin Kwon <gu...@apache.org>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 python/pyspark/sql/tests/streaming/test_streaming_listener.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/python/pyspark/sql/tests/streaming/test_streaming_listener.py b/python/pyspark/sql/tests/streaming/test_streaming_listener.py
index 1f5b0f573807..30d6eee93879 100644
--- a/python/pyspark/sql/tests/streaming/test_streaming_listener.py
+++ b/python/pyspark/sql/tests/streaming/test_streaming_listener.py
@@ -216,6 +216,7 @@ class StreamingListenerTestsMixin:
             def onQueryTerminated(self, event):
                 pass
 
+        q = None
         try:
             error_listener = MyErrorListener()
             self.spark.streams.addListener(error_listener)
@@ -238,10 +239,12 @@ class StreamingListenerTestsMixin:
             self.assertTrue(error_listener.num_error_rows > 0)
 
         finally:
-            q.stop()
+            if q is not None:
+                q.stop()
             self.spark.streams.removeListener(error_listener)
 
     def test_streaming_progress(self):
+        q = None
         try:
             # Test a fancier query with stateful operation and observed metrics
             df = self.spark.readStream.format("rate").option("rowsPerSecond", 10).load()
@@ -265,7 +268,8 @@ class StreamingListenerTestsMixin:
                 self.check_streaming_query_progress(p, True)
 
         finally:
-            q.stop()
+            if q is not None:
+                q.stop()
 
 
 class StreamingListenerTests(StreamingListenerTestsMixin, ReusedSQLTestCase):


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org