[jira] [Resolved] (SPARK-52731) Fix flaky test_listener_events in 'pyspark.sql.tests.streaming.test_streaming_listener'

["Attila Zsolt Piros (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-52731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Attila Zsolt Piros resolved SPARK-52731.
----------------------------------------
    Fix Version/s: 4.1.0
       Resolution: Fixed

Issue resolved by pull request 51423
[https://github.com/apache/spark/pull/51423]

> Fix flaky test_listener_events in 'pyspark.sql.tests.streaming.test_streaming_listener'
> ---------------------------------------------------------------------------------------
>
>                 Key: SPARK-52731
>                 URL: https://issues.apache.org/jira/browse/SPARK-52731
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark
>    Affects Versions: 4.1.0
>            Reporter: Srinivas Rishindra Pothireddi
>            Assignee: Srinivas Rishindra Pothireddi
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 4.1.0
>
>
> There is flakiness in the method 'test_listener_events' in test 'pyspark.sql.tests.streaming.test_streaming_listener'. This is because the spark events are not being reset between streaming queries. The was working most of the times because there was time delay and in the meantime if the second query executes then the terminated_event is being reset. But if the query fails to execute within the sleep period, then the old event is used. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org