You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2021/09/15 06:58:00 UTC
[jira] [Assigned] (SPARK-36764) Fix race-condition on "ensure
continuous stream is being used" in KafkaContinuousTest
[ https://issues.apache.org/jira/browse/SPARK-36764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-36764:
------------------------------------
Assignee: (was: Apache Spark)
> Fix race-condition on "ensure continuous stream is being used" in KafkaContinuousTest
> -------------------------------------------------------------------------------------
>
> Key: SPARK-36764
> URL: https://issues.apache.org/jira/browse/SPARK-36764
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 2.4.8, 3.0.3, 3.1.2, 3.2.0
> Reporter: Jungtaek Lim
> Priority: Major
>
> The test “ensure continuous stream is being used“ in KafkaContinuousTestquickly checks the actual type of the execution, and stop the query. Stopping the streaming query in continuous mode is done by interrupting query execution thread and join indefinitely.
> In parallel, started streaming query is going to generate execution plan, including running optimizer. Some parts of SessionState can be built at that time, as they are defined as lazy. The problem is, some of them seem to be able to “swallow” the InterruptedException and let the thread run continuously.
> That said, the query can’t indicate whether there is a request on stopping query, so the query won’t stop.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org