You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Burak Yavuz (Jira)" <ji...@apache.org> on 2019/12/05 22:34:00 UTC

[jira] [Created] (SPARK-30143) StreamingQuery.stop() should not block indefinitely

Burak Yavuz created SPARK-30143:
-----------------------------------

             Summary: StreamingQuery.stop() should not block indefinitely
                 Key: SPARK-30143
                 URL: https://issues.apache.org/jira/browse/SPARK-30143
             Project: Spark
          Issue Type: Improvement
          Components: Structured Streaming
    Affects Versions: 2.4.4
            Reporter: Burak Yavuz


The stop() method on a Streaming Query awaits the termination of the stream execution thread. However, the stream execution thread may block forever depending on the streaming source implementation (like in Kafka, which runs UninterruptibleThreads).

This causes control flow applications to hang indefinitely as well. We'd like to introduce a timeout to stop the execution thread, so that the control flow thread can decide to do an action if a timeout is hit. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org