You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by zs...@apache.org on 2017/03/10 07:02:16 UTC
spark git commit: [SPARK-19891][SS] Await Batch Lock notified on
stream execution exit
Repository: spark
Updated Branches:
refs/heads/master 5949e6c44 -> 501b71119
[SPARK-19891][SS] Await Batch Lock notified on stream execution exit
## What changes were proposed in this pull request?
We need to notify the await batch lock when the stream exits early e.g., when an exception has been thrown.
## How was this patch tested?
Current tests that throw exceptions at runtime will finish faster as a result of this update.
zsxwing
Please review http://spark.apache.org/contributing.html before opening a pull request.
Author: Tyson Condie <tc...@gmail.com>
Closes #17231 from tcondie/kafka-writer.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/501b7111
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/501b7111
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/501b7111
Branch: refs/heads/master
Commit: 501b7111997bc74754663348967104181b43319b
Parents: 5949e6c
Author: Tyson Condie <tc...@gmail.com>
Authored: Thu Mar 9 23:02:13 2017 -0800
Committer: Shixiong Zhu <sh...@databricks.com>
Committed: Thu Mar 9 23:02:13 2017 -0800
----------------------------------------------------------------------
.../spark/sql/execution/streaming/StreamExecution.scala | 7 +++++++
1 file changed, 7 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/501b7111/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala
index 70912d1..5292638 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala
@@ -361,6 +361,13 @@ class StreamExecution(
}
}
} finally {
+ awaitBatchLock.lock()
+ try {
+ // Wake up any threads that are waiting for the stream to progress.
+ awaitBatchLockCondition.signalAll()
+ } finally {
+ awaitBatchLock.unlock()
+ }
terminationLatch.countDown()
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org