You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by td...@apache.org on 2017/06/05 21:34:13 UTC
spark git commit: [SPARK-20957][SS][TESTS] Fix
o.a.s.sql.streaming.StreamingQueryManagerSuite listing
Repository: spark
Updated Branches:
refs/heads/master 06c054411 -> bc537e40a
[SPARK-20957][SS][TESTS] Fix o.a.s.sql.streaming.StreamingQueryManagerSuite listing
## What changes were proposed in this pull request?
When stopping StreamingQuery, StreamExecution will set `streamDeathCause` then notify StreamingQueryManager to remove this query. So it's possible that when `q2.exception.isDefined` returns `true`, StreamingQueryManager's active list still has `q2`.
This PR just puts the checks into `eventually` to fix the flaky test.
## How was this patch tested?
Jenkins
Author: Shixiong Zhu <sh...@databricks.com>
Closes #18180 from zsxwing/SPARK-20957.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/bc537e40
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/bc537e40
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/bc537e40
Branch: refs/heads/master
Commit: bc537e40ade0658aae7c6b5ddafb4cc038bdae2b
Parents: 06c0544
Author: Shixiong Zhu <sh...@databricks.com>
Authored: Mon Jun 5 14:34:10 2017 -0700
Committer: Tathagata Das <ta...@gmail.com>
Committed: Mon Jun 5 14:34:10 2017 -0700
----------------------------------------------------------------------
.../apache/spark/sql/streaming/StreamingQueryManagerSuite.scala | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/bc537e40/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala
index b49efa6..2986b7f 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala
@@ -78,9 +78,9 @@ class StreamingQueryManagerSuite extends StreamTest with BeforeAndAfter {
eventually(Timeout(streamingTimeout)) {
require(!q2.isActive)
require(q2.exception.isDefined)
+ assert(spark.streams.get(q2.id) === null)
+ assert(spark.streams.active.toSet === Set(q3))
}
- assert(spark.streams.get(q2.id) === null)
- assert(spark.streams.active.toSet === Set(q3))
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org