You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by zsxwing <gi...@git.apache.org> on 2016/12/01 21:47:36 UTC
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/16109
[SPARK-18588][Tests]Fix flaky test: KafkaSourceStressForDontFailOnDataLossSuite
## What changes were proposed in this pull request?
Fixed the following failures:
```
org.scalatest.exceptions.TestFailedDueToTimeoutException: The code passed to eventually never returned normally. Attempted 3745 times over 1.0000790851666665 minutes. Last failure message: assertion failed: failOnDataLoss-0 not deleted after timeout.
```
```
sbt.ForkMain$ForkError: org.apache.spark.sql.streaming.StreamingQueryException: Query query-66 terminated with exception: null
at org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runBatches(StreamExecution.scala:252)
at org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:146)
Caused by: sbt.ForkMain$ForkError: java.lang.NullPointerException: null
at java.util.ArrayList.addAll(ArrayList.java:577)
at org.apache.kafka.clients.Metadata.getClusterForCurrentTopics(Metadata.java:257)
at org.apache.kafka.clients.Metadata.update(Metadata.java:177)
at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.handleResponse(NetworkClient.java:605)
at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeHandleCompletedReceive(NetworkClient.java:582)
at org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:450)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:269)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:360)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:192)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.awaitPendingRequests(ConsumerNetworkClient.java:260)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:222)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.ensurePartitionAssignment(ConsumerCoordinator.java:366)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:978)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:938)
at
...
```
## How was this patch tested?
Tested in #16048 by running many times.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zsxwing/spark fix-kafka-flaky-test
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16109.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16109
----
commit 0901a65fadc7053bac325552bd1822108a246e75
Author: Shixiong Zhu <sh...@databricks.com>
Date: 2016-11-29T01:02:58Z
Fix flaky test: KafkaSourceStressForDontFailOnDataLossSuite
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3462 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3462/consoleFull)** for PR 16109 at commit [`8c5ae2d`](https://github.com/apache/spark/commit/8c5ae2d4b123f7872365cc2e27f2c93cdd74af4e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3452 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3452/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69508 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69508/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69784 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69784/consoleFull)** for PR 16109 at commit [`c9d5ba9`](https://github.com/apache/spark/commit/c9d5ba9fb841afe3d4e400855cfc2c93e110dfde).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3463 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3463/consoleFull)** for PR 16109 at commit [`8c5ae2d`](https://github.com/apache/spark/commit/8c5ae2d4b123f7872365cc2e27f2c93cdd74af4e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69784 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69784/consoleFull)** for PR 16109 at commit [`c9d5ba9`](https://github.com/apache/spark/commit/c9d5ba9fb841afe3d4e400855cfc2c93e110dfde).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69752 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69752/consoleFull)** for PR 16109 at commit [`20b01d4`](https://github.com/apache/spark/commit/20b01d4819e3236432d026f95698371a41e3e6df).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69598/consoleFull)** for PR 16109 at commit [`8c5ae2d`](https://github.com/apache/spark/commit/8c5ae2d4b123f7872365cc2e27f2c93cdd74af4e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/16109#discussion_r91210556
--- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala ---
@@ -286,36 +286,56 @@ class KafkaTestUtils extends Logging {
props
}
+ /** Assert topic is deleted in all places, e.g, brokers, zookeeper. */
+ private def assertTopicDeleted(
+ topic: String,
+ numPartitions: Int,
+ servers: Seq[KafkaServer]): Unit = {
+ val topicAndPartitions = (0 until numPartitions).map(TopicAndPartition(topic, _))
+
+ import ZkUtils._
+ // wait until admin path for delete topic is deleted, signaling completion of topic deletion
+ assert(
+ !zkUtils.pathExists(getDeleteTopicPath(topic)),
+ s"${getDeleteTopicPath(topic)} still exists")
+ assert(!zkUtils.pathExists(getTopicPath(topic)), s"${getTopicPath(topic)} still exists")
+ // ensure that the topic-partition has been deleted from all brokers' replica managers
+ assert(servers.forall(server => topicAndPartitions.forall(tp =>
+ server.replicaManager.getPartition(tp.topic, tp.partition) == None)),
+ s"topic $topic still exists in the replica manager")
+ // ensure that logs from all replicas are deleted if delete topic is marked successful
+ assert(servers.forall(server => topicAndPartitions.forall(tp =>
+ server.getLogManager().getLog(tp).isEmpty)),
+ s"topic $topic still exists in log mananger")
+ // ensure that topic is removed from all cleaner offsets
+ assert(servers.forall(server => topicAndPartitions.forall { tp =>
+ val checkpoints = server.getLogManager().logDirs.map { logDir =>
+ new OffsetCheckpoint(new File(logDir, "cleaner-offset-checkpoint")).read()
+ }
+ checkpoints.forall(checkpointsPerLogDir => !checkpointsPerLogDir.contains(tp))
+ }), s"checkpoint for topic $topic still exists")
+ // ensure the topic is gone
+ assert(
+ !zkUtils.getAllTopics().contains(topic),
+ s"topic $topic still exists on zookeeper")
+ }
+
private def verifyTopicDeletion(
--- End diff --
its hard to differentiate the semantics of `verifyTopicDeletion` and `assertTopicDeleted`. How about renamed them to `verifyTopicDeletionWithRetries` and `verifyTopicDeletion`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/16109
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3458 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3458/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3460 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3460/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3464 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3464/consoleFull)** for PR 16109 at commit [`8c5ae2d`](https://github.com/apache/spark/commit/8c5ae2d4b123f7872365cc2e27f2c93cdd74af4e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69752/consoleFull)** for PR 16109 at commit [`20b01d4`](https://github.com/apache/spark/commit/20b01d4819e3236432d026f95698371a41e3e6df).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69786/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3453 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3453/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69508/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/16109#discussion_r91242466
--- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala ---
@@ -811,6 +812,12 @@ class KafkaSourceStressForDontFailOnDataLossSuite extends StreamTest with Shared
private def newTopic(): String = s"failOnDataLoss-${topicId.getAndIncrement()}"
+ override def createSparkSession(): TestSparkSession = {
+ // Set maxRetries to 3 to handle NPE from `poll` when deleting a topic
+ new TestSparkSession(new SparkContext("local[2,3]", "test-sql-context",
+ sparkConf.set("spark.sql.testkey", "true")))
--- End diff --
It's only used on SQLConfSuite. Removed it from this test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69752/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3461 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3461/consoleFull)** for PR 16109 at commit [`8c5ae2d`](https://github.com/apache/spark/commit/8c5ae2d4b123f7872365cc2e27f2c93cdd74af4e).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3450 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3450/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/16109
LGTM. Merging this to master and 2.1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69508 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69508/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/16109
overall LGTM, just a few nits that would be good to fix.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3451 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3451/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69598/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/16109#discussion_r90544756
--- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala ---
@@ -102,7 +102,7 @@ private[kafka010] case class KafkaSource(
sourceOptions.getOrElse("fetchOffset.numRetries", "3").toInt
private val offsetFetchAttemptIntervalMs =
- sourceOptions.getOrElse("fetchOffset.retryIntervalMs", "10").toLong
+ sourceOptions.getOrElse("fetchOffset.retryIntervalMs", "100").toLong
--- End diff --
Increase this value because if not sleeping enough time, it's easy to fail with the same error when deleting a topic.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3450 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3450/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/16109#discussion_r90544901
--- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala ---
@@ -286,36 +286,56 @@ class KafkaTestUtils extends Logging {
props
}
+ /** Assert topic is deleted in all places, e.g, brokers, zookeeper. */
+ private def assertTopicDeleted(
+ topic: String,
+ numPartitions: Int,
+ servers: Seq[KafkaServer]): Unit = {
+ val topicAndPartitions = (0 until numPartitions).map(TopicAndPartition(topic, _))
+
+ import ZkUtils._
+ // wait until admin path for delete topic is deleted, signaling completion of topic deletion
+ assert(
+ !zkUtils.pathExists(getDeleteTopicPath(topic)),
+ s"${getDeleteTopicPath(topic)} still exists")
+ assert(!zkUtils.pathExists(getTopicPath(topic)), s"${getTopicPath(topic)} still exists")
+ // ensure that the topic-partition has been deleted from all brokers' replica managers
+ assert(servers.forall(server => topicAndPartitions.forall(tp =>
+ server.replicaManager.getPartition(tp.topic, tp.partition) == None)),
+ s"topic $topic still exists in the replica manager")
+ // ensure that logs from all replicas are deleted if delete topic is marked successful
+ assert(servers.forall(server => topicAndPartitions.forall(tp =>
+ server.getLogManager().getLog(tp).isEmpty)),
+ s"topic $topic still exists in log mananger")
+ // ensure that topic is removed from all cleaner offsets
+ assert(servers.forall(server => topicAndPartitions.forall { tp =>
+ val checkpoints = server.getLogManager().logDirs.map { logDir =>
+ new OffsetCheckpoint(new File(logDir, "cleaner-offset-checkpoint")).read()
+ }
+ checkpoints.forall(checkpointsPerLogDir => !checkpointsPerLogDir.contains(tp))
+ }), s"checkpoint for topic $topic still exists")
+ // ensure the topic is gone
+ assert(
+ !zkUtils.getAllTopics().contains(topic),
+ s"topic $topic still exists on zookeeper")
+ }
+
private def verifyTopicDeletion(
zkUtils: ZkUtils,
topic: String,
numPartitions: Int,
servers: Seq[KafkaServer]) {
- import ZkUtils._
- val topicAndPartitions = (0 until numPartitions).map(TopicAndPartition(topic, _))
- def isDeleted(): Boolean = {
--- End diff --
Renamed this method to `assertTopicDeleted` and moved out of this method. I also rewrote it to make it easy to tell us which line fails.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/16109#discussion_r91242517
--- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala ---
@@ -102,7 +102,7 @@ private[kafka010] case class KafkaSource(
sourceOptions.getOrElse("fetchOffset.numRetries", "3").toInt
private val offsetFetchAttemptIntervalMs =
- sourceOptions.getOrElse("fetchOffset.retryIntervalMs", "10").toLong
+ sourceOptions.getOrElse("fetchOffset.retryIntervalMs", "100").toLong
--- End diff --
Increased the default value to 1 second.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/16109
The NPE without stack trace is probably because http://jawspeak.com/2010/05/26/hotspot-caused-exceptions-to-lose-their-stack-traces-in-production-and-the-fix/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/16109#discussion_r91210695
--- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala ---
@@ -811,6 +812,12 @@ class KafkaSourceStressForDontFailOnDataLossSuite extends StreamTest with Shared
private def newTopic(): String = s"failOnDataLoss-${topicId.getAndIncrement()}"
+ override def createSparkSession(): TestSparkSession = {
+ // Set maxRetries to 3 to handle NPE from `poll` when deleting a topic
+ new TestSparkSession(new SparkContext("local[2,3]", "test-sql-context",
+ sparkConf.set("spark.sql.testkey", "true")))
--- End diff --
what is this key for?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3456 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3456/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3462/consoleFull)** for PR 16109 at commit [`8c5ae2d`](https://github.com/apache/spark/commit/8c5ae2d4b123f7872365cc2e27f2c93cdd74af4e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69746/consoleFull)** for PR 16109 at commit [`20b01d4`](https://github.com/apache/spark/commit/20b01d4819e3236432d026f95698371a41e3e6df).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3460 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3460/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69786 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69786/consoleFull)** for PR 16109 at commit [`c70ff19`](https://github.com/apache/spark/commit/c70ff19fb404689e4106278e917b466a6475241a).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69746/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3457 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3457/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3453 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3453/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3461 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3461/consoleFull)** for PR 16109 at commit [`8c5ae2d`](https://github.com/apache/spark/commit/8c5ae2d4b123f7872365cc2e27f2c93cdd74af4e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/16109#discussion_r91176023
--- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/CachedKafkaConsumer.scala ---
@@ -190,10 +194,31 @@ private[kafka010] case class CachedKafkaConsumer private(
} else {
val record = fetchedData.next()
nextOffsetInFetchedData = record.offset + 1
- // `seek` is always called before "poll". So "record.offset" must be same as "offset".
- assert(record.offset == offset,
- s"The fetched data has a different offset: expected $offset but was ${record.offset}")
- record
+ // In general, Kafka uses the specified offset as the start point, and tries to fetch the next
--- End diff --
Based on https://github.com/apache/kafka/blob/0.10.0.1/core/src/main/scala/kafka/log/Log.scala#L488
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3457 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3457/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/16109#discussion_r90545011
--- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala ---
@@ -331,7 +351,7 @@ class KafkaTestUtils extends Logging {
case _ =>
false
}
- eventually(timeout(10.seconds)) {
+ eventually(timeout(60.seconds)) {
--- End diff --
Increase this timeout as it seems `10 seconds` is not enough sometimes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3463 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3463/consoleFull)** for PR 16109 at commit [`8c5ae2d`](https://github.com/apache/spark/commit/8c5ae2d4b123f7872365cc2e27f2c93cdd74af4e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16109
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69784/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69598 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69598/consoleFull)** for PR 16109 at commit [`8c5ae2d`](https://github.com/apache/spark/commit/8c5ae2d4b123f7872365cc2e27f2c93cdd74af4e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3459 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3459/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3459 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3459/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/16109
> Test build #3459 has finished for PR 16109 at commit 0901a65.
>
> This patch fails PySpark unit tests.
> This patch merges cleanly.
> This patch adds no public classes.
Not related to this PR.
> Test build #3450 has finished for PR 16109 at commit 0901a65.
>
> This patch fails Spark unit tests.
> This patch merges cleanly.
> This patch adds no public classes.
Not related to this PR.
------------------------------------
> Test build #3453 has finished for PR 16109 at commit 0901a65.
>
> This patch fails Spark unit tests.
> This patch merges cleanly.
> This patch adds no public classes.
Looks like the retry interval is no enough. Found the following logs:
```
16/12/01 16:14:27.199 stream execution thread for query-62 WARN KafkaSource: Error in attempt 1 getting Kafka offsets:
java.lang.NullPointerException
16/12/01 16:14:30.832 stream execution thread for query-62 WARN KafkaSource: Error in attempt 2 getting Kafka offsets:
java.lang.NullPointerException
16/12/01 16:14:34.271 stream execution thread for query-62 WARN KafkaSource: Error in attempt 3 getting Kafka offsets:
java.lang.NullPointerException
```
I will increase the interval.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/16109#discussion_r91242485
--- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala ---
@@ -286,36 +286,56 @@ class KafkaTestUtils extends Logging {
props
}
+ /** Assert topic is deleted in all places, e.g, brokers, zookeeper. */
+ private def assertTopicDeleted(
+ topic: String,
+ numPartitions: Int,
+ servers: Seq[KafkaServer]): Unit = {
+ val topicAndPartitions = (0 until numPartitions).map(TopicAndPartition(topic, _))
+
+ import ZkUtils._
+ // wait until admin path for delete topic is deleted, signaling completion of topic deletion
+ assert(
+ !zkUtils.pathExists(getDeleteTopicPath(topic)),
+ s"${getDeleteTopicPath(topic)} still exists")
+ assert(!zkUtils.pathExists(getTopicPath(topic)), s"${getTopicPath(topic)} still exists")
+ // ensure that the topic-partition has been deleted from all brokers' replica managers
+ assert(servers.forall(server => topicAndPartitions.forall(tp =>
+ server.replicaManager.getPartition(tp.topic, tp.partition) == None)),
+ s"topic $topic still exists in the replica manager")
+ // ensure that logs from all replicas are deleted if delete topic is marked successful
+ assert(servers.forall(server => topicAndPartitions.forall(tp =>
+ server.getLogManager().getLog(tp).isEmpty)),
+ s"topic $topic still exists in log mananger")
+ // ensure that topic is removed from all cleaner offsets
+ assert(servers.forall(server => topicAndPartitions.forall { tp =>
+ val checkpoints = server.getLogManager().logDirs.map { logDir =>
+ new OffsetCheckpoint(new File(logDir, "cleaner-offset-checkpoint")).read()
+ }
+ checkpoints.forall(checkpointsPerLogDir => !checkpointsPerLogDir.contains(tp))
+ }), s"checkpoint for topic $topic still exists")
+ // ensure the topic is gone
+ assert(
+ !zkUtils.getAllTopics().contains(topic),
+ s"topic $topic still exists on zookeeper")
+ }
+
private def verifyTopicDeletion(
--- End diff --
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69786 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69786/consoleFull)** for PR 16109 at commit [`c70ff19`](https://github.com/apache/spark/commit/c70ff19fb404689e4106278e917b466a6475241a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #69746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69746/consoleFull)** for PR 16109 at commit [`20b01d4`](https://github.com/apache/spark/commit/20b01d4819e3236432d026f95698371a41e3e6df).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3458 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3458/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3464 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3464/consoleFull)** for PR 16109 at commit [`8c5ae2d`](https://github.com/apache/spark/commit/8c5ae2d4b123f7872365cc2e27f2c93cdd74af4e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/16109
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceStressFor...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16109
**[Test build #3456 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3456/consoleFull)** for PR 16109 at commit [`0901a65`](https://github.com/apache/spark/commit/0901a65fadc7053bac325552bd1822108a246e75).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16109: [SPARK-18588][Tests]Fix flaky test: KafkaSourceSt...
Posted by tdas <gi...@git.apache.org>.
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/16109#discussion_r91210829
--- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala ---
@@ -102,7 +102,7 @@ private[kafka010] case class KafkaSource(
sourceOptions.getOrElse("fetchOffset.numRetries", "3").toInt
private val offsetFetchAttemptIntervalMs =
- sourceOptions.getOrElse("fetchOffset.retryIntervalMs", "10").toLong
+ sourceOptions.getOrElse("fetchOffset.retryIntervalMs", "100").toLong
--- End diff --
Is 100ms enough or should this be increase to 1 second?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org