[jira] [Commented] (SPARK-5153) flaky test of "Reliable Kafka input stream with multiple topics"

["Saisai Shao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14270655#comment-14270655 ] 

Saisai Shao commented on SPARK-5153:
------------------------------------

Hi [~CodingCat], thanks a lot for your reporting, mainly the problem is caused in here:

{code}
    eventually(timeout(1000 milliseconds), interval(100 milliseconds)) {
      assert(
        server.apis.leaderCache.keySet.contains(TopicAndPartition(topic, partition)),
        s"Partition [$topic, $partition] metadata not propagated after timeout"
      )
    }
{code}

which trying to verify if topic is created and propagated accordingly, currently the longest timeout is 1000 ms, which might be fine for local test, but probably it will be timeout if the machine is in high load. Actually I've tested in my local machine for several rounds, this issue is hard to reproduce.

I think a possible way is to increase the timeout (1000 ms) to a large value.



> flaky test of "Reliable Kafka input stream with multiple topics"
> ----------------------------------------------------------------
>
>                 Key: SPARK-5153
>                 URL: https://issues.apache.org/jira/browse/SPARK-5153
>             Project: Spark
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 1.2.0
>            Reporter: Nan Zhu
>
> I have seen several irrelevant PR failed on this test
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25254/consoleFull
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25248/
> https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25251/console



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org