You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Marcelo Vanzin (JIRA)" <ji...@apache.org> on 2019/07/27 00:00:00 UTC

[jira] [Created] (SPARK-28535) Flaky test: JobCancellationSuite."interruptible iterator of shuffle reader"

Marcelo Vanzin created SPARK-28535:
--------------------------------------

             Summary: Flaky test: JobCancellationSuite."interruptible iterator of shuffle reader"
                 Key: SPARK-28535
                 URL: https://issues.apache.org/jira/browse/SPARK-28535
             Project: Spark
          Issue Type: Bug
          Components: Tests
    Affects Versions: 3.0.0
            Reporter: Marcelo Vanzin


This is the same flakiness as in SPARK-23881, except the fix there didn't really take, at least on our build machines.

{noformat}
org.scalatest.exceptions.TestFailedException: 10000 was not less than 10000
      at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:528)
      at org.scalatest.FunSuite.newAssertionFailedException(FunSuite.scala:1560)
      at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:501)
{noformat}

Since that bug is short on explanations, the issue is that there's a race between the thread posting the "stage completed" event to the listener which unblocks the test, and the thread killing the task in the executor. If the even arrives first, it will unblock task execution, and there's a chance that all elements will actually be processed before the executor has a chance to stop the task.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org