[jira] [Created] (SPARK-32000) Fix the flaky testcase for partially launched task in barrier-mode.

["Kousuke Saruta (Jira)" <ji...@apache.org>]

Kousuke Saruta created SPARK-32000:
--------------------------------------

             Summary: Fix the flaky testcase for partially launched task in barrier-mode.
                 Key: SPARK-32000
                 URL: https://issues.apache.org/jira/browse/SPARK-32000
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core, Tests
    Affects Versions: 3.1.0
            Reporter: Kousuke Saruta
            Assignee: Kousuke Saruta


I noticed sometimes the testcase for SPARK-31485 fails.
The reason should be related to the locality wait. 

If the scheduler waits for a resource offer which meets the preferred location for a task until the time-limit of process-local but no resource can be offered for the locality level, the scheduler will give up the preferred location. In this case, such task can be assigned to off-preferred location.

The testcase for SPARK-31485, there are two tasks and only one task is supposed to be assigned at one schedule round but both two tasks can be assigned in that situation mentioned above and the testcase will fail.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org