[GitHub] [spark] attilapiros opened a new pull request #24474: [SPARK-26891][BACKPORT-2.4][YARN] Fixing flaky test in YarnSchedulerBackendSuite

[GitBox <gi...@apache.org>]

attilapiros opened a new pull request #24474: [SPARK-26891][BACKPORT-2.4][YARN] Fixing flaky test in YarnSchedulerBackendSuite
URL: https://github.com/apache/spark/pull/24474
 
 
   ## What changes were proposed in this pull request?
   
   The test "RequestExecutors reflects node blacklist and is serializable" is flaky because of multi threaded access of the mock task scheduler. For details check [Mockito FAQ (occasional exceptions like: WrongTypeOfReturnValue)](https://github.com/mockito/mockito/wiki/FAQ#is-mockito-thread-safe). So instead of mocking the task scheduler in the test TaskSchedulerImpl is simply subclassed.
   
   This multithreaded access of the `nodeBlacklist()` method is coming from:
   1) the unit test thread via calling of the method `prepareRequestExecutors()`
   2) the `DriverEndpoint.onStart` which runs a periodic task that ends up calling this method
   
   ## How was this patch tested?
   
   Existing unittest.
   
   (cherry picked from commit e4e4e2b842bffba6805623f2258b27b162b451ba)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org