[jira] [Created] (SPARK-29780) The UI can access into the ResourceAllocator, whose data structures are being updated from scheduler threads

["Alessandro Bellina (Jira)" <ji...@apache.org>]

Alessandro Bellina created SPARK-29780:
------------------------------------------

             Summary: The UI can access into the ResourceAllocator, whose data structures are being updated from scheduler threads
                 Key: SPARK-29780
                 URL: https://issues.apache.org/jira/browse/SPARK-29780
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 3.0.0
            Reporter: Alessandro Bellina


A class extending ResourceAllocator (WorkerResourceInfo), has some potential issues (raised here: https://github.com/apache/spark/pull/26078#discussion_r340342820)

The WorkerInfo class is calling into availableAddrs and assignedAddrs but those calls appear to be coming from the UI (looking at the resourcesInfo* functions), e.g. JsonProtocol and MasterPage call this. Since the datastructures in ResourceAllocator are not concurrent, we could end up with bad data or potentially crashes, depending on when the calls are made. 

Note that there are other calls to the resourceInfo* functions, but those are from the event loop.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org