[GitHub] StefanRRichter opened a new pull request #7568: [FLINK-11417] Make access to ExecutionGraph single threaded from JobMaster main thread

[GitBox <gi...@apache.org>]

StefanRRichter opened a new pull request #7568: [FLINK-11417] Make access to ExecutionGraph single threaded from JobMaster main thread
URL: https://github.com/apache/flink/pull/7568
 
 
   ## What is the purpose of the change
   
   The PR makes the interaction/modifiations to the `ExecutionGraph` single threaded to simplify future developments in this area.  Expensive tasks will still be performed by background threads so that the `JobMaster` main thread will never block. However, all resulting modifications and interaction with the EG itself should only go through the JM main thread.
   
   ## Brief change log
   
   - Introduce assertions to check the new threading contraints in all relevant places.
   - Pass down the job master's executor to all places that want to interact with the `ExecutionGraph`.
   - Sync back expensive background operations to the job master main thread for modifications to the `ExecutionGraph`.
   - Fix unit tests, for some of them we have to invoke test methods via an executor now.
   
   ## Verifying this change
   
   This change is already covered by existing tests.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (no)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
     - The serializers: (no)
     - The runtime per-record code paths (performance sensitive): (no)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes )
     - The S3 file system connector: (no)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? ( no)
     - If yes, how is the feature documented? (not applicable)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services