[GitHub] [flink] tillrohrmann opened a new pull request #9058: [FLINK-13166] Add support for batch slot requests to SlotPoolImpl

[GitBox <gi...@apache.org>]

tillrohrmann opened a new pull request #9058: [FLINK-13166] Add support for batch slot requests to SlotPoolImpl
URL: https://github.com/apache/flink/pull/9058
 
 
   ## What is the purpose of the change
   
   This commit adds a new type of slot request which can be issued to the SlotPoolImpl.
   The batch slot request is intended for batch jobs which can be executed with a single
   slot (having at least one slot for every requested resource profile equivalence class).
   Usually, a job which fulfills this criterion must not contain a pipelined shuffle.
   
   The new batch slot request behaves in the following aspects differently than the normal
   slot request:
   
   * Batch slot request don't time out if the SlotPool contains at least one allocated slot
   which can fulfill the pending slot request
   * Batch slot request don't react to the failAllocation signal from the ResourceManager
   * Batch slot request don't fail if the slot request to the resource manager fails
   
   In order to time out batch slot request which cannot be fulfilled with an allocated slot,
   the SlotPoolImpl schedules a periodic task which checks for this condition. If a slot cannot
   be fulfilled, it is marked as unfulfillable and the current timestamp is recorded. If the
   slot cannot be marked as fulfillable until the batch slot timeout has been exceeded, the
   slot request will be timed out.
   
   The batch slot request will be requested by calling SlotPool#requestNewAllocatedBatchSlot.
   
   cc @xintongsong @StephanEwen 
   
   ## Verifying this change
   
   Added `SlotPoolBatchSlotRequestTest`
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (no)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
     - The serializers: (no)
     - The runtime per-record code paths (performance sensitive): (no)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)
     - The S3 file system connector: (no)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (no)
     - If yes, how is the feature documented? (not applicable)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services