[jira] [Commented] (FLINK-14106) Make SlotManager pluggable

["Xintong Song (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-14106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039901#comment-17039901 ] 

Xintong Song commented on FLINK-14106:
--------------------------------------

The WIP FLIP-56 (Dynamic Slot Allocation) tries to introduce significant changes to the slot allocations. Given the efforts needed and potential risk of changing the current slot allocation behaviors directly, after an offline discussion with [~sewen], [~trohrmann] and some other people, we decided to first make it a plugin beside the current slot allocation strategy. Therefore, we are reviving this issue again to make Flink's slot allocation pluggable.

A design doc of proposed changes is linked to the ticket.

> Make SlotManager pluggable
> --------------------------
>
>                 Key: FLINK-14106
>                 URL: https://issues.apache.org/jira/browse/FLINK-14106
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Coordination
>    Affects Versions: 1.9.0
>            Reporter: Xintong Song
>            Priority: Major
>             Fix For: 1.11.0
>
>
> As we are enabling fine grained resource management in 1.10, we can have various resource scheduling strategies. Such strategies generally should make the following three decisions.
>  * When to launch new / release existing TMs? (How many TMs)
>  * What and how many resources should TMs be started with?
>  * How to allocate between slot requests and TM resources?
> We may want to make above decisions differently in different scenarios (active/reactive mode, perjob/session mode, etc.). Therefore, we propose to make the scheduling strategies pluggable.
> We propose to make the following changes:
>  * Make SlotManager an interface, and implements it differently for different strategies strategies.
>  * Modify ResourceManager-SlotManager interfaces to cover all the three decisions mentioned above in SlotManager. In particular, SlotManager needs to allocate TM resources instead of slot resources from ResourceActions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)