[jira] [Commented] (FLINK-24900) Support to run multiple shuffle plugins in one session cluster

["Yingjie Cao (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-24900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17468523#comment-17468523 ] 

Yingjie Cao commented on FLINK-24900:
-------------------------------------

[~monster#12] Hi, thanks for your interests, this improvement still need some discussion. The change is not big, but there are several details which need to be discussed in the community openly. I will open a discussion this week and we can move forward when a consensus is reached. This change will be split into multiple sub-tasks and you may pick some if you are still interest in this. Here is a simple PoC implementation: [https://github.com/apache/flink/pull/18253] and you can take a look. Any feedback is appreciated.

> Support to run multiple shuffle plugins in one session cluster
> --------------------------------------------------------------
>
>                 Key: FLINK-24900
>                 URL: https://issues.apache.org/jira/browse/FLINK-24900
>             Project: Flink
>          Issue Type: Sub-task
>            Reporter: Yingjie Cao
>            Priority: Major
>              Labels: pull-request-available
>
> Currently, one Flink cluster can only use one shuffle plugin. However, there are cases where different jobs may need different shuffle implementations. By loading shuffle plugin with the plugin manager and letting jobs select their shuffle service freely, Flink can support to run multiple shuffle plugins in one session cluster.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)