You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Zhu Zhu (Jira)" <ji...@apache.org> on 2021/08/05 02:44:00 UTC

[jira] [Assigned] (FLINK-22674) Provide JobID when apply shuffle resource by ShuffleMaster

     [ https://issues.apache.org/jira/browse/FLINK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhu Zhu reassigned FLINK-22674:
-------------------------------

    Assignee: Yingjie Cao

> Provide JobID when apply shuffle resource by ShuffleMaster
> ----------------------------------------------------------
>
>                 Key: FLINK-22674
>                 URL: https://issues.apache.org/jira/browse/FLINK-22674
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / Network
>            Reporter: Jin Xing
>            Assignee: Yingjie Cao
>            Priority: Major
>              Labels: pull-request-available
>
> In current Flink 'pluggable shuffle service' framework, only PartitionDescriptor and ProducerDescriptor are included as parameters in ShuffleMaster#registerPartitionWithProducer.
> But when extending a remote shuffle service based on 'pluggable shuffle service', JobID is also needed when apply shuffle resource from remote cluster. It can be used as an identification to link shuffle resource with the corresponding job:
>  # Remote shuffle cluster can isolate or do capacity control on shuffle resource between jobs;
>  # Remote shuffle cluster can use JobID for shuffle data cleanup when job is lost thus to avoid file leak;



--
This message was sent by Atlassian Jira
(v8.3.4#803005)