You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Yingjie Cao (Jira)" <ji...@apache.org> on 2022/08/09 02:00:00 UTC

[jira] [Closed] (FLINK-28380) Produce one intermediate dataset for multiple consumers consuming the same data

     [ https://issues.apache.org/jira/browse/FLINK-28380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yingjie Cao closed FLINK-28380.
-------------------------------
    Resolution: Fixed

Merged into master via 60bc87c0b83149c4f19d7e54af2d967087a277fb

> Produce one intermediate dataset for multiple consumers consuming the same data
> -------------------------------------------------------------------------------
>
>                 Key: FLINK-28380
>                 URL: https://issues.apache.org/jira/browse/FLINK-28380
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Client / Job Submission, Runtime / Coordination, Runtime / Network
>            Reporter: Yingjie Cao
>            Assignee: Yingjie Cao
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.16.0
>
>
> Currently, if one output of an upstream job vertex is consumed by multiple downstream job vertices, the upstream vertex will produce multiple dataset. For blocking shuffle, it means serialize and persist the same data multiple times. This ticket aims to optimize this behavior and make the upstream job vertex produce one dataset which will be read by multiple downstream vertex.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)