[jira] [Closed] (FLINK-31634) FLIP-301: Hybrid Shuffle supports Remote Storage

["Yuxin Tan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-31634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yuxin Tan closed FLINK-31634.
-----------------------------
    Release Note: 
This introduces a new Hybrid Shuffle mode to support remote storage. The remote storage path is specified by `taskmanager.network.hybrid-shuffle.remote.path`.  The new Hybrid Shuffle mode has resolved existing issues in the legacy mode. Firstly, the new mode uses less required network memory. Secondly, the new mode can store shuffle data in remote storage when the disk space is not enough, which could avoid insufficient disk space errors. This feature is only enabled when `taskmanager.network.hybrid-shuffle.remote.path` is configured.

Currently, the new Hybrid Shuffle mode is in an experimental phase. In case of unexpected issues, the mode can fallback to the legacy mode by setting `taskmanager.network.hybrid-shuffle.enable-new-mode` to `false`. Once the new mode is stable, the legacy mode and its option will be removed.
      Resolution: Done

> FLIP-301: Hybrid Shuffle supports Remote Storage
> ------------------------------------------------
>
>                 Key: FLINK-31634
>                 URL: https://issues.apache.org/jira/browse/FLINK-31634
>             Project: Flink
>          Issue Type: New Feature
>          Components: Runtime / Network
>    Affects Versions: 1.18.0
>            Reporter: Yuxin Tan
>            Assignee: Yuxin Tan
>            Priority: Major
>              Labels: Umbrella
>
> This is an umbrella ticket for [FLIP-301|https://cwiki.apache.org/confluence/display/FLINK/FLIP-301%3A+Hybrid+Shuffle+supports+Remote+Storage].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)