You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "dalongliu (Jira)" <ji...@apache.org> on 2023/08/31 02:30:00 UTC

[jira] [Commented] (FLINK-33004) Decoupling topology and network memory to support complex job topologies

    [ https://issues.apache.org/jira/browse/FLINK-33004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760673#comment-17760673 ] 

dalongliu commented on FLINK-33004:
-----------------------------------

cc [~guoweijie] 

> Decoupling topology and network memory to support complex job topologies
> ------------------------------------------------------------------------
>
>                 Key: FLINK-33004
>                 URL: https://issues.apache.org/jira/browse/FLINK-33004
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Network
>    Affects Versions: 1.18.0, 1.19.0
>            Reporter: dalongliu
>            Priority: Major
>
> Currently, the default value of taskmanager.memory.network.fraction option in Flink is 0.1, and after the topology of the job is complex enough, it will run with an insufficient network buffer. We currently encountered this issue when running TPC-DS test set q9, and bypassed it by adjusting taskmanager.memory.network.fraction to 0.2. Theoretically, we should have network memory decoupled from the job topology so that arbitrarily complex jobs can be supported.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)