You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "dalongliu (Jira)" <ji...@apache.org> on 2023/08/31 02:29:00 UTC

[jira] [Created] (FLINK-33004) Decoupling topology and network memory to support complex job topologies

dalongliu created FLINK-33004:
---------------------------------

             Summary: Decoupling topology and network memory to support complex job topologies
                 Key: FLINK-33004
                 URL: https://issues.apache.org/jira/browse/FLINK-33004
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Network
    Affects Versions: 1.18.0, 1.19.0
            Reporter: dalongliu


Currently, the default value of taskmanager.memory.network.fraction option in Flink is 0.1, and after the topology of the job is complex enough, it will run with an insufficient network buffer. We currently encountered this issue when running TPC-DS test set q9, and bypassed it by adjusting taskmanager.memory.network.fraction to 0.2. Theoretically, we should have network memory decoupled from the job topology so that arbitrarily complex jobs can be supported.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)