[jira] [Created] (FLINK-10462) Remove ConnectionIndex for further sharing tcp connection in credit-based mode

["zhijiang (JIRA)" <ji...@apache.org>]

zhijiang created FLINK-10462:
--------------------------------

             Summary: Remove ConnectionIndex for further sharing tcp connection in credit-based mode 
                 Key: FLINK-10462
                 URL: https://issues.apache.org/jira/browse/FLINK-10462
             Project: Flink
          Issue Type: Improvement
          Components: Network
    Affects Versions: 1.5.4, 1.6.1, 1.6.0, 1.5.3, 1.5.2, 1.5.1, 1.5.0
            Reporter: zhijiang
            Assignee: zhijiang


Every {{IntermediateResult}} generates a random {{ConnectionIndex}} which will be included in {{ConnectionID}}.

The {{RemoteInputChannel}} requests to establish tcp connection via {{ConnectionID}}. That means one tcp connection may be shared by multiple {{RemoteInputChannel}}s which have the same {{ConnectionID}}. To do so, we can reduce the physical connections between two {{TaskManager}}s, and it bring benefits for large scale jobs.

But this sharing is limited only for the same {{IntermediateResult}}, and I think it is mainly because we may temporarily switch off {{autoread}} for the channel during back pressure in previous network flow control. For credit-based mode, the channel is always open for transporting different intermediate data, so we can further share the tcp connection for different {{IntermediateResult}} to remove the limit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)