[jira] [Commented] (FLINK-9253) Make buffer count per InputGate always #channels*buffersPerChannel + ExclusiveBuffers

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-9253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462535#comment-16462535 ] 

ASF GitHub Bot commented on FLINK-9253:
---------------------------------------

Github user pnowojski commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5923#discussion_r185819830
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/NetworkEnvironment.java ---
    @@ -228,20 +228,19 @@ public void setupPartition(ResultPartition partition) throws IOException {
     	@VisibleForTesting
     	public void setupInputGate(SingleInputGate gate) throws IOException {
     		BufferPool bufferPool = null;
    -		int maxNumberOfMemorySegments;
     		try {
     			if (enableCreditBased) {
    -				maxNumberOfMemorySegments = gate.getConsumedPartitionType().isBounded() ?
    -					extraNetworkBuffersPerGate : Integer.MAX_VALUE;
    -
     				// assign exclusive buffers to input channels directly and use the rest for floating buffers
    -				gate.assignExclusiveSegments(networkBufferPool, networkBuffersPerChannel);
    -				bufferPool = networkBufferPool.createBufferPool(0, maxNumberOfMemorySegments);
    +				int nrExclusiveMemorySegments = gate.assignExclusiveSegments(networkBufferPool, networkBuffersPerChannel);
    +				int maxNumberOfMemorySegments = gate.getConsumedPartitionType().isBounded() ?
    +					gate.getNumberOfInputChannels() * networkBuffersPerChannel +
    +						extraNetworkBuffersPerGate - nrExclusiveMemorySegments : Integer.MAX_VALUE;
    +				bufferPool = networkBufferPool
    +					.createBufferPool(0, maxNumberOfMemorySegments);
    --- End diff --
    
    I think that express this way:
    ```
    if (enableCreditBased) {
    	int desiredMaxNumberOfMemorySegments = gate.getNumberOfInputChannels() * networkBuffersPerChannel + extraNetworkBuffersPerGate;
    	int assignedExclusiveMemorySegments = gate.assignExclusiveSegments(networkBufferPool, networkBuffersPerChannel);
    	int floatingMemorySegments = desiredMaxNumberOfMemorySegments - assignedExclusiveMemorySegments;
    
    	bufferPool = networkBufferPool
    		.createBufferPool(0, gate.getConsumedPartitionType().isBounded() ? floatingMemorySegments : Integer.MAX_VALUE);
    }
    ```
    
    it's more easier to understand and allows us to skip the redundant comment. Especially current `maxNumberOfMemorySegments` is strange name.


> Make buffer count per InputGate always #channels*buffersPerChannel + ExclusiveBuffers
> -------------------------------------------------------------------------------------
>
>                 Key: FLINK-9253
>                 URL: https://issues.apache.org/jira/browse/FLINK-9253
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>    Affects Versions: 1.5.0, 1.6.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>            Priority: Major
>             Fix For: 1.5.0
>
>
> The credit-based flow control path assigns exclusive buffers only to remote channels (which makes sense since local channels don't use any own buffers). However, this is a bit intransparent with respect to how much data may be in buffers since this depends on the actual schedule of the job and not the job graph.
> By adapting the floating buffers to use a maximum of {{#channels*buffersPerChannel + floatingBuffersPerGate - #exclusiveBuffers}}, we would be channel-type agnostic and keep the old behaviour.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)