[jira] [Commented] (FLINK-7406) Implement Netty receiver incoming pipeline for credit-based

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-7406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16187902#comment-16187902 ] 

ASF GitHub Bot commented on FLINK-7406:
---------------------------------------

Github user zhijiangW commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4509#discussion_r142120202
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPool.java ---
    @@ -131,6 +136,63 @@ public void recycle(MemorySegment segment) {
     		availableMemorySegments.add(segment);
     	}
     
    +	public List<MemorySegment> requestMemorySegments(int numRequiredBuffers) throws IOException {
    +		checkArgument(numRequiredBuffers > 0, "The number of required buffers should be larger than 0.");
    +
    +		synchronized (factoryLock) {
    +			if (isDestroyed) {
    +				throw new IllegalStateException("Network buffer pool has already been destroyed.");
    +			}
    +
    +			if (numTotalRequiredBuffers + numRequiredBuffers > totalNumberOfMemorySegments) {
    +				throw new IOException(String.format("Insufficient number of network buffers: " +
    --- End diff --
    
    yes, I think it can be addressed in a separate JIRA. :)


> Implement Netty receiver incoming pipeline for credit-based
> -----------------------------------------------------------
>
>                 Key: FLINK-7406
>                 URL: https://issues.apache.org/jira/browse/FLINK-7406
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>            Reporter: zhijiang
>            Assignee: zhijiang
>             Fix For: 1.4.0
>
>
> This is a part of work for credit-based network flow control.
> Currently {{PartitionRequestClientHandler}} receives and reads {{BufferResponse}} from producer. It will request buffer from {{BufferPool}} for holding the message. If not got, the message is staged temporarily and {{autoread}} for channel is set false.
> For credit-based mode, {{PartitionRequestClientHandler}} can always get buffer from {{RemoteInputChannel}} for reading messages from producer.
> The related works are:
> * Add the backlog of producer in {{BufferResponse}} message structure
> * {{PartitionRequestClientHandler}} requests buffer from {{RemoteInputChannel}} directly
> * {{PartitionRequestClientHandler}} updates backlog for {{RemoteInputChannel}}, and it may trigger requests of floating buffers from {{BufferPool}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)