[jira] [Commented] (FLINK-7456) Implement Netty sender incoming pipeline for credit-based

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-7456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129035#comment-16129035 ] 

ASF GitHub Bot commented on FLINK-7456:
---------------------------------------

GitHub user zhijiangW opened a pull request:

    https://github.com/apache/flink/pull/4552

    [FLINK-7456][network]Implement Netty sender incoming pipeline for credit-based

    ## What is the purpose of the change
    
    This PR is based on #4533 whose commits are also included for passing travis. Review the last commit for this PR change.
    
    On sender side, it maintains credit from receiver's `PartitionRequest` and `AddCredit` messages, then sends buffer based on credit and network capacity. This PR is mainly involved in incoming pipeline logic for credit-based.
    
    ## Brief change log
    
      - *Each subpartition view maintains current credit and a boolean field to mark whether it is already registered available for transfer*
      - *Update current credit in processing `PartitionRequest` and `AddCredit` messages*
      - *The mechanism of enqueue the subpartition view and update the registered status field*
    
    ## Verifying this change
    
    This change added tests and can be verified as follows:
    
      - *Added test to verify that current credit is updated correctly and subpartition view is enqueued when received `AddCredit` message*
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): (no)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
      - The serializers: (no)
      - The runtime per-record code paths (performance sensitive): (no)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
    
    ## Documentation
    
      - Does this pull request introduce a new feature? (no)
      - If yes, how is the feature documented? (not applicable)
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zhijiangW/flink FLINK-7456

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/4552.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4552
    
----
commit e35c1ff8066bf44344495d132a1092b9db3ef182
Author: Zhijiang <wa...@aliyun.com>
Date:   2017-08-07T09:31:17Z

    [FLINK-7378][core]Create a fix size (non rebalancing) buffer pool type for the floating buffers

commit 969c24d3bf80c1ff89ada11e81b9bf4fea14066f
Author: Zhijiang <wa...@aliyun.com>
Date:   2017-08-14T06:30:47Z

    [FLINK-7394][core]Implement basic InputChannel with free buffers,credit and backlog

commit 15fa828449d73f53042c57e9c5494d75ddee575f
Author: Zhijiang <wa...@aliyun.com>
Date:   2017-08-10T05:29:13Z

    [FLINK-7406][network]Implement Netty receiver incoming pipeline for credit-based

commit d0674244f15701863a5dd3f68b7274b3bd49c64d
Author: Zhijiang <wa...@aliyun.com>
Date:   2017-08-12T14:13:25Z

    [FLINK-7416][network] Implement Netty receiver outgoing pipeline for credit-based

commit 6eaff7877ad43eab674e184153365b50ec8e1559
Author: Zhijiang <wa...@aliyun.com>
Date:   2017-08-16T13:24:53Z

    [FLINK-7456][network]Implement Netty sender incoming pipeline for credit-based

----


> Implement Netty sender incoming pipeline for credit-based
> ---------------------------------------------------------
>
>                 Key: FLINK-7456
>                 URL: https://issues.apache.org/jira/browse/FLINK-7456
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>            Reporter: zhijiang
>            Assignee: zhijiang
>             Fix For: 1.4.0
>
>
> This is a part of work for credit-based network flow control.
> On sender side, each subpartition view maintains an atomic integer {{currentCredit}} from receiver. Once receiving the messages of {{PartitionRequest}} and {{AddCredit}}, the {{currentCredit}} is added by deltas.
> Each view also maintains an atomic boolean field to mark it as registered available for transfer to make sure it is enqueued in handler only once. If the {{currentCredit}} increases from zero and there are available buffers in the subpartition, the corresponding view will be enqueued for transferring data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)