You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "zhijiang (JIRA)" <ji...@apache.org> on 2018/11/30 07:59:00 UTC

[jira] [Created] (FLINK-11035) Notify data available to network stack immediately after finishing BufferBuilder

zhijiang created FLINK-11035:
--------------------------------

             Summary: Notify data available to network stack immediately after finishing BufferBuilder
                 Key: FLINK-11035
                 URL: https://issues.apache.org/jira/browse/FLINK-11035
             Project: Flink
          Issue Type: Sub-task
          Components: Network
    Affects Versions: 1.8.0
            Reporter: zhijiang


The data availability notification for network relies on whether there are finished _BufferBuilder_ or flush triggered. If flush is not triggered and the first _BufferBuilder_ enqueues into the subpartition, although this _BufferBuilder_ is finished on _RecordWriter_ side, it has to rely on enqueuing the second _BufferBuilder_ to trigger notification available.  It may bring some delays for transporting the finished _BufferBuilder_ in network, especially there has a blocking operation for requesting the second _BufferBuilder_ from pool.

Supposing there is only one available buffer in LocalBufferPool in extreme scenarios, if the first _BufferBuilder_ is not transported and recycled, the requesting for second _BufferBuilder_ will be blocked all the time.

I propose to add a _notifyBufferFinished_ method in _ResultPartitionWriter_ interface, then _RecordWriter_ can notify via it after _BufferBuilder_ finished_._

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)