[GitHub] [flink] zhijiangW opened a new pull request #9062: [FLINK-13100][network] Fix the bug of throwing IOException while FileChannelBoundedData#nextBuffer

[GitBox <gi...@apache.org>]

zhijiangW opened a new pull request #9062: [FLINK-13100][network] Fix the bug of throwing IOException while FileChannelBoundedData#nextBuffer
URL: https://github.com/apache/flink/pull/9062
 
 
   ## What is the purpose of the change 
   
   *In the process of `FileChannelBoundedData#nextBuffer`, the implementation is based on the assumption that there always has an available buffer, otherwise an `IOException` is thrown. But in the case of pending flush operation that means the socket channel is not writable sometimes while netty thread calling `writeAndFlush` method, the first fetched buffer from `FileChannelBoundedData` has not been recycled while fetching the second buffer to trigger next read ahead. Then it would cause this problem to break above assumption.*
       
   *In order to fix this problem, we could make the assumption that the read ahead is not always available for `FileChannelBoundedData`. If there are no available buffers to read the next data, we could retrigger the read ahead while recycling buffer via `ResultSubpartitionView#notifyDataAvailable`.*
   
   ## Brief change log
   
     - *Move private `NoOpResultSubpartitionView` class out of `PartitionRequestQueueTest`*
     - *Make `BoundedBlockingSubpartitionType` configurable in `ResultPartitionBuilder` for tests usage*
     - *Pass `BufferAvailabilityListener` in the constructor of `BoundedBlockingSubpartitionReader`*
     - *Implement the logic of `notifyDataAvailable` in `BoundedBlockingSubpartitionReader `*
     - *Fix the logic of `FileBufferReader#nextBuffer`*
     - *Fix the logic of `FileBufferReader#recycleBuffer`*
   
   ## Verifying this change
   
     - *Added `FileChannelBoundedDataTest#testReadNextBuffer` for verifying the logic of `nextBuffer`*
     - *Added `FileChannelBoundedDataTest#testRecycleBufferForNotifyingSubpartitionView` for verifying the notification of `ResultSubpartitionView`*
     - *Added `FileChannelBoundedDataTest#testRecycleBufferForNotifyingBufferAvailabilityListener` for verifying the notification of `BufferAvailabilityListener`*
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (yes / **no**)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
     - The serializers: (yes / **no** / don't know)
     - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
     - The S3 file system connector: (yes / **no** / don't know)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (yes / **no**)
     - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services