[GitHub] [flink] 1996fanrui commented on a diff in pull request #19499: [FLINK-26762][network] Add the overdraft buffer in ResultPartition

[GitBox <gi...@apache.org>]

1996fanrui commented on code in PR #19499:
URL: https://github.com/apache/flink/pull/19499#discussion_r899769698


##########
docs/layouts/shortcodes/generated/all_taskmanager_network_section.html:
##########
@@ -80,6 +80,12 @@
             <td>Integer</td>
             <td>Number of max buffers that can be used for each channel. If a channel exceeds the number of max buffers, it will make the task become unavailable, cause the back pressure and block the data processing. This might speed up checkpoint alignment by preventing excessive growth of the buffered in-flight data in case of data skew and high number of configured floating buffers. This limit is not strictly guaranteed, and can be ignored by things like flatMap operators, records spanning multiple buffers or single timer producing large amount of data.</td>
         </tr>
+        <tr>
+            <td><h5>taskmanager.network.memory.max-overdraft-buffers-per-gate</h5></td>
+            <td style="word-wrap: break-word;">5</td>
+            <td>Integer</td>
+            <td>Number of max overdraft network buffers to use for each ResultPartition. The overdraft buffers will be used when the ResultPartition cannot apply to the normal buffers, e.g: the all buffers of ResultPartition be applied or the number of buffers for a single channel has reached taskmanager.network.memory.max-buffers-per-channel. When the ResultPartition is unavailable, the ResultPartition can provide some additional buffers to allow overdraft. It can effectively solve the problem that multiple output buffers are required to process a single data, causing the Task to block in the requestMemory, such as: flatMap operator, records spanning multiple buffers or a single timer generates a large amount of data. It doesn't need to wait for ResultPartition is available to start Unaligned Checkpoint directly.</td>
+        </tr>

Review Comment:
   Yes, CI will fail if the code and doc don't match.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org