[GitHub] [flink] reswqa commented on a diff in pull request #22381: [FLINK-31763][runtime] Ensure that the total number of requested buffers does not exceed poolSize + maxOverdraftBuffersPerGate

["reswqa (via GitHub)" <gi...@apache.org>]

reswqa commented on code in PR #22381:
URL: https://github.com/apache/flink/pull/22381#discussion_r1164030247


##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/LocalBufferPool.java:
##########
@@ -671,13 +675,20 @@ public void setNumBuffers(int numBuffers) {
 
             currentPoolSize = Math.min(numBuffers, maxNumberOfMemorySegments);
 
-            // reset overdraft buffers
+            // If pool size increases, try to convert overdraft buffer to ordinary buffer.
             while (numberOfRequestedOverdraftMemorySegments > 0
                     && numberOfRequestedMemorySegments < currentPoolSize) {
                 numberOfRequestedOverdraftMemorySegments--;
                 numberOfRequestedMemorySegments++;
             }
 
+            // If pool size decreases, try to convert ordinary buffer to overdraft buffer.
+            while (numberOfRequestedMemorySegments > currentPoolSize
+                    && numberOfRequestedOverdraftMemorySegments < maxOverdraftBuffersPerGate) {
+                numberOfRequestedMemorySegments--;
+                numberOfRequestedOverdraftMemorySegments++;
+            }

Review Comment:
   Thanks @1996fanrui for the quick reply.
   
   The reason for this restriction(`numberOfRequestedOverdraftMemorySegments < maxOverdraftBuffersPerGate`) is to ensure that the case where `numberOfRequestedOverdraftMemorySegments` is larger than `maxOverdraftBuffersPerGate` does not happen. If we consider all buffers exceeding poolSize as overdraft buffers, then `numberOfRequestedOverdraftMemorySegments` is actually a field that can be removed. Actually I created FLINK-31764 to do this. But for the current PR, I don't plan to do this refactoring, the changes here are just to ensure that the total number of requested buffers does not exceed `poolSize + maxOverdraftBuffersPerGate`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org