You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "reswqa (via GitHub)" <gi...@apache.org> on 2023/03/02 07:49:02 UTC

[GitHub] [flink] reswqa opened a new pull request, #22067: [FLINK-31288][runtime] Disable overdraft buffer for non pipelined result partition

reswqa opened a new pull request, #22067:
URL: https://github.com/apache/flink/pull/22067

   ## What is the purpose of the change
   
   *Only pipelined / pipelined-bounded partition needs overdraft buffer. More specifically, there is no reason to request more buffers for non-pipelined (i.e. batch) shuffle. The reasons are as follows:*
   
    - *For BoundedBlockingShuffle, each full buffer will be directly released.*
    - *For SortMergeShuffle, the maximum capacity of buffer pool is 4 * numSubpartitions. It is efficient enough to spill this part of memory to disk.*
    - *For Hybrid Shuffle, the buffer pool is unbounded. If it can't get a normal buffer, it also can't get an overdraft buffer.*
   
   
   ## Brief change log
   
     - *Disable overdraft buffer for non pipelined result partition.*
   
   
   ## Verifying this change
   
   
   This change is a trivial rework / code cleanup without any test coverage.
   
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #22067: [FLINK-31288][runtime] Disable overdraft buffer for non pipelined result partition

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #22067:
URL: https://github.com/apache/flink/pull/22067#issuecomment-1451441513

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "dc1a58eb2e7966be6aed8b3cf3d21b047b5b4e53",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "dc1a58eb2e7966be6aed8b3cf3d21b047b5b4e53",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * dc1a58eb2e7966be6aed8b3cf3d21b047b5b4e53 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] reswqa merged pull request #22067: [FLINK-31288][runtime] Disable overdraft buffer for non pipelined result partition

Posted by "reswqa (via GitHub)" <gi...@apache.org>.
reswqa merged PR #22067:
URL: https://github.com/apache/flink/pull/22067


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] reswqa commented on pull request #22067: [FLINK-31288][runtime] Disable overdraft buffer for non pipelined result partition

Posted by "reswqa (via GitHub)" <gi...@apache.org>.
reswqa commented on PR #22067:
URL: https://github.com/apache/flink/pull/22067#issuecomment-1451817638

   Very thanks @1996fanrui for the quick review, merged % CI green.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org