You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Lijie Wang (Jira)" <ji...@apache.org> on 2022/10/17 11:42:00 UTC

[jira] [Created] (FLINK-29663) Further improvements of adaptive batch scheduler

Lijie Wang created FLINK-29663:
----------------------------------

             Summary: Further improvements of adaptive batch scheduler
                 Key: FLINK-29663
                 URL: https://issues.apache.org/jira/browse/FLINK-29663
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Coordination
            Reporter: Lijie Wang
            Assignee: Lijie Wang
             Fix For: 1.17.0


In Flink 1.15, we introduced the adaptive batch scheduler to automatically decide parallelisms of job vertices for batch jobs.  In this issue, we will  further optimize it by changing the subpartition range division algorithm: change it from dividing according to the number of subpartitions(the number of subpartitions within each subpartition range is basically the same) to dividing according to the amount of data in subpartition ranges (the amount of data within each subpartition range is basically the same).

More details see [https://docs.google.com/document/d/1Qyq3qDkBCUNupajVJpFTKp3fHQJtwIu9luM7T52k1Oo]

 
This is the umbrella ticket for the improvements.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)