You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Lu (Jira)" <ji...@apache.org> on 2022/11/24 10:01:00 UTC

[jira] [Created] (FLINK-30193) could flink has dynamic rebalance according to the load of downstream operators

Lu created FLINK-30193:
--------------------------

             Summary: could flink has dynamic rebalance  according to the load of downstream operators 
                 Key: FLINK-30193
                 URL: https://issues.apache.org/jira/browse/FLINK-30193
             Project: Flink
          Issue Type: New Feature
          Components: API / DataStream
    Affects Versions: 1.15.2
            Reporter: Lu


Different records cause different loads to the downstream and different task slot has different compute power. So rebalancing according to the load of downstream operators will greatly reduce data skew and improve resource utilization.

I thaught the point is how to gain the load of downstream.  I have seen the 

StreamPartitioner , and it dosen't has any context or method to get the load information of downstream operators. So ordinary developers cannot easily implement such functions.

finally，Could flink provide such a function?

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)