which is the most affecting operator when multi bottlenecks exist in application

[vtygoss <vt...@126.com>]

Hi, community!


When there are multi bottlenecks(especially discontinuous operators) in the application shown as below, how do we determine which operator is the most affecting performance? It's the question of which one to optimize firstly and the most effectively. 


In my mind, back pressure progagtes from back to forward due to network-credit-based data transformance, so the thoughput of upstream operator which is not at back pressure is higher than downstream operator which is at back pressure. So, i think the closest operator which is at back pressure to sink is the most affecting performance. 


Do i get that wrong? Is there any documents? Thanks for your any replies or suggestions. 


Best Regards!

Re: which is the most affecting operator when multi bottlenecks exist in application

[Hangxiang Yu <ma...@gmail.com>]

Hi, vtygoss.
I think your analysis is basically right.
You could refer to [1] to see more details about how to find the source of
backpressure.

[1] https://flink.apache.org/2021/07/07/backpressure.html

Best,
Hangxiang.

On Wed, Jul 27, 2022 at 4:35 PM vtygoss <vt...@126.com> wrote:

> Hi, community!
>
>
> When there are multi bottlenecks(especially discontinuous operators) in
> the application shown as below, how do we determine which operator is the
> most affecting performance? It's the question of which one to optimize
> firstly and the most effectively.
>
>
> In my mind, back pressure progagtes from back to forward due to
> network-credit-based data transformance, so the thoughput of upstream
> operator which is not at back pressure is higher than downstream operator
> which is at back pressure. So, i think the closest operator which is at
> back pressure to sink is the most affecting performance.
>
>
> Do i get that wrong? Is there any documents? Thanks for your any replies
> or suggestions.
>
>
> Best Regards!
>
>
>
>
>
>