You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Suraj Puvvada <su...@traceable.ai> on 2020/07/30 18:48:31 UTC

Does Flink automatically apply any backpressure ?

Hello

I am trying to understand if Flink has a mechanism to automatically apply
any backpressure by throttling any operators ? For example if I have a
Process function that reads from a Kafkaa source and writes to a Kafka
sink. If the process function is slow will the kafka source be
automatically throttled ?

Thanks
Suraj

Re: Does Flink automatically apply any backpressure ?

Posted by Danny Chan <yu...@gmail.com>.
Yes, just like Jake said, the back pressure happened automatically and usually there is no need to tweak it, you actually can have configure the metrics about it, see [1]

[1] https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/back_pressure.html

Best,
Danny Chan
在 2020年7月31日 +0800 AM10:28，Jake <ft...@qq.com>，写道：
>
> Hi Suraj Puvvada
>
> Yes, Flink back pressure depend on the Flink task buffer。process task will sends buffer remaining size to source, source will slow down.
>
> https://www.ververica.com/blog/how-flink-handles-backpressure
>
> Jake
>
>
> > On Jul 31, 2020, at 2:48 AM, Suraj Puvvada <su...@traceable.ai> wrote:
> >
> > Hello
> >
> > I am trying to understand if Flink has a mechanism to automatically apply any backpressure by throttling any operators ? For example if I have a Process function that reads from a Kafkaa source and writes to a Kafka sink. If the process function is slow will the kafka source be automatically throttled ?
> >
> > Thanks
> > Suraj
>

Re: Does Flink automatically apply any backpressure ?

Posted by Jake <ft...@qq.com>.
Hi Suraj Puvvada

Yes, Flink back pressure depend on the Flink task buffer。process task will sends buffer remaining size to source, source will slow down.

https://www.ververica.com/blog/how-flink-handles-backpressure <https://www.ververica.com/blog/how-flink-handles-backpressure>

Jake


> On Jul 31, 2020, at 2:48 AM, Suraj Puvvada <su...@traceable.ai> wrote:
> 
> Hello
> 
> I am trying to understand if Flink has a mechanism to automatically apply any backpressure by throttling any operators ? For example if I have a Process function that reads from a Kafkaa source and writes to a Kafka sink. If the process function is slow will the kafka source be automatically throttled ?
> 
> Thanks
> Suraj