You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Le Xu <sh...@gmail.com> on 2022/03/03 05:31:45 UTC

Customizing backpressure mechanism for RichParallelSourceFunction

Hello!

I have a dataflow pipeline built using Flink's RichParallelSourceFunction
as parallel sources. I'm wondering if there are any mechanisms that I could
use to implement *ack-based* back-pressure mechanism by sending ACK
messages from operators (within the runtime) to source functions to achieve
user-level back-pressure mechanisms? I understand that Flink uses flow
control to perform back pressure internally but I'd like to check whether
it is possible at all to send any events from operator to sources.

Thanks in advance!

Le

Re: Customizing backpressure mechanism for RichParallelSourceFunction

Posted by Chesnay Schepler <ch...@apache.org>.

It's not possible to send events to sources; data only flows in 1 direction.

On 03/03/2022 06:31, Le Xu wrote:
> Hello!
>
> I have a dataflow pipeline built using Flink's 
> RichParallelSourceFunction as parallel sources. I'm wondering if there 
> are any mechanisms that I could use to implement *ack-based* 
> back-pressure mechanism by sending ACK messages from operators (within 
> the runtime) to source functions to achieve user-level back-pressure 
> mechanisms? I understand that Flink uses flow control to perform back 
> pressure internally but I'd like to check whether it is possible at 
> all to send any events from operator to sources.
>
> Thanks in advance!
>
> Le