How to access Task.isBackPressured() from a SourceFunction?

[Darren Whobrey <da...@oracle.com>]

Hi, is there a way for the UDF of a source function, extended from RichParallelSourceFunction, to access its Task instance, so as to call Task.isBackPressured()?
I'm trying to give priorities to different input sources that need to be managed from within the same source function and want to stop reading from one source according to a utilization statistic that is partially based on whether there is backpressure.
Thank you!

Re: How to access Task.isBackPressured() from a SourceFunction?

[Niklas Semmler <ni...@ververica.com>]

Hi Darren,

No, you cannot access the Task from the operator. You can access some metrics via the RuntimeContext.

getRuntimeContext().getMetricGroup() 

How does the backpressure help you here? Backpressure can originate in any operator or network connection. If it's an operator further downstream, it may take some time till it arrives at the source. What do you want to do in response? Switch to a slower source and then switch back?

Best regards,
Niklas

> On 9. Feb 2022, at 07:01, Darren Whobrey <da...@oracle.com> wrote:
> 
> Hi, is there a way for the UDF of a source function, extended from RichParallelSourceFunction, to access its Task instance, so as to call Task.isBackPressured()?
> I’m trying to give priorities to different input sources that need to be managed from within the same source function and want to stop reading from one source according to a utilization statistic that is partially based on whether there is backpressure. 
> Thank you!