Flink credit based flow control

[Brian Ramprasad <br...@cs.utoronto.ca>]

Hi,
I am trying to use the most recent version of Flink over a
high latency network and I am trying to measure how long a sender may
wait for credits before it can send buffers to the receiver. Does anyone
know which function/class where I can measure at the sender side the time
spent waiting to receive the incoming credit
announcements?
 
Thanks
Brian R
 

Re: Flink credit based flow control

[zhijiang <wa...@aliyun.com>]

Hi Brian,

Actually I also thought of adding the metrics you mentioned after contributing the credit-based flow control. It should help performance tuning sometimes. If you want to add this metirc, you could trace the related process in `ResultSubpartition`. When the backlog is increasd during adding `BufferConsumer` in queue, you can check the current credits for this sub partition. Another possible way is when the subpartition view is changed from available to unavailable because of no credits, wecould add some metrics here. But even though we found the sender is blocked because of no credits, we should still need distinguish two conditions. If the sender is backpressured, then this condition of no credits is within expectation. 

Maybe it does not need to add extra metric to trace your problem. You can check whether the high latency network is caused by backpressure. And what is the flush timeout you config. Also you can trace the current metrics of outqueue.usages|length and inqueue.usags|length to find something. 

Best,
Zhijiang
------------------------------------------------------------------
From:Brian Ramprasad <br...@cs.utoronto.ca>
Send Time:2019年3月12日(星期二) 03:47
To:user <us...@flink.apache.org>
Subject:Flink credit based flow control

Hi,
I am trying to use the most recent version of Flink over a high latency network and I am trying to measure how long a sender may wait for credits before it can send buffers to the receiver. Does anyone know which function/class where I can measure at the sender side the time spent waiting to receive the incoming credit announcements?
Thanks
Brian R