You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by Ufuk Celebi <uc...@apache.org> on 2015/10/08 13:06:57 UTC

Re: How do network transmissions in Flink work?

Hey Niklas,

I’ve missed your answer. Sorry for the delay!

> On 20 Jul 2015, at 19:00, Niklas Semmler <ns...@inet.tu-berlin.de> wrote:
> 
> Hello Ufuk,
> 
> thank you very much for the answer. You helped me to bring a great deal of context into the problem :).
> 
> I have one final question: What is a good indicator that the transfer of data contained in a single ResultPartition is finished? Is there any? Or can the amount of retrieved data be retrieved from the consumer?

The result life-cycle is independent of the task life-cycle.

On the output side you can look into the “notifySubpartitionConsumed()” notification (in ResultSubpartition). This signals that every buffer has been handed over to the network or consuming task (but buffers might still be in flight).

On the input side you can look into the SingleInputGate. The FINISHED event signals that the result partition has been fully received.

Does this help? What are you trying to do with the partitions?

Again, sorry for the delay.

– Ufuk