[GitHub] [flink] zentol commented on a change in pull request #9296: [FLINK-13476][coordination] Release pipelined partitions on vertex restart / job failure

[GitBox <gi...@apache.org>]

zentol commented on a change in pull request #9296: [FLINK-13476][coordination] Release pipelined partitions on vertex restart / job failure
URL: https://github.com/apache/flink/pull/9296#discussion_r309380890
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java
 ##########
 @@ -789,8 +789,21 @@ else if (current == RUNNING || current == DEPLOYING) {
 				// else: fall through the loop
 			}
 
-			else if (current == FINISHED || current == FAILED) {
-				// nothing to do any more. finished/failed before it could be cancelled.
+			else if (current == FINISHED) {
+				// finished before it could be cancelled.
+				// in any case, the task is removed from the TaskManager already
+
+				// a pipelined partition whose consumer has never been deployed could still be buffered on the TM
+				// release it here since pipelined partitions for FINISHED executions aren't handled elsewhere
+				// covers the following cases:
+				// 		a) restarts of this vertex
+				// 		b) a global failure (which may result in a FAILED job state)
+				sendReleaseIntermediateResultPartitionsRpcCall();
 
 Review comment:
   sigh...I figured we wouldn't need it since in case of a suspend (and the corresponding JM disconnect) we'd be releasing all partition, but I forgot about those pesky pipelined partitions again....

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services