[GitHub] [flink] zhuzhurk commented on a change in pull request #10278: [FLINK-14735][scheduler] Improve scheduling of all-to-all partitions with ALL input constraint for legacy scheduler

[GitBox <gi...@apache.org>]

zhuzhurk commented on a change in pull request #10278: [FLINK-14735][scheduler] Improve scheduling of all-to-all partitions with ALL input constraint for legacy scheduler
URL: https://github.com/apache/flink/pull/10278#discussion_r349057067
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java
 ##########
 @@ -1090,6 +1104,26 @@ else if (current == CANCELED || current == FAILED) {
 		}
 	}
 
+	private void finishPartitionsAndScheduleOrUpdateConsumers() {
+		final List<IntermediateResultPartition> newlyFinishedResults = getVertex().finishAllBlockingPartitions();
+		if (newlyFinishedResults.isEmpty()) {
+			return;
+		}
+
+		final HashSet<ExecutionVertex> consumersToSchedule = new HashSet<>();
 
 Review comment:
   One side effect of using a Set is that the vertices are disordered.
   It's not problematic but may make the scheduling logs a bit mess.
   
   e.g. A has 2 downstream JobVertices B, C, when all instances of A finishes
   previously it is:
   
   > ... B1 ... transitioned from CREATED to SCHEDULED
   > ... B2 ... 
   > ...
   > ... Bn ... 
   > ... C1 ... 
   > ... C2 ... 
   > ...
   > ... Cn ... transitioned from CREATED to SCHEDULED
   
   With this change, it might be:
   
   > ... B2 ... transitioned from CREATED to SCHEDULED
   > ... Cn ... 
   > ...
   > ... B3 ... 
   > ... C5 ... 
   > ... C2 ... 
   > ...
   > ... B1 ... transitioned from CREATED to SCHEDULED
   
   We can avoid the disorders by adding the vertices to schedule in a list and using the set to track vertices added. But I'm not sure whether it's needed.
   WDYT?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services