[GitHub] tweise commented on a change in pull request #7249: [FLINK-11048] Ability to programmatically execute streaming pipeline with savepoint restore

[GitBox <gi...@apache.org>]

tweise commented on a change in pull request #7249: [FLINK-11048] Ability to programmatically execute streaming pipeline with savepoint restore
URL: https://github.com/apache/flink/pull/7249#discussion_r239665450
 
 

 ##########
 File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/RemoteStreamEnvironment.java
 ##########
 @@ -169,39 +170,46 @@ public RemoteStreamEnvironment(String host, int port, Configuration clientConfig
 		}
 	}
 
-	@Override
-	public JobExecutionResult execute(String jobName) throws ProgramInvocationException {
-		StreamGraph streamGraph = getStreamGraph();
-		streamGraph.setJobName(jobName);
-		transformations.clear();
-		return executeRemotely(streamGraph, jarFiles);
+	protected List<URL> getJarFiles() throws ProgramInvocationException {
+		return jarFiles;
 	}
 
 	/**
 	 * Executes the remote job.
 	 *
-	 * @param streamGraph
-	 *            Stream Graph to execute
+	 * @param streamExecutionEnvironment
+	 *            Execution Environment with Stream Graph to execute
 	 * @param jarFiles
 	 * 			  List of jar file URLs to ship to the cluster
 	 * @return The result of the job execution, containing elapsed time and accumulators.
 	 */
-	protected JobExecutionResult executeRemotely(StreamGraph streamGraph, List<URL> jarFiles) throws ProgramInvocationException {
+	public static JobExecutionResult executeRemotely(StreamExecutionEnvironment streamExecutionEnvironment,
 
 Review comment:
   If we want to retain / put back the existing `protected JobExecutionResult executeRemotely`, then we would need to pass around `StreamGraph` instead of the environment. There are then other bits and pieces that also need to be passed around, such as the class loader. That's a bit difficult to see unless you actually change the code.
   
   That protected method should ideally not be referenced outside of Flink, but as it happens we just used it to bridge the gap in Beam.. 
   
   Regarding the static method, it isn't ideal to have this in the public API either, but that's the purpose of the change.. Every change to the parameter list becomes an incompatible change and may cause pain going forward. 
   
   I have also added the ability to set the savepoint option on the environment, which seems more user friendly. The minimum change btw is here, in case we want to reconsider the route we are taking here: https://github.com/apache/flink/compare/master...tweise:FLINK-11048.savepointRestore.min?expand=1 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services