[GitHub] [incubator-celeborn] FMX commented on a diff in pull request #1260: [CELEBORN-326)] [Flink] lifecycleManager supports flink-yarn-session mode to handle multiple Flink jobs.

["FMX (via GitHub)" <gi...@apache.org>]

FMX commented on code in PR #1260:
URL: https://github.com/apache/incubator-celeborn/pull/1260#discussion_r1113903442


##########
client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/RemoteShuffleMaster.java:
##########
@@ -57,28 +61,36 @@ public RemoteShuffleMaster(ShuffleMasterContext shuffleMasterContext) {
 
   @Override
   public void registerJob(JobShuffleContext context) {
-    JobID jobId = context.getJobId();
+    JobID jobID = context.getJobId();
+    if (lifecycleManager == null) {
+      synchronized (RemoteShuffleMaster.class) {
+        if (lifecycleManager == null) {
+          // use first jobID as celeborn shared appId for all other flink jobs
+          celebornAppId = FlinkUtils.toCelebornAppId(jobID);
+          CelebornConf celebornConf =
+              FlinkUtils.toCelebornConf(shuffleMasterContext.getConfiguration());
+          lifecycleManager = new LifecycleManager(celebornAppId, celebornConf);
+        }
+      }
+    }
+
     Future<?> submit =

Review Comment:
   Looks like unnecessary async block.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org