You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/09/15 06:51:40 UTC
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #2699: [Feature][ST-Engine] Put Job Status in IMap
Hisoka-X commented on code in PR #2699:
URL: https://github.com/apache/incubator-seatunnel/pull/2699#discussion_r971559969
##########
seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/SeaTunnelServer.java:
##########
@@ -73,12 +81,52 @@ public class SeaTunnelServer implements ManagedService, MembershipAwareService,
private final SeaTunnelConfig seaTunnelConfig;
+ /**
+ * IMap key is jobId and value is a Tuple2
Review Comment:
Vaule not Tuple2
##########
seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/SeaTunnelServer.java:
##########
@@ -73,12 +81,52 @@ public class SeaTunnelServer implements ManagedService, MembershipAwareService,
private final SeaTunnelConfig seaTunnelConfig;
+ /**
+ * IMap key is jobId and value is a Tuple2
+ * Tuple2 key is JobMaster init timestamp and value is the jobImmutableInformation which is sent by client when submit job
+ * <p>
+ * This IMap is used to recovery runningJobInfoIMap in JobMaster when a new master node active
+ */
+ private IMap<Long, RunningJobInfo> runningJobInfoIMap;
+
+ /**
+ * IMap key is one of jobId {@link org.apache.seatunnel.engine.server.dag.physical.PipelineLocation} and
+ * {@link org.apache.seatunnel.engine.server.execution.TaskGroupLocation}
+ * <p>
+ * The value of IMap is one of {@link JobStatus} {@link org.apache.seatunnel.engine.core.job.PipelineState}
+ * {@link org.apache.seatunnel.engine.server.execution.ExecutionState}
+ * <p>
+ * This IMap is used to recovery runningJobStateIMap in JobMaster when a new master node active
+ */
+ IMap<Object, Object> runningJobStateIMap;
Review Comment:
It's strange jobID map PipelineState, does JobStatus can't contains PipelineState?
##########
seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/SeaTunnelServer.java:
##########
@@ -73,12 +81,52 @@ public class SeaTunnelServer implements ManagedService, MembershipAwareService,
private final SeaTunnelConfig seaTunnelConfig;
+ /**
+ * IMap key is jobId and value is a Tuple2
+ * Tuple2 key is JobMaster init timestamp and value is the jobImmutableInformation which is sent by client when submit job
+ * <p>
+ * This IMap is used to recovery runningJobInfoIMap in JobMaster when a new master node active
+ */
+ private IMap<Long, RunningJobInfo> runningJobInfoIMap;
+
+ /**
+ * IMap key is one of jobId {@link org.apache.seatunnel.engine.server.dag.physical.PipelineLocation} and
+ * {@link org.apache.seatunnel.engine.server.execution.TaskGroupLocation}
+ * <p>
+ * The value of IMap is one of {@link JobStatus} {@link org.apache.seatunnel.engine.core.job.PipelineState}
+ * {@link org.apache.seatunnel.engine.server.execution.ExecutionState}
+ * <p>
+ * This IMap is used to recovery runningJobStateIMap in JobMaster when a new master node active
+ */
+ IMap<Object, Object> runningJobStateIMap;
+
+ /**
+ * IMap key is one of jobId {@link org.apache.seatunnel.engine.server.dag.physical.PipelineLocation} and
+ * {@link org.apache.seatunnel.engine.server.execution.TaskGroupLocation}
+ * <p>
+ * The value of IMap is one of {@link org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan} stateTimestamps
+ * {@link org.apache.seatunnel.engine.server.dag.physical.SubPlan} stateTimestamps
+ * {@link org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex} stateTimestamps
+ * <p>
+ * This IMap is used to recovery runningJobStateTimestampsIMap in JobMaster when a new master node active
+ */
+ IMap<Object, Long[]> runningJobStateTimestampsIMap;
Review Comment:
I think this information just subset of RunningJobInfo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org