You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/08/12 10:32:10 UTC

[GitHub] [incubator-seatunnel] EricJoy2048 opened a new issue, #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design

EricJoy2048 opened a new issue, #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398

   ### Search before asking
   
   - [X] I had searched in the [feature](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
   
   
   ### Description
   
   Now, We only record and maintain the status of jobs and job list in memory. If the server done, everything will lost. So
   we need design the `Job History Storage` to save status and job list to disk( Or HDFS).
   
   ### Usage Scenario
   
   _No response_
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-seatunnel] ic4y commented on issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design

Posted by GitBox <gi...@apache.org>.
ic4y commented on issue #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398#issuecomment-1217974650

   <img width="820" alt="image" src="https://user-images.githubusercontent.com/83933160/185138149-517b5d43-5b30-45db-865d-68888764b1a4.png">
   
   **Cluster restart**
       When the cluster restarts, the Job State will be restored from the file system to Imap. At this time, SeatunnelServer only needs to take out the Job that needs to be resumed from Imap to resume execution.
   
   **Master Drift**
   When the Master node fails, the cluster will select another node as the Master at this time. At this time, it is necessary to restore the state of the JobMaster to continue processing tasks on the new node. The deployed Task will report the status to the new Master.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design
URL: https://github.com/apache/incubator-seatunnel/issues/2398


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-seatunnel] EricJoy2048 commented on issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design

Posted by GitBox <gi...@apache.org>.
EricJoy2048 commented on issue #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398#issuecomment-1214767948

   > Let me do it
   
   Ok, Thank you


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398#issuecomment-1447334725

   This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-seatunnel] ic4y commented on issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design

Posted by GitBox <gi...@apache.org>.
ic4y commented on issue #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398#issuecomment-1214545370

   Let me do it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [ST-Engine][JobHistoryStorage] Job History Storage Design [seatunnel]

Posted by "sheng-jie (via GitHub)" <gi...@apache.org>.
sheng-jie commented on issue #2398:
URL: https://github.com/apache/seatunnel/issues/2398#issuecomment-1865433175

   @ic4y Any Progress  on that?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398#issuecomment-1482082442

   This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org