You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/08/12 10:32:10 UTC
[GitHub] [incubator-seatunnel] EricJoy2048 opened a new issue, #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design
EricJoy2048 opened a new issue, #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398
### Search before asking
- [X] I had searched in the [feature](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
### Description
Now, We only record and maintain the status of jobs and job list in memory. If the server done, everything will lost. So
we need design the `Job History Storage` to save status and job list to disk( Or HDFS).
### Usage Scenario
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ic4y commented on issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design
Posted by GitBox <gi...@apache.org>.
ic4y commented on issue #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398#issuecomment-1217974650
<img width="820" alt="image" src="https://user-images.githubusercontent.com/83933160/185138149-517b5d43-5b30-45db-865d-68888764b1a4.png">
**Cluster restart**
When the cluster restarts, the Job State will be restored from the file system to Imap. At this time, SeatunnelServer only needs to take out the Job that needs to be resumed from Imap to resume execution.
**Master Drift**
When the Master node fails, the cluster will select another node as the Master at this time. At this time, it is necessary to restore the state of the JobMaster to continue processing tasks on the new node. The deployed Task will report the status to the new Master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design
URL: https://github.com/apache/incubator-seatunnel/issues/2398
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design
Posted by GitBox <gi...@apache.org>.
EricJoy2048 commented on issue #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398#issuecomment-1214767948
> Let me do it
Ok, Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398#issuecomment-1447334725
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ic4y commented on issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design
Posted by GitBox <gi...@apache.org>.
ic4y commented on issue #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398#issuecomment-1214545370
Let me do it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [ST-Engine][JobHistoryStorage] Job History Storage Design [seatunnel]
Posted by "sheng-jie (via GitHub)" <gi...@apache.org>.
sheng-jie commented on issue #2398:
URL: https://github.com/apache/seatunnel/issues/2398#issuecomment-1865433175
@ic4y Any Progress on that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2398: [ST-Engine][JobHistoryStorage] Job History Storage Design
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #2398:
URL: https://github.com/apache/incubator-seatunnel/issues/2398#issuecomment-1482082442
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org