You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "Hisoka-X (via GitHub)" <gi...@apache.org> on 2024/04/09 07:00:13 UTC

[I] [Improve][Zeta] Does not save checkpoint file when job finished. [seatunnel]

Hisoka-X opened a new issue, #6670:
URL: https://github.com/apache/seatunnel/issues/6670

   ### Search before asking
   
   - [X] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
   
   
   ### Description
   
   Currently, at the end of the job (with `FINISHED` status), the last checkpoint will be called to perform shutdown and resource cleanup operations, but when the checkpoint finished, the corresponding checkpoint file will be kept in the file system. A job that ends with `FINISHED` does not need to save the checkpoint file, so we need to skip the logic of saving the checkpoint file when the last checkpoint is completed.
   
   ### Usage Scenario
   
   _No response_
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Improve][Zeta] Does not save checkpoint file when job finished. [seatunnel]

Posted by "liunaijie (via GitHub)" <gi...@apache.org>.
liunaijie commented on issue #6670:
URL: https://github.com/apache/seatunnel/issues/6670#issuecomment-2046467177

   @Hisoka-X From your description and the code, i find the checkpoint is write to file by this code： https://github.com/apache/seatunnel/blob/dev/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/CheckpointCoordinator.java#L771
   
   So i am going to add some logical to check job status, if it's finished, skip this method call, is this right?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Improve][Zeta] Does not save checkpoint file when job finished. [seatunnel]

Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X closed issue #6670: [Improve][Zeta] Does not save checkpoint file when job finished.
URL: https://github.com/apache/seatunnel/issues/6670


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Improve][Zeta] Does not save checkpoint file when job finished. [seatunnel]

Posted by "liunaijie (via GitHub)" <gi...@apache.org>.
liunaijie commented on issue #6670:
URL: https://github.com/apache/seatunnel/issues/6670#issuecomment-2046338162

   I want take this task


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Improve][Zeta] Does not save checkpoint file when job finished. [seatunnel]

Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on issue #6670:
URL: https://github.com/apache/seatunnel/issues/6670#issuecomment-2046472054

   > @Hisoka-X From your description and the code, i find the checkpoint is write to file by this code： https://github.com/apache/seatunnel/blob/dev/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/CheckpointCoordinator.java#L771
   > 
   > So i am going to add some logical to check job status, if it's finished, skip this method call, is this right?
   
   Almost right. But just need check `completedCheckpoint`'s checkpointType is `COMPLETED_POINT_TYPE` is enough. You can use https://github.com/apache/seatunnel/blob/19888e7969475389b2e9f120a92267a2294dc8cd/seatunnel-engine/seatunnel-engine-core/src/main/java/org/apache/seatunnel/engine/core/checkpoint/CheckpointType.java#L94


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org