[PR] feat: resume pipelines on restart [incubator-devlake]

["klesh (via GitHub)" <gi...@apache.org>]

klesh opened a new pull request, #7229:
URL: https://github.com/apache/incubator-devlake/pull/7229

   ### Summary
   
   Use cases: `devlake` container gets restarted accidentally
   Why: currently, all running pipelines would be marked as FAILED on restart and all progress would be dropped, one would have to rerun all of them which is quite costly.
   What: this PR would resume the running pipelines on the Subtask level.
   How: resuming pipeline on subtask level could pick up most of the work which could reduce the cost drastically
   Todo: `gitextractor` requires the repo to be cloned into the local storage for data collection, unfortunately, the repo would be gone after the container restarted, either we cache it by using a volume or we have to optimize the clone speed for the whole thing to make sense.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat: resume pipelines on restart [incubator-devlake]

["abeizn (via GitHub)" <gi...@apache.org>]

abeizn merged PR #7229:
URL: https://github.com/apache/incubator-devlake/pull/7229


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org