[GitHub] [buildstream] gtristan commented on issue #534: Failed builds not pushing artifacts on quit

[GitBox <gi...@apache.org>]

gtristan commented on issue #534:
URL: https://github.com/apache/buildstream/issues/534#issuecomment-1220307175

   After discussing this in the meeting yesterday, it is worth mentioning that whether this is a bug is not very clear.
   
   Clearly, it is desirable that if a build fails, it is uploaded to a cache by default (default strategy is `--on-error quit`).
   
   This *quit* means to process any ongoing jobs and exit - which BuildStream clearly does correctly.
   
   If we want this to work in an improved manner, it raises some questions:
   * When running `bst build`:
     * When a *build* job fails, should *that* artifact be uploaded even when instructed to *quit*
     * When a *build* job fails, should *all* artifacts which were built in this session still be *pushed*, even if those have not been scheduled yet and we have been instructed to *quit* ?
   * When running `bst push`:
     * When a *push* job fails, should we ignore the instruction to *quit* ?
     * When a *push* job fails, should we have a double standard and obey the instruction to *quit* even though it is contrary to `bst build` behavior ?
   
   Perhaps there is something intuitive here which is to *relate the command name with the queue performing that action* and to infer that the `--on-error` is related to the imperative of *that command* ?
   
   In other words:
   * When running `bst pull` and a `pull` fails, or any job placed *before* the `pull` queue fails, we cease queuing jobs into the `pull` queue and any queues before the `pull` queue, but we *continue* to process jobs in queues that post process `pull` jobs
     * It is intentional that this `bst pull` feels unnatural, there happens to not be any post processing to `pull` actions
   * When running `bst build` and a `build` fails, or any job placed *before* the `build` queue fails (e.g. `pull` or the fallback to `fetch`), we cease queuing jobs into the `build` queue and any queues before the `build` queue, but we *continue* to process jobs in queues that post process `build` jobs, e.g. `push`
   
   With this approach, we can encode one of the scheduler queues to be *the imperative queue* and treat the `--on-error quit` strategy generically for all commands, in a way which appears to make most sense to the user, given the context of the command they are running.
   
   Would this make sense ?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org