You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@buildstream.apache.org by GitBox <gi...@apache.org> on 2021/01/12 23:55:46 UTC
[GitHub] [buildstream] cs-shadow opened a new pull request #1441: Improve CI setup for pre-merge checks
cs-shadow opened a new pull request #1441:
URL: https://github.com/apache/buildstream/pull/1441
I understand that we have other more pressing things. So, this is not urgent as such but I'm putting it out there to see how people would feel about Compose.
---
Since the YAML configuration offered by GitHub Actions leaves a lot to be desired, I have switched to docker-compose for doing all the heavy-lifting and have GitHub Actions map almost 1:1 to services defined in the Compose file.
This setup also makes it easy to test changes locally, for the most part. So, we can consider documenting this as a local development thing too if we go with this.
While we're here, I've also enabled parallelism for the tests. For the degree of parallelism, I've arrived at 4 as the "sweet spot" for the runners provided by GitHub. Althoough my data set was rather small, so take that with a grain of salt :)
---
* ci: Replace docker CLI monstrosity with Compose
Replace our bespoke and rather long to type `docker` CLI invocations
with more manageable docker-compose commands. This should also be useful
for better replicating what the CI does on laptops, and also to help
users to run tests inside a container (especially when that's not
possible natively like on MacOS).
On a technical note, Compose does support YAML anchors, unlike GitHub
Actions. That allows us to reuse the common configuration options
without having to duplicate them everywhere.
This also removes most of the duplication in the image names, save for
the test about missing dependencies. That should also be fixable easily.
* ci: Enable parallelism for tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] cs-shadow commented on pull request #1441: ci: Simplify pipelines using Docker Compose
Posted by GitBox <gi...@apache.org>.
cs-shadow commented on pull request #1441:
URL: https://github.com/apache/buildstream/pull/1441#issuecomment-760577415
This should now be all rebased and ready to go. The two remaining workflows have now been updated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] cs-shadow commented on pull request #1441: Improve CI setup for pre-merge checks
Posted by GitBox <gi...@apache.org>.
cs-shadow commented on pull request #1441:
URL: https://github.com/apache/buildstream/pull/1441#issuecomment-759417558
> Will this approach help us run multiple parallel containers for the sake of the remote execution tests ?
Indeed. Compose supports bringing up multiple services, have dependencies across them etc so it should be reasonably straightforward to add remote executetion tests. In fact, the old remote execution tests already were using such a compose file: https://gitlab.com/BuildStream/buildstream/-/blob/master/.gitlab-ci/buildgrid-remote-execution.yml
> Overall it looks good, but I think we should update the merge.yml and release.yml workflows to build the docs this way also.
Thanks! I'll fix this and fix a couple of other reamining TODOs later today.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] cs-shadow merged pull request #1441: ci: Simplify pipelines using Docker Compose
Posted by GitBox <gi...@apache.org>.
cs-shadow merged pull request #1441:
URL: https://github.com/apache/buildstream/pull/1441
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] gtristan commented on pull request #1441: Improve CI setup for pre-merge checks
Posted by GitBox <gi...@apache.org>.
gtristan commented on pull request #1441:
URL: https://github.com/apache/buildstream/pull/1441#issuecomment-759247905
Will this approach help us run multiple parallel containers for the sake of the remote execution tests ?
Overall it looks good, but I think we should update the `merge.yml` and `release.yml` workflows to build the docs this way also.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org