You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by Thomas Bouron <th...@cloudsoft.io> on 2019/12/05 15:37:49 UTC
CI/CD pipelines
Hi Brooklyners!
So you may have noticed a few more emails these past few days coming from
Jenkins. I'm happy to announce that I finally managed to make all git
submodules build into docker containers on Jenkins, piloted by Jenkins
pipelines (Jenkinsfile). And so far, it is incredibly stable! 🎉
We now have multibranch pipelines for:
- brooklyn-server:
https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-server/
- brooklyn-library:
https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-library/
- brooklyn-ui:
https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-ui/
- brooklyn-client:
https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-client/
- brooklyn-dist:
https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-dist/
- brooklyn: https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn/
Only unstable `master` builds will trigger an email alert to `
dev@brooklyn.apache.org`.
Multibranch means that `master` branch of each git repo (as well as all
PRs) are built automatically on each commit. In the case of `master`
branch, the artifacts are also deployed onto maven central. In the case of
PRs, the status is reported back to GitHub and a comment `restest this
please` *should* trigger a rebuild (however, I don't think this is
necessary anymore as the builds are stable)
Each build first create their own docker image, which is then used to build
the project inside it. The jenkins pipeline configuration has also been
added to git for an easier maintenance (`Jenkinsfile` at the root of each
git repo)
There are still things I should / would like to do though:
*1. Optimise artifact deployments*
Currently, artifacts are deployed onto maven central by each build
triggered by the `master` branch of each git repo. This is somewhat
inefficient so I would propose to deploy artifacts only when the main
`brooklyn` project is built. This build is triggered for each commit on
`master` branch as well as each successful build of downstream projects.
WDYT?
*2. Automate website/doc deployment*
`brooklyn-docs` is currently not built automatically by Jenkins. But this
can be added relatively easily and on top of that, the deployment of the
website/doc can also be automated with the new .asfyaml feature [1]
*3. Fix integration tests*
I created a PR [2] to apply all the changes I talked about above,
specifically for the Integration tests. On top of a review, we should also
look at fixing the failing tests.
*4. Fix windows tests*
We have a Jenkins job to build Brooklyn on windows, but a couple of tests
fail despite the push from few contributors few months ago. It would be
great to iron out the remaining ones. WDYT?
[1]
https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories
[2] https://github.com/apache/brooklyn-dist/pull/144
--
Thomas Bouron
Senior Software Engineer
*Cloudsoft <https://cloudsoft.io/> *| Bringing Business to the Cloud
GitHub: https://github.com/tbouron
Twitter: https://twitter.com/eltibouron
Re: CI/CD pipelines
Posted by Thomas Bouron <th...@cloudsoft.io>.
I forgot another item to add to the list:
*5. Build and publish Brooklyn docker image*
We already have the dockerfile necessary to do this. But I went ahead and
implemented to be built AND published under `apache/brooklyn` on docker
hub, by Jenkins! Here is the PR [1] but unfortunately, Jenkins does not
want to play ball [2] so we are blocked on this front until it gets fixed.
But after that, a simple `docker run apache/brooklyn` will give you a
running Brooklyn.
[1] https://github.com/apache/brooklyn-dist/pull/148
[2] https://issues.apache.org/jira/browse/INFRA-19523
On Thu, 5 Dec 2019 at 15:37, Thomas Bouron <th...@cloudsoft.io>
wrote:
> Hi Brooklyners!
>
> So you may have noticed a few more emails these past few days coming from
> Jenkins. I'm happy to announce that I finally managed to make all git
> submodules build into docker containers on Jenkins, piloted by Jenkins
> pipelines (Jenkinsfile). And so far, it is incredibly stable! 🎉
>
> We now have multibranch pipelines for:
> - brooklyn-server:
> https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-server/
> - brooklyn-library:
> https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-library/
> - brooklyn-ui:
> https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-ui/
> - brooklyn-client:
> https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-client/
> - brooklyn-dist:
> https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn-dist/
> - brooklyn: https://builds.apache.org/view/B/view/Brooklyn/job/brooklyn/
> Only unstable `master` builds will trigger an email alert to `
> dev@brooklyn.apache.org`.
>
> Multibranch means that `master` branch of each git repo (as well as all
> PRs) are built automatically on each commit. In the case of `master`
> branch, the artifacts are also deployed onto maven central. In the case of
> PRs, the status is reported back to GitHub and a comment `restest this
> please` *should* trigger a rebuild (however, I don't think this is
> necessary anymore as the builds are stable)
> Each build first create their own docker image, which is then used to
> build the project inside it. The jenkins pipeline configuration has also
> been added to git for an easier maintenance (`Jenkinsfile` at the root of
> each git repo)
>
> There are still things I should / would like to do though:
>
> *1. Optimise artifact deployments*
> Currently, artifacts are deployed onto maven central by each build
> triggered by the `master` branch of each git repo. This is somewhat
> inefficient so I would propose to deploy artifacts only when the main
> `brooklyn` project is built. This build is triggered for each commit on
> `master` branch as well as each successful build of downstream projects.
> WDYT?
>
> *2. Automate website/doc deployment*
> `brooklyn-docs` is currently not built automatically by Jenkins. But this
> can be added relatively easily and on top of that, the deployment of the
> website/doc can also be automated with the new .asfyaml feature [1]
>
> *3. Fix integration tests*
> I created a PR [2] to apply all the changes I talked about above,
> specifically for the Integration tests. On top of a review, we should also
> look at fixing the failing tests.
>
> *4. Fix windows tests*
> We have a Jenkins job to build Brooklyn on windows, but a couple of tests
> fail despite the push from few contributors few months ago. It would be
> great to iron out the remaining ones. WDYT?
>
> [1]
> https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories
> [2] https://github.com/apache/brooklyn-dist/pull/144
>
> --
> Thomas Bouron
> Senior Software Engineer
>
> *Cloudsoft <https://cloudsoft.io/> *| Bringing Business to the Cloud
>
> GitHub: https://github.com/tbouron
> Twitter: https://twitter.com/eltibouron
>
--
Thomas Bouron
Senior Software Engineer
*Cloudsoft <https://cloudsoft.io/> *| Bringing Business to the Cloud
GitHub: https://github.com/tbouron
Twitter: https://twitter.com/eltibouron