You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kie.apache.org by Toshiya Kobayashi <to...@gmail.com> on 2024/07/11 08:18:05 UTC
[DISCUSS][architecture] knowledge around Jenkins and kogito-pipelines
Hi,
This thread is to share knowledge around Jenkins and kogito-pipelines.
I'm Toshiya Kobayashi working on drools, not being familiar with Jenkins
and kogito-pipelines, so I started learning them.
I wrote slides to illustrate relationships of jobs and jenkinsfile etc.
It's work-in-progress, but it may help the understanding. Please feel free
to validate and/or share your thoughts on this. Thanks!
https://docs.google.com/presentation/d/13IwLkQOOQcB4-9_uMgLFvGNKFkc682crsE9MDKvLa3M
Feel free to use this thread to share such knowledge, so we can become more
comfortable in contributing to CI/CD.
Cheers,
Toshiya
Re: [DISCUSS][architecture] knowledge around Jenkins and kogito-pipelines
Posted by Tiago Bento <ti...@apache.org>.
Hi Toshiya and Paolo,
Thanks for this effort, indeed helpful. Sharing my knowledge about KIE Tools.
There are only two folders to know -- .ci (Jenkins) and
.github/workflows (GitHub Actions). Everything else can be found
starting from there. Here's a breakdown:
## RELEASES (Jenkins)
- .ci/incubator-kie-tools-ci-build.Dockerfile
Image used to run all Jenkins jobs (I.e., AUTOMATIONs defined on the
release procedure document)
- .ci/jenkins/Jenkinsfile.*
Implementation of AUTOMATIONs defined on the release procedure
document and other misc. jobs like "daily-dev", which publishes the
`main` branch everyday. E.g., https://sandbox.kie.org/dev, and
"weekly" job, for weekly timestamped SNAPSHOTs.
## PR CHECKS AND CI (GitHub Actions)
- .github/workflows/ci_build.yml
Runs on every PR (partial mode enabled) and on every commit to
`main` (always full mode).
- .github/workflows/ci_check_code_formatting.yml
Checks if code is properly formatted. Runs on every PR and commit to `main`.
- .github/workflows/ci_check_dependencies_consistency.yaml
Checks if dependencies are correctly aligned between all packages.
Runs on every PR and commit to `main`.
- .github/workflows/ci_codeql.yml
Static code analysis. Runs on every PR and commit to `main`.
## MISC. (GitHub Actions)
- .github/workflows/publish_emscripten_fs.yml
Manually-triggered job to build and publish new versions of
https://www.npmjs.com/package/@kie-tools/emscripten-fs. Will run on
PRs (skipping publication) if `publish_emscripten_fs.yml` is changed.
Very stable, so almost never executed. This could be "inlined" into
the `packages` folder though, as a regular package. Was done like that
to prevent the need of having Docker installed in the early days of
`kie-tools`...
## EXTENDED SERVICES NATIVE COMPILATION RELEASE, CI & PR CHECKS
(GitHub Actions, for macOS / Windows, which are not available on
Jenkins)
- .github/workflows/release_build_extended_services.yml
Invoked by the two jobs below with publication skipped or not.
Builds and publishes `extended-services` for macOS and Windows.
- .github/workflows/release_dry_run_extended_services.yml
Runs the jobs above with publication skipped on PRs that change relevant code.
- .github/workflows/release_publish_extended_services.yml
Runs the jobs above with publication enabled. Manually invoked.
##
Let me know if you have questions or simply want to know more.
Regards,
Tiago Bento
On Thu, Jul 11, 2024 at 5:05 AM Paolo Bizzarri <pi...@gmail.com> wrote:
>
> Hi Toshiya,
>
> thank you a lot for the diagram, it is very clear and very well done.
>
> I am myself learning the ropes here, and here are some pointers to code and
> documentation that is good to use.
>
> 1) the code for the pipelines is in
> https://github.com/apache/incubator-kie-kogito-pipelines/ and there is a
> very nice initial readme, that seems reasonably up to date and detailed. If
> you know nothing about the topic, I strongly suggest looking into this.
>
> 2) for the jenkins files, there is a good documentation in the directory
> https://github.com/apache/incubator-kie-kogito-pipelines/tree/main/docs. it
> gives a good overview of the various type of pipelines and of the jenkins
> jobs. For what I can see, again, it is reasonably up to date. I suggest
> starting from
> https://github.com/apache/incubator-kie-kogito-pipelines/blob/main/docs/jenkins.md
>
> 3) kie kogito pipelines contains some general code for pipelines,
> configurations and the likes. HOWEVER, each repository has some extra code
> that you should check, which is under the .ci root directory. for example,
> for drools you can check it here.
> https://github.com/apache/incubator-kie-drools/tree/main/.ci
>
> 4) kie-tools apparently is a different story and I know little about it, so
> all of this is just for the first three parts - drools, optaplanner, kogito.
>
> Regards
>
> Paolo
>
> On Thu, Jul 11, 2024 at 10:19 AM Toshiya Kobayashi <
> toshiyakobayashi@gmail.com> wrote:
>
> > Hi,
> >
> > This thread is to share knowledge around Jenkins and kogito-pipelines.
> >
> > I'm Toshiya Kobayashi working on drools, not being familiar with Jenkins
> > and kogito-pipelines, so I started learning them.
> >
> > I wrote slides to illustrate relationships of jobs and jenkinsfile etc.
> > It's work-in-progress, but it may help the understanding. Please feel free
> > to validate and/or share your thoughts on this. Thanks!
> >
> >
> > https://docs.google.com/presentation/d/13IwLkQOOQcB4-9_uMgLFvGNKFkc682crsE9MDKvLa3M
> >
> > Feel free to use this thread to share such knowledge, so we can become more
> > comfortable in contributing to CI/CD.
> >
> > Cheers,
> > Toshiya
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@kie.apache.org
For additional commands, e-mail: dev-help@kie.apache.org
Re: [DISCUSS][architecture] knowledge around Jenkins and kogito-pipelines
Posted by Paolo Bizzarri <pi...@gmail.com>.
Hi Toshiya,
thank you a lot for the diagram, it is very clear and very well done.
I am myself learning the ropes here, and here are some pointers to code and
documentation that is good to use.
1) the code for the pipelines is in
https://github.com/apache/incubator-kie-kogito-pipelines/ and there is a
very nice initial readme, that seems reasonably up to date and detailed. If
you know nothing about the topic, I strongly suggest looking into this.
2) for the jenkins files, there is a good documentation in the directory
https://github.com/apache/incubator-kie-kogito-pipelines/tree/main/docs. it
gives a good overview of the various type of pipelines and of the jenkins
jobs. For what I can see, again, it is reasonably up to date. I suggest
starting from
https://github.com/apache/incubator-kie-kogito-pipelines/blob/main/docs/jenkins.md
3) kie kogito pipelines contains some general code for pipelines,
configurations and the likes. HOWEVER, each repository has some extra code
that you should check, which is under the .ci root directory. for example,
for drools you can check it here.
https://github.com/apache/incubator-kie-drools/tree/main/.ci
4) kie-tools apparently is a different story and I know little about it, so
all of this is just for the first three parts - drools, optaplanner, kogito.
Regards
Paolo
On Thu, Jul 11, 2024 at 10:19 AM Toshiya Kobayashi <
toshiyakobayashi@gmail.com> wrote:
> Hi,
>
> This thread is to share knowledge around Jenkins and kogito-pipelines.
>
> I'm Toshiya Kobayashi working on drools, not being familiar with Jenkins
> and kogito-pipelines, so I started learning them.
>
> I wrote slides to illustrate relationships of jobs and jenkinsfile etc.
> It's work-in-progress, but it may help the understanding. Please feel free
> to validate and/or share your thoughts on this. Thanks!
>
>
> https://docs.google.com/presentation/d/13IwLkQOOQcB4-9_uMgLFvGNKFkc682crsE9MDKvLa3M
>
> Feel free to use this thread to share such knowledge, so we can become more
> comfortable in contributing to CI/CD.
>
> Cheers,
> Toshiya
>