Migrating a project from GitLab CI, setting up GitHub Actions

[Sander Striker <s....@striker.nl>]

Hi,

For anyone interested in following along, or providing any guidance or
help: we're looking to migrate BuildStream (see:
https://petri.apache.org/projects)  from GitLab to GitHub, including moving
from GitLab CI to GitHub Actions (https://s.apache.org/3tp4q
).

Cheers,

Sander

Re: Migrating a project from GitLab CI, setting up GitHub Actions

[Jarek Potiuk <Ja...@polidea.com>]

Hey Sander,

Happy to help.

Side comment: I love the way GitHub actions evolve and I have quite an
experience with running and optimizing our builds for Apache Airflow. We
are really happy with it especially comparing to Travis CI  we used
previously. We've been waiting > 1 year (!!!) for GitLab to add running CI
jobs for fork pull requests (
https://gitlab.com/gitlab-org/gitlab/-/issues/5667 - the issue is already 2
years old and keeps on being pushed to the next release with every release
so we stopped waiting for it.

The speed with which Github Actions adds new features is impressive, to say
the least, and their public roadmap
https://github.com/github/roadmap/projects/1 is even more impressive).

I am happy to provide help with my experience (I will subscribe to your
devlist now).

I am right now in the process of testing a big overhaul of the way how
Apache Airflow uses Github Actions.  Mostly it is about consistency and
optimizations of waiting time + decreasing the pressure on the Github
Actions workers we use (we have rather long and comprehensive tests and big
matrix of jobs). You can see some (almost completed) work in progress here
https://github.com/potiuk/airflow/tree/master/.github/workflows.

Along the way I am also finishing up a Github Action that is going to make
full use of recently released "workflow_run" type of triggers - that
enabled automated canceling of duplicated Pull Request builds from forks,
which was a major pain for us (and so far we only workaround it with
cancel run scheduled every 5 minutes - which was pretty nasty :(. With the
new "workflow_run" it seems possible, though a bit complex, that's why I
took an existing cancel-workflow action written by someone else and added
full support for workflow_runs (workflow_runs have been added ~ 1 week ago
or so so they are rather fresh). The action is here:
https://github.com/potiuk/cancel-workflow-runs#cancel-duplicate-jobs-for-triggered-workflow
(master
version is what I am testing right now and will release v2 shortly with
full support for workflow_run but the documentation is already rather
comprehensive and contains examples and explanation how things work

I am also helping the Apache Beam team in their Github Action migration, so
happy to help you as well :). I am also working on handling flaky tests
(and sharing experiences with BEam team as well). We've introduced concept
of "Quarantined tests" and I am looking into using
https://github.com/GoogleCloudPlatform/flaky-service - a service developed
by Google Cloud Platform to manage flaky tests (It has Gihub Action
integration out-of-the box).

You might also find useful the architecture of our integration between
Github Actions and DockerHub and the approach we've taken so far (I
contributed it to infra some time ago
https://cwiki.apache.org/confluence/display/INFRA/Github+Actions+to+DockerHub).
It will be slightly updated with the Overhaul I am working on, so I am
going to update it shortly, the in-progress version of it is in
https://github.com/potiuk/airflow/blob/master/CI.rst

J.



On Fri, Aug 14, 2020 at 12:00 PM Sander Striker <s....@striker.nl>
wrote:

> Hi,
>
> For anyone interested in following along, or providing any guidance or
> help: we're looking to migrate BuildStream (see:
> https://petri.apache.org/projects)  from GitLab to GitHub, including
> moving
> from GitLab CI to GitHub Actions (https://s.apache.org/3tp4q
> ).
>
> Cheers,
>
> Sander
>


-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>