Status of Travis queuing

[Jarek Potiuk <Ja...@polidea.com>]

Hello everyone,

I wanted to provide some status of Travis queuing problems we experience
recently.

The queues in Travis are reaching fairly ridiculous level. Other Apache
projects are experiencing it as well. There is an open discussion about it
in the build apache list
https://lists.apache.org/thread.html/af52e2a3e865c01596d46374e8b294f2740587dbd59d85e132429b6c@%3Cbuilds.apache.org%3E
and
JIRA ticket opened: https://issues.apache.org/jira/browse/INFRA-18533

I think the changes I worked on for quite some time (namely AIP-10 - docker
image for CI, AIP-7 - simplified development workflow and recent
"pre-commit hooks" proposal) might be a short term way to decrease pressure
on Travis. I will focus on it now to pass the last rounds of reviews on
AIP-10 and speed up work on the other two.

If you have also other ideas how to speed up the builds in CI / decrease
number of jobs, they are more than welcome.

In the discussion in the build list I am also encouraging the other two
biggest projects of Apache (Fink and Arrow) to look for optimisations of
their builds - hopefully when we apply some coordinated effort we can
decrease the queuing time for everyone.

I also reached out to my close friend who is GitLab CI maintainer (and
added him to the discussion in the build list). He is an early employee of
GitLab CI and he will speak to CEO of GitLab and product managers of GitLab
CI and there is a high chance Apache Infra can reach an agreement with
GitLab to replace Travis CI as the "official" CI engine.

Plus Google promised to help with some credits for Google Cloud Builds.

The changes I implemented should make it rather easy to migrate to GitLab
CI or Google Cloud Build.

I hope we can solve this problem - both short and long term.

J.


-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>